It's 1954 Baltimore and brothers Van (ADRIEN BRODY) and Ben Kurtzman (BEN FOSTER) have grown up near the predominantly Jewish neighbor of Liberty Heights. Just until recently, and thanks to their parents, Nate (JOE MANTEGNA) and Ada (BEBE NEUWIRTH) and grandmother Rose (FRANIA RUBINEK), they've thought that most everyone in the world was Jewish.
Yet this was a time of anti-Semitism, racism and distinct class differences, where attending a party in a Gentile neighborhood might mean ending up in a fight, or being attracted to a person of another race brings outrage from the involved parents.
That's exactly what happens with the Kurtzman brothers. The older one, Van, attends a Halloween party with his buddies Yussel (DAVID KRUMHOLTZ) and Alan (KEVIN SUSSMAN) where the former gets into just such a anti-Semitic fight while Van falls for Dubbie (CAROLYN MURPHY), a pretty but somewhat snobbish young woman. Of course he doesn't know that she's already seeing Trey (JUSTIN CHAMBERS), a dashing young man with a penchant for dangerous behavior including drinking and driving.
Meanwhile, Ben finds himself falling for Sylvia (REBEKAH JOHNSON), the first black student integrated into his school. While he anticipates the problems such a friendship and potential relationship can generate, especially with her father, he finds himself learning a thing or two about being black in the 1950s.
Meanwhile, the boys' dad, Nate, runs a burlesque business with his partners, Louie (CHARLEY SCALIES), Charlie (RICHARD KLINE) and Pete (VINCENT GUASTAFERRO), but they've discovered that they can make more money from their neighborhood numbers game.
Things go well until they add a bonus number and Little Melvin (ORLANDO JONES), a small-time, black drug dealer hits the numbers with a huge bet, thus winning a $100,000 payoff. Not having the money to pay him off, Nate and his partners try to figure out to solve their dilemma, and their solution eventually involves Ben and the rest of the family in ways they never expected.