It's Hong Kong in 1962, and two young couples are moving into the same apartment building on the same day. Chow Mo-wan (TONY LEUNG CHIU-WAI) is a journalist who'd rather be writing martial arts serials, and his wife, a hotel receptionist, is working and can't help him move in. Mrs. Chan (MAGGIE CHEUNG MAN-YUK), who works as a secretary to Mr. Ho (LAI CHIN) at a shipping company, moves into Mrs. Suen's (REBECCA PAN) place, but her husband is reportedly out of town on a business trip and thus can't help her.
As the two get comfortable in their respective apartments, Mr. Chow and Mrs. Chan occasionally interact, with each noting that the other's spouse never seems to be around. While they may suspect that something suspicious is going on, the two always end up making excuses for the whereabouts and behavior of their partners.
That changes when Chow's friend, Ah Ping (SIU PING-IAM), informs him that he saw his wife with another man, and after Mr. Chow and Mrs. Chan have dinner and come to the realization that their spouses are having an affair with each other. The two then become closer friends and as they discuss the various aspects of the unseen affair, they start to fall for each other. From that point on, they must figure out how to handle their feelings for one another.