Both computations and experiments have confirmed that amides have stronger self-associations than imides. While this intriguing phenomenon is usually explained in the term of secondary electrostatic repulsion from the additional spectator carbonyl groups in imides, recently it was proposed that the π resonance effect from the spectator carbonyl which alters the balance between the acidity of the hydrogen-bond (H-bond) donor and the basicity of the H-bond acceptor is the major cause. In this work, we examined the roles of π resonance and the secondary electrostatic interaction in the formation of amide and imide dimers by deactivating the π conjugation from the spectator carbonyl and flipping the spectator carbonyl using the block-localized wave function method which is the simplest variant of valence bond theory. Energetic, geometrical, and spectral results show that three major forces, namely the σ induction effect (IE), π resonance effect (RE), and secondary electrostatic interaction (SEI), contribute to the different binding energies in the dimers of amides and imides. Whereas IE favors stronger binding among imides, both RE and SEI diminish the self-association of imides. Obviously, the negative force from RE and SEI exceeds the positive force from IE. Relatively, SEI plays a little bigger role than RE.