The mechanisms responsible for the propensity to gain weight or remain normal weight are poorly understood. The objective of this study was to study the neuronal response to visual food cues during short-term energy imbalance in healthy adults recruited as obesity-resistant (OR) or obesity-prone (OP) based on self-identification, BMI, and personal/family weight history. 25 OR and 28 OP subjects were studied in underfed (UF) and overfed (OF) as compared to eucaloric (EU) conditions in a randomized crossover design. Each study phase included a 3 day run-in diet, 1 day of controlled feeding (basal energy needs for EU, 40% above/below basal energy needs for OF/UF), and a test day. On the test day fMRI was performed in the acute fed stated (30 minutes after a test meal) while subjects viewed images of foods of high hedonic value and neutral non-food objects. Measures of appetite and hormones were also performed before and every 30 minutes after the test meal. UF was associated with significantly increased activation of insula, somatosensory cortex, inferior and medial prefrontal cortex, parahippocampus, precuneus, cingulate and visual cortex in OR. However, UF had no impact in OP. As a result, UF was associated with significantly greater activation, specifically in the insula, inferior prefrontal cortex, and somatosensory cortex in OR as compared to OP. While OF was overall associated with reduced activation of inferior visual cortex, no group interaction was observed with OF. In summary, these findings suggest that individuals resistant to weight gain and obesity are more sensitive to short-term energy imbalance, particularly with UF, than those prone to weight gain. The inability to sense or adapt to changes in energy balance may represent an important mechanism contributing to excess energy intake and risk for obesity.