Newborns and infants communicate their needs and physiological states through crying and emotional facial expressions. Little is known about individual differences in responding to infant crying. Several theories suggest that people vary in their environmental sensitivity with some responding generally more and some generally less to environmental stimuli. Such differences in environmental sensitivity have been associated with personality traits, including neuroticism. This study investigated whether neuroticism impacts neuronal, physiological, and emotional responses to infant crying by investigating blood-oxygenation-level dependent (BOLD) responses using functional magnetic resonance imaging (fMRI) in a large sample of healthy women (N = 102) with simultaneous skin conductance recordings. Participants were repeatedly exposed to a video clip that showed crying infants and emotional responses (valence, arousal, and irritation) were assessed after every video clip presentation. Increased BOLD signal during the perception of crying infants was found in brain regions that are associated with emotional responding, the amygdala and anterior insula. Significant BOLD signal decrements (i.e., habituation) were found in the fusiform gyrus, middle temporal gyrus, superior temporal gyrus, Broca's homologue on the right hemisphere, (laterobasal) amygdala, and hippocampus. Individuals with high neuroticism showed stronger activation in the amygdala and subgenual anterior cingulate cortex (sgACC) when exposed to infant crying compared to individuals with low neuroticism. In contrast to our prediction we found no evidence that neuroticism impacts fMRI-based measures of habituation. Individuals with high neuroticism showed elevated skin conductance responses, experienced more irritation, and perceived infant crying as more unpleasant. The results support the hypothesis that individuals high in neuroticism are more emotionally responsive, experience more negative emotions, and may show enhanced cognitive control during the exposure to infant distress, which may impact infant-directed behavior.