Empathy is the capacity to understand and experience the feeling state of others. While individuals attribute negative empathic responses to their own feelings, they would endure personal distress that can be harmful to social interaction. However, the neural mechanism of personal distress remains unclear. Here, we examined the neural substrates of personal distress by combining structural (Voxel-based morphometry (VBM)) and functional (resting-state functional connectivity (FC) analysis) MRI approaches in 53 college students (aged 19–26). A negative correlation was found between a trait measure of personal distress and gray matter (GM) volume in the dorsal medial prefrontal cortex (dmPFC). FC analyses with the dmPFC as a seed further revealed that the connectivity between the dmPFC and posterior insula was positively correlated with the personal distress, and the connectivities between the dmPFC and the anterior middle cingulate cortex, left lateral frontal cortex, and left inferior parietal gyrus were negatively correlated with the personal distress. Our results suggested that personal distress is underlain by neural substrates associated with both cognitive and affective mechanisms. Taken together, the structural and functional correlates of personal distress revealed in the present findings shed new light into the understanding of empathy.