We aimed to identify reorganization processes of episodic memory networks in patients with left and right temporal lobe epilepsy (TLE) due to hippocampal sclerosis as well as their relations to neuropsychological memory performance. We investigated 28 healthy subjects, 12 patients with left TLE (LTLE) and 9 patients with right TLE (RTLE) with hippocampal sclerosis by means of functional magnetic resonance imaging (fMRI) using a face-name association task, which combines verbal and non-verbal memory functions. Regions-of-interest (ROIs) were defined based on the group results of the healthy subjects. In each ROI, fMRI activations were compared across groups and correlated with verbal and non-verbal memory scores. The face-name association task yielded activations in bilateral hippocampus (HC), left inferior frontal gyrus (IFG), left superior frontal gyrus (SFG), left superior temporal gyrus, bilateral angular gyrus (AG), bilateral medial prefrontal cortex and right anterior temporal lobe (ATL). LTLE patients demonstrated significantly less activation in the left HC and left SFG, whereas RTLE patients showed significantly less activation in the HC bilaterally, the left SFG and right AG. Verbal memory scores correlated with activations in the left and right HC, left SFG and right ATL and non-verbal memory scores with fMRI activations in the left and right HC and left SFG. The face-name association task can be employed to examine functional alterations of hippocampal activation during encoding of both verbal and non-verbal material in one fMRI paradigm. Further, the left SFG seems to be a convergence region for encoding of verbal and non-verbal material.