Developing more reliable predictors of seizure outcome following temporal lobe surgery for intractable epilepsy is an important clinical goal. In this context, we investigated patients with refractory temporal lobe epilepsy (TLE) before and after temporal resection. In detail, we explored gray matter (GM) volume change in relation with seizure outcome, using a voxel-based morphometry (VBM) approach. To do so, this study was divided into two parts. The first one involved group analysis of differences in regional GM volume between the groups (good outcome (GO), e.g., no seizures after surgery; poor outcome (PO), e.g., persistent postoperative seizures; and controls, N = 24 in each group), pre- and post-surgery. The second part of the study focused on pre-surgical data only (N = 61), determining whether the degree of GM abnormalities can predict surgical outcomes. For this second step, GM abnormalities were identified, within each lobe, in each patient when compared with an ad hoc sample of age-matched controls. For the first analysis, the results showed larger GM atrophy, mostly in the frontal lobe, in PO patients, relative to both GO patients and controls, pre-surgery. When comparing pre-to-post changes, we found relative GM gains in the GO but not in the PO patients, mostly in the non-resected hemisphere. For the second analysis, only the frontal lobe displayed reliable prediction of seizure outcome. 81% of the patients showing pre-surgical increased GM volume in the frontal lobe became seizure free, post-surgery; while 77% of the patients with pre-surgical reduced frontal GM volume had refractory seizures, post-surgery. A regression analysis revealed that the proportion of voxels with reduced frontal GM volume was a significant predictor of seizure outcome (p = 0.014). Importantly, having less than 1% of the frontal voxels with GM atrophy increased the likelihood of being seizure-free, post-surgery, by seven times. Overall, our results suggest that using pre-surgical GM abnormalities within the frontal lobe is a reliable predictor of seizure outcome post-surgery in TLE. We believe that this frontal GM atrophy captures seizure burden outside the pre-existing ictal temporal lobe, reflecting either the development of epileptogenesis or the loss of a protective, adaptive force helping to control or limit seizures. This study provides evidence of the potential of VBM-based approaches to predict surgical outcomes in refractory TLE candidates.