Are mathematical learning difficulties caused by impairment of the abilities that underlie mathematical talent? Or are mathematical difficulties and talent qualitatively different? The main goal of this study was to determine whether mathematical learning difficulties are explained by the same executive functions as mathematical talent. We screened a pool of 2,682 first-year high school students and selected 48 for evaluation, dividing them into three groups: those with mathematical learning difficulties (n = 16), those with typical performance (n = 16), and those with mathematical talent (n = 16). Adolescents from the learning difficulties and talented groups had age, reading skills, and verbal and non-verbal intelligence that were similar to those of the typical performance group. Participants were administered a suite of tasks to evaluate verbal and visual short-term memory and executive functions of inhibition, shifting, and updating. Different executive functions showed different contributions at the two ends of the math ability continuum: lower levels of performance in updating visual information were related to mathematical learning difficulties, while greater shifting abilities were related to mathematical talent. Effect sizes for the differences in performance between groups were large (Hedges' g > 0.8). These results suggest that different executive functions are associated with mathematical learning difficulties and mathematical talent. We discuss how these differences in executive functions could be related to the different types of mathematical abilities that distinguish the three groups.