Metacognition, self-efficacy, and motivation are important components of interaction in self-regulated learning (SRL). However, the psychological mechanism underlying the association among them in mathematical learning remained ambiguous. The present study investigated whether the relationship between metacognitive knowledge (MK) and mathematics performance can be mediated by self-efficacy and motivation. The sample comprised 569 students (245 male, Mage = 16.39, = 0.63) of Grade 10 in China. The MK in mathematics questionnaire, the self-efficacy questionnaire, the academic motivation scale, Raven advanced progressive matrix, and mathematics tests were used for data collection. Our results suggested that the mathematics performance could be predicted by MK, self-efficacy and intrinsic motivation. Moreover, the association between MK and mathematics performance was mediated by self-efficacy and intrinsic motivation, as revealed by a multiple mediation analysis. Additionally, there were sex differences in MK, self-efficacy and intrinsic motivation. The findings highlight the psychological mechanism in the mathematics of Chinese students and will help teachers to improve students' mathematical learning in SRL framework more effectively. Implications for education and further studies are discussed.