Understanding of the influencing factors of industrial sector carbon dioxide emissions is essential to reduce natural and anthropogenic greenhouse gas emissions. In this paper, we applied the Logarithmic Mean Divisia Index (LMDI) decomposition method based on the extended Kaya identity to analyze the changes in industrial carbon dioxide emissions resulting from 39 industrial sectors in Inner Mongolia northeast of China over the period 2003–2012. The factors were divided into five types of effects i.e., industrial growth effect, industrial structure effect, energy effect, energy intensity effect, population effect and comparative analysis of differential influences of various factors on industrial sector. Our results clearly show that (1) Industrial sector carbon dioxide emissions have increased from 134.00 million ton in 2003 to 513.46 million ton in 2012, with an annual average growth rate of 16.097%. The industrial carbon dioxide emissions intensity has decreased from 0.99 million ton/billion yuan to 0.28 million ton/billion yuan. Also, the energy structure has been dominated by coal; (2) Production and supply of electric power, steam and hot water, coal mining and dressing, smelting and pressing of ferrous metals, petroleum processing, coking and nuclear fuel processing, and raw chemical materials and chemical products account for 89.74% of total increased industrial carbon dioxide emissions; (3) The industrial growth effect and population effect are found to be a critical driving force for increasing industrial sector carbon dioxide emissions over the research period. The energy intensity effect is the crucial drivers of the decrease of carbon dioxide emissions. However, the energy structure effect and industrial structure effect have considerably varied over the study years without displaying any clear trend.