To investigate the progressive failure process of coal, a series of uniaxial and triaxial compression tests were conducted and a novel 3D digital image correlation instrument with six cameras combined with a special transparent pressure cell was used for the strain measurement. The stress thresholds of coal were obtained in uniaxial and triaxial compression. The energy evolution during the compression was discussed, coupled with the crack volumetric strain. The field strain of the whole specimen surface and crack propagation at different stress levels were described to study the progressive failure mechanism of coal. The average stress level of crack initiation and crack damage of coal in uniaxial compression are 43.75% and 63.03%, while that in the triaxial compression are 74.53% and 89.84%, respectively. The dissipation energy evolution corresponds to the crack volumetric strain, while the elastic energy release leads to flake ejection and coal failure. The crack evolution and localization of coal indicated the progressive failure process that the coal sample undergoes in tension failure in uniaxial compression and in tension-shear failure in triaxial compression. The findings of this study can serve as a reference to understand the failure process of coal and improve the stability and safety of mining engineering.