Hyperspectral imaging, with the hundreds of bands and high spectral resolution, offers a promising approach for estimation of heavy metal concentration in agricultural soils. Using airborne imagery over a large-scale area for fast retrieval is of great importance for environmental monitoring and further decision support. However, few studies have focused on the estimation of soil heavy metal concentration by airborne hyperspectral imaging. In this study, we utilized the airborne hyperspectral data in LiuXin Mine of China obtained from HySpex VNIR-1600 and HySpex SWIR-384 sensor to establish the spectral-analysis-based model for retrieval of heavy metals concentration. Firstly, sixty soil samples were collected in situ, and their heavy metal concentrations (Cr, Cu, Pb) were determined by inductively coupled plasma-mass spectrometry analysis. Due to mixed pixels widespread in airborne hyperspectral images, spectral unmixing was conducted to obtain purer spectra of the soil and to improve the estimation accuracy. Ten of estimated models, including four different random forest models (RF)-standard random forest (SRF), regularized random forest (RRF), guided random forest (GRF), and guided regularized random forest (GRRF)-were introduced for hyperspectral estimated model in this paper. Compared with the estimation results, the best accuracy for Cr, Cu, and Pb is obtained by RF. It shows that RF can predict the three heavy metals better than other models in this area. For Cr, Cu, Pb, the best model of RF yields R values of 0.75,0.68 and 0.74 respectively, and the values of RMSE are 5.62, 8.24, and 2.81 (mg/kg), respectively. The experiments show the average estimated values are close to the truth condition and the high estimated values concentrated near several industries, valifating the effectiveness of the presented method.