Real-time recognition of risky driving behavior and aggressive drivers is a promising research domain, thanks to powerful machine learning algorithms and the big data provided by in-vehicle and roadside sensors. However, since the occurrence of aggressive drivers in real traffic is infrequent, most machine learning algorithms treat each sample equally and prone to better predict normal drivers rather than aggressive drivers, which is our real interest. This paper aims to test the advantage of imbalanced class boosting algorithms in aggressive driver recognition using vehicle trajectory data. First, a surrogate measurement of collision risk, called Average Crash Risk (ACR), is proposed to calculate a vehicle's crash risk. Second, the driver's driving aggressiveness is determined by his/her ACR with three anomaly detection methods. Third, we train classification models to identify aggressive drivers using partial trajectory data. Three imbalanced class boosting algorithms, SMOTEBoost, RUSBoost, and CUSBoost, are compared with cost-sensitive AdaBoost and cost-sensitive XGBoost. Additionally, we try two resampling techniques with AdaBoost and XGBoost. Among all algorithms tested, CUSBoost achieves the highest or the second-highest Area Under Precision-Recall Curve (AUPRC) in most datasets. We find the discrete Fourier coefficients of gap as the key feature to identify aggressive drivers.