In this study, a novel analytical approach is proposed for the identification of pesticide residues in tea by combining surface-enhanced Raman scattering (SERS) with a deep learning method one-dimensional convolutional neural network (1D CNN). First, a handheld Raman spectrometer was used for rapid on-site collection of SERS spectra. Second, the collected SERS spectra were augmented by a data augmentation strategy. Third, based on the augmented SERS spectra, the 1D CNN models were established on the cloud server, and then the trained 1D CNN models were used for subsequent pesticide residue identification analysis. In addition, to investigate the identification performance of the 1D CNN method, four conventional identification methods, including partial least square-discriminant analysis (PLS-DA), k-nearest neighbour (k-NN), support vector machine (SVM) and random forest (RF), were also developed on the basis of the augmented SERS spectra and applied for pesticide residue identification analysis. The comparative studies show that the 1D CNN method possesses better identification accuracy, stability and sensitivity than the other four conventional identification methods. In conclusion, the proposed novel analytical approach that exploits the advantages of SERS and a deep learning method (1D CNN) is a promising method for rapid on-site identification of pesticide residues in tea.