Recently the American Society for Gastrointestinal Endoscopy addressed the 'resect and discard' strategy, determining that accurate differentiation of colorectal polyps (CP) is necessary. Previous studies have suggested a promising application of artificial intelligence (AI), using deep learning in object recognition. Therefore, we aimed to construct an AI system that can accurately detect and classify CP using stored still images during colonoscopy.We used a deep convolutional neural network (CNN) architecture called Single Shot MultiBox Detector. We trained the CNN using 16,418 images from 4752 CPs and 4013 images of normal colorectums, and subsequently validated the performance of the trained CNN in 7077 colonoscopy images, including 1172 CP images from 309 various types of CP. Diagnostic speed and yields for the detection and classification of CP were evaluated as a measure of performance of the trained CNN.The processing time of the CNN was 20 ms per frame. The trained CNN detected 1246 CP with a sensitivity of 92% and a positive predictive value (PPV) of 86%. The sensitivity and PPV were 90% and 83%, respectively, for the white light images, and 97% and 98% for the narrow band images. Among the correctly detected polyps, 83% of the CP were accurately classified through images. Furthermore, 97% of adenomas were precisely identified under the white light imaging.Our CNN showed promise in being able to detect and classify CP through endoscopic images, highlighting its high potential for future application as an AI-based CP diagnosis support system for colonoscopy.