We aimed to develop a deep learning model for the prediction of the risk of advanced colorectal cancer in Taiwanese adults. We collected data of 58152 patients from the Taiwan National Health Insurance database from 1999 to 2013. All patients' comorbidities and medications history were included in the development of the convolution neural network (CNN) model. We also used 3-year medical data of all patients before the diagnosed colorectal cancer (CRC) as the dimensional time in the model. The area under the receiver operating characteristic curve (AUC), sensitivity, and specificity were computed to measure the performance of the model. The results showed the mean (SD) of AUC of the model was 0.922 (0.004). Moreover, the performance of the model observed the sensitivity of 0.837, specificity of 0.867, and 0.532 for PPV value. Our study utilized CNN to develop a prediction model for CRC, based on non-image and multi-dimensional medical records.