Infertility is a global health concern, and couples are increasingly seeking medical assistance to achieve reproduction. Semen analysis is a primary assessment performed by a clinician, in which the morphology of the sperm population is evaluated. Machine learning algorithms that automate, standardize, and expedite sperm classification are the subject of ongoing research.We demonstrate a deep learning method to classify sperm into one of several World Health Organization (WHO) shape-based categories. Our method uses VGG16, a deep convolutional neural network (CNN) initially trained on ImageNet, a collection of human-annotated everyday images, which we retrain for sperm classification using two freely-available sperm head datasets (HuSHeM and SCIAN).Our deep learning approach classifies sperm at high accuracy and performs well in head-to-head comparisons with earlier approaches using identical datasets. We demonstrate improvement in true positive rate over a classifier approach based on a cascade ensemble of support vector machines (CE-SVM) and show similar true positive rates as compared to an adaptive patch-based dictionary learning (APDL) method. Retraining an off-the-shelf VGG16 network avoids excessive neural network computation or having to learn and use the massive dictionaries required for sparse representation, both of which can be computationally expensive.We show that our deep learning approach to sperm head classification represents a viable method to automate, standardize, and accelerate semen analysis. Our approach highlights the potential of artificial intelligence technologies to eventually exceed human experts in terms of accuracy, reliability, and throughput.