This study develops a novel automatic all-sky imaging system, namely, an all-sky camera (ASC) system, for cloud cover assessment. The proposed system does not require conventional solar occulting devices and can capture complete hemispheric sky images. Cloud detection is performed innovatively using a convolutional neural network model (i.e., the optimized U-Net model). Experiments demonstrate that the optimized U-Net model can effectively detect clouds from sky images. In terms of cloud cover, the estimation results of the ASC system exhibit a high correlation with those obtained via manual observation, thereby indicating the applicability of the ASC system in ground-based cloud observation and analysis.