Although a number of studies have been conducted to explore the effect of water quality improvement, the majority of them have focused mainly on point-of-use water treatment, and the studies investigating the effect of improved water supply have been based on observational or inadequately randomized trials. We report the results of a matched cluster randomized trial investigating the effect of improved water supply on diarrheal prevalence of children under five living in rural areas of the Volta Region in Ghana. We compared the diarrheal prevalence of 305 children in 10 communities of intervention with 302 children in 10 matched communities with no intervention (October 2012 to February 2014). A modified Poisson regression was used to estimate the prevalence ratio. An intention-to-treat analysis was undertaken. The crude prevalence ratio of diarrhea in the intervention compared with the control communities was 0.85 (95% CI 0.74–0.97) for Krachi West, 0.96 (0.87–1.05) for Krachi East, and 0.91 (0.83–0.98) for both districts. Sanitation was adjusted for in the model to remove the bias due to residual imbalance since it was not balanced even after randomization. The adjusted prevalence ratio was 0.82 (95% CI 0.71–0.96) for Krachi West, 0.95 (0.86–1.04) for Krachi East, and 0.89 (0.82–0.97) for both districts. This study provides a basis for a better approach to water quality interventions.