Using the electric field to manipulate the magnetization of materials is a potential way of making low-power-consumption nonvolatile magnetic memory devices. Despite concentrated effort in the last 15 years on magnetic multilayers and magnetoelectric multiferroic thin films, there has been no report on the reversal of out-of-plane magnetization by an electric field at room temperature without the aid of an electric current. Here, we report direct observation of out-of-plane magnetization reversal at room temperature by magnetic force microscopy after electric polarization switching of cobalt-substituted bismuth ferrite thin film grown on (110)-oriented GdScO substrate. A striped pattern of ferroelectric and weakly ferromagnetic domains was preserved after reversal of the out-of-plane electric polarization.