Removal of phosphorus (P) from municipal wastewater is of vital importance to the control of eutrophication in receiving freshwater bodies. Typical cations such as Ca and Mg generally exist in municipal wastewater, and they may affect the sorption behavior and mechanism of iron oxide-based materials for aqueous phosphate (HPO, x = 0, 1, 2, or 3 depending on solution pH). To better apply iron oxide-containing materials as adsorbents to eliminate HPO in municipal wastewater, a hydrous ferric oxide (HFEO) was prepared and characterized at first and then the impact of coexisting Ca and Mg on the uptake of HPO by HFEO was studied. The results showed that, without coexisting Ca and Mg, the kinetic data for HPO sorption onto HFEO were better described by the Elovich model (R = 0.953) than the pseudo-second-order (R = 0.838) and pseudo-first-order (R = 0.641) models, and the isotherm data were fitted better with the Dubinin-Radushkevich (R = 0.966) and Freundlich (R = 0.953) models than with the Langmuir (R = 0.924) model. The ligand exchange of the Fe-bound hydroxyl group with HPO and the generation of Fe-O-P bonding played a key role in the uptake of HPO by HFEO in the absence of Ca and Mg. Coexisting Ca and Mg greatly improved the adsorptive removal of HPO by HFEO, including the adsorption capacity and initial adsorption rate. According to the Langmuir isotherm equation, the predicted maximum HPO adsorption capacity for HFEO at pH 7 in the presence of 2 mmol/L Ca (24.7 mg P/g) or 2 mmol/L Mg (18.4 mg P/g) was much larger than that without coexisting Ca and Mg (10.7 mg P/g). The formation of aqueous CaHPO and MgHPO species firstly and then the adsorption of the formed CaHPO and MgHPO species on the HFEO surface to generate the HPO-bridged ternary complexes (i.e., Fe(OPOH)Ca and Fe(OPOH)Mg) had an important role in the improvement of HPO adsorption onto HFEO by coexisting Ca and Mg.