Seagrass meadows are vulnerable to fine sediment (mud) pollution, with impacts usually attributed to reduction in submerged light. Here we tested two non-exclusive hypotheses, that mud particles (<63 µm) impact seagrasses through both (1) the light climate and (2) changes in substrate physico-chemistry. We tested these hypotheses in Pāuatahanui Inlet, New Zealand, by comparing seagrass presence, abundance, and health, together with light climate and substrate physico-chemistry at contrasting habitats where (1) seagrass used to thrive but no longer grows (historical seagrass), (2) seagrass still persists (existing seagrass) and (3) seagrass has been present recently, but not currently (potential seagrass). Historical seagrass substrate had significantly higher mud (35% average), bulk density (1.5 g cm−3), porewater ammonium concentration (65 µM), and a more reduced redox profile (negative redox at only 2 cm soil depth) as well as a lower light availability when submerged compared to other habitats, while total daily light exposure differed little between habitats. This suggests that failure of seagrass to recolonize historical seagrass habitat reflects substrate muddiness and consequent unfavorable rhizosphere conditions. Our results provide evidence for the multi-stressor effects of fine sediment on seagrasses, with substrate suitability for seagrass being detrimentally affected even where light exposure seems sufficient.