Vegetable contamination in mining and smelting areas has resulted in high dietary intakes of heavy metals, which pose potential health risks to local residents. In this study, paired soil-vegetable samples were collected around Pb/Zn smelters in Southwest China. Probabilistic risks to local residents via vegetable consumption were evaluated with a Monte Carlo simulation. The mean concentrations of As, Cd, Cu, Pb, and Zn in the soils were 116.76, 3.59, 158.56, 196.96, and 236.74 mg/kg, respectively. About 38.18%, 58.49%, and 52.83% of the vegetable samples exceeded the maximum allowable concentrations for As, Cd, and Pb, respectively. The daily dietary intake of As, Cd, and Pb exceeded the provisional tolerable daily intakes for local residents, with children showing the highest intake via vegetable consumption. The percentages of the target hazard quotients of As, Cd, and Pb for local residents exceeding the safe value of one were about 95%, 50%, and 25%, respectively. The 95th percentiles of the hazard index for children, adolescents, and adults were 15.71, 11.15, and 9.34, respectively, indicating significant risks to local residents, especially children. These results highlight a need to develop effective strategies to reduce heavy metal contamination and exposure to protect human health.