Under the Grain for Green Project in China, vegetation recovery construction has been widely implemented on the Loess Plateau for the purpose of soil and water conservation. Now it is becoming controversial whether the recovery construction involving vegetation, particularly forest, is reducing the streamflow in the rivers of the Yellow River basin. In this study, we chose the Wei River, the largest branch of the Yellow River, with revegetated construction area as the study area. To do that, we apply the widely used Soil and Water Assessment Tool (SWAT) model for the upper and middle reaches of the Wei River basin. The SWAT model was forced with daily observed meteorological forcings (1960–2009) calibrated against daily streamflow for 1960–1969, validated for the period of 1970–1979, and used for analysis for 1980–2009. To investigate the impact of LUCC (land use and land cover change) on the streamflow, we firstly use two observed land use maps from 1980 and 2005 that are based on national land survey statistics merged with satellite observations. We found that the mean streamflow generated by using the 2005 land use map decreased in comparison with that using the 1980 one, with the same meteorological forcings. Of particular interest here is that the streamflow decreased on agricultural land but increased in forest areas. More specifically, the surface runoff, soil flow, and baseflow all decreased on agricultural land, while the soil flow and baseflow of forest areas increased. To investigate that, we then designed five scenarios: (S1) the present land use (1980) and (S2) 10 %, (S3) 20 %, (S4) 40 %, and (S5) 100 % of agricultural land that was converted into mixed forest. We found that the streamflow consistently increased with agricultural land converted into forest by about 7.4 mm per 10 %. Our modeling results suggest that forest recovery construction has a positive impact on both soil flow and baseflow by compensating for reduced surface runoff, which leads to a slight increase in the streamflow in the Wei River with the mixed landscapes on the Loess Plateau that include earth–rock mountain area.