Most city agglomerations of developing countries face water shortages and pollution due to population growth and industrial aggregation. To meet such water security challenges, policy makers need to evaluate water use efficiency at the regional or basin level because the prosperity of city agglomerations is indispensable to the sustainable development of the region or basin. To solve the issue, this paper adopts a non-directional distance function within the framework of environmental production technology to measure water use efficiency. Based on the distance between actual water use efficiency and the ideal efficiency, it calculates the potential reduction space of water input and pollutants by slack adjustment. Added to the Malmquist index, it forms a non-radial Malmquist water use performance index, which can be divided into technological change and technical efficiency change, to measure dynamic water use efficiency. Further, water use efficiency change is analyzed from the perspectives of technological improvement and institutional construction. Bohai Bay city agglomeration, a typical water-deficient city agglomeration in China, is taken as a case study, and data on water resource, environment, and economy from 2011 to 2014 have been used. In conclusion, there is much space for water use efficiency improvement on the whole. However, even having considered potential reduction space of water input and pollutant discharge under current environmental production technology, it is still not enough to support the city agglomeration’s sustainable development. To relieve current potential water safety hazards, not only technical improvement but also institution innovation for highly efficient water use should be kept accelerating in Bohai Bay region. In terms of urban water management in developing countries, the research conclusion is of theoretical and practical significance.