It is highly possible that future wireless communication systems will adopt ultra-dense deployment to cope with the increasing demand on spectrum efficiency and energy efficiency. The pivotal issue to achieve the potential benefits of the ultra-dense network is to deal with the complex inter-site interference. In this paper, in order to maximize the spectrum efficiency of the system, we first make a reasonable approximation on the inter-site interference to convert the problem into a convex optimization problem. Then, the Lagrangian Multiplier method is adopted to obtain the expression of the optimum power allocation, and the water filling algorithm, as one of the most classical algorithms in the information theory, can be applied to maximize the sum rate or spectrum efficiency of the system. Since the classical iteratively searching water filling algorithm needs many iterations to converge to the optimal solution, we develop a low-complexity iterative approximate water filling algorithm. Simulation results show that the developed algorithm can achieve very close performance to the classical iteratively searching water filling based power allocation with only a few iterations under different scenarios, which leads to a significant complexity reduction.