Housing finance market in Uganda is highly segmented. At the upper end it is rare for more than 1 percent of housing construction to be financed through the formal mortgage banking system. At the other end, microfinance clients may have access to loans that can be used to improve their housing. It is in the large gap between these two segments that development of the mortgage market can help. The very limited availability of long-term mortgage finance leaves the majority of the population in the unsatisfactory situation of having to improve their housing incrementally based on a build-as-you-earn basis with the inefficiencies and deferrals that entails. The paper which is based on literature review, interviews and questionnaires discusses why the majority of the population does not apply for mortgages and why people not being accepted by the bank for mortgages. This paper therefore seeks to address the question on how to increase the outreach of the mortgage market. This is based on the legal and financial barriers, both on the supply and demand side.