In the context of population ageing in many developed and developing countries, encouraging active transport behaviors of older adults, is a key public health priority. However, many cross-sectional studies assessing the impact of built environment characteristics on travel behavior fail to address residential self-selection bias, and hence the causal relationship is uncertain. A large-scale public housing scheme provided this study with a unique research opportunity to distinguish residential self-selection from the effects of built environment characteristics on the travel behaviors of older adults (N = 13,468 and 3,961 in two analyses respectively) in Hong Kong, because public housing residents have little freedom to choose their residential locations. The results showed that the elderly living in public housing estates generally have fewer trips, shorter overall travel times and distances, and fewer motorized trips including those by rail or private car than those living in private housing estates. In addition, the results for walking, walking times, numbers of trips, and travel distance for elderly people in public and private housing all exhibited markedly different associations with built environment characteristics. Strength of built environment-travel behavior associations dropped by approximately 30-50% after controlling for the effect of residential self-selection. The results indicate that both built environment characteristics and residential self-selection affect travel behaviors.