It is known that overweight is associated with chronic knee pain (CKP) and chronic low back pain (CLBP). Several risk factors for these conditions have been postulated, including age, sex, overweight, occupation, and socioeconomic factors. In addition, physical environment has been studied as a potential risk factor in recent years. However, the associations of overweight and physical environment with CKP and CLBP remains unclear. The aim of this study conducted in a rural mountainous region was to examine whether overweight individuals living at higher elevations have an increased probability of experiencing CKP and CLBP. In 2009, we conducted a mail survey with a random sample aged between 40 to 79 years. Questionnaires were sent to 6,000 individuals and a total of 4,559 individuals responded to this survey. After excluding the respondents with missing data, we conducted a logistic regression analysis of the data for 3,109 individuals. There was statistically significantly higher adjusted odds ratio (aOR) of CKP for those who were overweight living at low elevation (aOR = 1.90, 95% CI = 1.21–2.98), moderate elevation (aOR = 1.73, 95% CI = 1.05–2.87), and high elevation (aOR = 2.13, 95% CI = 1.31–3.46) than those who were not overweight living at low elevation. However, similar patterns were not observed for CLBP. Our results show that specific overweight–elevation associations were observed for CKP in a rural mountainous region.