Background: Previous studies have proposed various physical tests for screening fall risk in older adults. However, older adults may have physical or cognitive impairments that make testing difficult. This study describes the differences in individual, physical, and psychological factors between adults in good and poor self-rated health statuses. Further, we identified the physical or psychological factors associated with self-rated health by controlling for individual variables. Methods: Data from a total of 1577 adults aged 65 years or over with a history of falls were analyzed, using the 2017 National Survey of Older Persons in South Korea. Self-reported health status was dichotomized as good versus poor using the 5-point Likert question: “poor” (very poor and poor) and “good” (fair, good, and very good). Results: Visual/hearing impairments, ADL/IADL restriction, poor nutrition, and depression were more frequently observed in the group with poor self-rated health. Multivariable logistic regression revealed that poor self-reported health was significantly associated with hearing impairments (OR: 1.51, 95% CI 1.12–2.03), ADL limitation (OR: 1.77, 95% CI 1.11–2.81), IADL limitation (OR: 2.27, 95% CI 1.68–3.06), poor nutrition (OR: 1.36, 95% CI 1.05–1.77), and depression (OR 3.77, 95% CI 2.81–5.06). Conclusions: Auditory impairment, ADL/IADL limitations, poor nutrition, and depression were significantly associated with poor self-reported health. A self-rated health assessment could be an alternative tool for older adults who are not able to perform physical tests.