Background: The purpose of this cross-sectional study was to describe the patterns and levels of sedentary time and physical activity (PA) in a general Japanese population. Methods: A total of 1,740 community-dwelling Japanese adults aged ≥40 years participated in this study. Sedentary time and PA were assessed for 7 consecutive days using a tri-axial accelerometer. Daily patterns and levels of sedentary time and PA were calculated by sex, age group (40–64, 65–74, and ≥75 years), and body mass index (BMI; <25 and ≥25 kg/m2). Results: Participants spent half of their waking time being sedentary, 32.7% of which was accumulated in prolonged bouts ≥30 minutes, versus only 54.4 minutes/day (7% of waking time) as moderate-to-vigorous PA (MVPA) (11.8 minutes/day in bouts ≥10 minutes). In addition to total sedentary time, men had longer prolonged sedentary bouts and fewer breaks per sedentary hour than women. Similar trends were observed in participants aged ≥75 years and those with a higher BMI (≥25 kg/m2) compared to those with a younger age and lower BMI. Moreover, participants aged ≥75 years and those with a higher BMI accumulated fewer MVPA minutes in bouts ≥10 minutes. Only 34.8% of the population met the recommended level of ≥150 minutes/week MVPA in bouts ≥10 minutes. Conclusion: Japanese adults accumulated a large proportion of total sedentary time in prolonged bouts but few minutes in sustained bouts of MVPA, and few of them met the current PA guideline.