Physical inactivity is a global pandemic with no signs of improvement. Prolonged sitting time is an emerging risk factor that exacerbates the health consequences of physical inactivity. Both behaviours are influenced by various individual and environmental factors but it remains unknown whether early-life exposures "program" these behaviours in later life. The current evidence is limited by a small number of studies which were primarily conducted in high-income countries, and a narrow range of early-life variables examined. Using data from three population-based Brazilian birth cohorts (analytical samples: n = 2740 for 1982 cohort, aged 30 years; n = 3592 for 1993 cohort, aged 18; n = 2603 for 2004 cohort, aged 6), we show that being female and higher family socioeconomic status at birth are strong and consistent predictors of lower physical activity and higher sedentary time from childhood to adulthood. Meanwhile, higher birth weight and lower birth order may also predict lower physical activity and higher sedentary time. Our findings are distinct from evidence from high-income countries, suggesting the importance of broader socioeconomic context in determining individual's activity patterns through the life- course. Such evidence is essential for understanding the biological etiology and socioeconomic context of physical activity and sedentary behaviour at an early stage in life.