Adiposity is an established risk factor for pediatric nonalcoholic fatty liver disease (NAFLD), but little is known about the influence of body composition patterns earlier in life on NAFLD risk. We aimed to examine associations of body composition at birth and body composition trajectories from birth to early childhood with hepatic fat in early childhood. Data were from the longitudinal Healthy Start Study in Colorado. Fat-free mass index (FFMI), fat mass index (FMI), percentage body fat (BF%), and BMI were assessed at birth and/or ∼5 y in >1200 children by air displacement plethysmography and anthropometrics. In a subset (n = 285), hepatic fat was also assessed at ∼5 y by MRI. We used a 2-stage modeling approach: first, we fit body composition trajectories from birth to early childhood using mixed models with participant-specific intercepts and linear slopes (i.e., individual deviations from the population average at birth and rate of change per year, respectively); second, associations of participant-specific trajectory deviations with hepatic fat were assessed by multivariable-adjusted linear regression. Participant-specific intercepts at birth for FFMI, FMI, BF%, and BMI were inversely associated with log-hepatic fat in early childhood in models adjusted for offspring demographics and maternal/prenatal variables [back-transformed β (95% CI) per 1 SD: 0.93 (0.88, 0.99), 0.94 (0.88, 0.99), 0.94 (0.89, 0.99), and 0.90 (0.85, 0.96), respectively]. Whereas, faster velocities for BF% and BMI from birth to ∼5 y were positively associated with log-hepatic fat [back-transformed β (95% CI) per 1 SD: 1.08 (1.01, 1.15) and 1.08 (1.02, 1.15), respectively]. These latter associations of BF% and BMI velocities with childhood hepatic fat were attenuated to the null when adjusted for participant-specific intercepts at birth. Our findings suggest that a smaller birth weight, combined with faster adiposity accretion in the first 5 y, predicts higher hepatic fat in early childhood. Strategies aiming to promote healthy body composition early in life may be critical for pediatric NAFLD prevention.This study was registered voluntarily at clinicaltrials.gov as NCT02273297.