Despite the benefits of youth sports, most children drop out by the age of 13 years. A better understanding of the etiology of sports dropout in children will serve to improve interventions to keep children active. The objective of this review was to investigate the associations between body image and perceived physical competence and sports attrition in children. A systematic literature review was conducted using PubMed and MEDLINE database searches in compliance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). Included studies were full-text English articles that addressed body image or perceived competence and attrition or dropout with subjects 21 years and younger. The results were organized by age to demonstrate the changing relationship body image has on physical activity. Evidence revealed that children younger than 7 years have an inflated self-perception and are eager to participate in activities regardless of competence. Between 7 to 10 years of age, children begin to more accurately perceive their skills and draw comparisons with their peers. Elevated body mass index (BMI) becomes a deterrent between 6 to 11 years. After 12 years, teasing and gender identification issues become causes of attrition. In adolescence, body image becomes a significant determinant of continuation of physical activity, more than actual skill. Perceived physical competency and body image do not appear to affect sports attrition in children younger than 7 years. As children get older, BMI/body image and physical competency become greater factors in sports attrition, with body image playing a significant role in adolescents.