This article is part of a Special Issue "Puberty and Adolescence". Sexual differentiation is the process by which the nervous system becomes structurally and functionally dissimilar in females and males. In mammals, this process has been thought to occur during prenatal and early postnatal development, when a transient increase in testosterone secretion masculinizes and defeminizes the developing male nervous system. Decades of research have led to the views that structural sexual dimorphisms created during perinatal development are passively maintained throughout life, and that ovarian hormones do not play an active role in feminization of the nervous system. Furthermore, perinatal testosterone was thought to determine sex differences in neuron number by regulating cell death and cell survival, and not by regulating cell proliferation. As investigations of neural development during adolescence became more prominent in the late 20th century and revealed the extent of brain remodeling during this time, each of these tenets has been challenged and modified. Here we review evidence from the animal literature that 1) the brain is further sexually differentiated during puberty and adolescence; 2) ovarian hormones play an active role in the feminization of the brain during puberty; and 3) hormonally modulated, sex-specific addition of new neurons and glial cells, as well as loss of neurons, contribute to sexual differentiation of hypothalamic, limbic, and cortical regions during adolescence. This architectural remodeling during the adolescent phase of sexual differentiation of the brain may underlie the known sex differences in vulnerability to addiction and psychiatric disorders that emerge during this developmental period.