Substance use by peers and family may affect adolescent substance use, yet the relative influence may shift during adolescence as youth differentiate themselves from family and more closely affiliate with peers. This study examined trends in concordance of adolescent cigarette, alcohol and marijuana use and corresponding perceived use by friends and family members during middle and high school. Data are from a longitudinal cohort of 12,038 youth who completed up to five surveys during grades 6-12. At each wave, adolescents reported past month use of cigarettes, alcohol and marijuana, as well as perceived use by their best friend, older sibling and most important adult figure. For each substance, we used time-varying effect models to estimate how associations between adolescent use and perceived use varied across grade. For all substances, concordance with best friend use was positive and stronger than concordance with older sibling or adult use at all grades. Concordance with both best friend and older sibling use of all substances was pronounced in 6th grade. Concordance peaked again during mid-high school for smoking (best friend, older sibling) and marijuana (best friend). Concordance with adult marijuana use peaked in middle school, yet associations with adult alcohol and cigarette use were relatively stable. Substance use prevention efforts that seek to counter peer normative pressures should begin prior to middle school and span high school. Such efforts should address the role of peer and family environments, as both were found to be relevant during middle and high school.