Age-related hearing loss, or presbyacusis, is a major public health problem that causes communication difficulties and is associated with diminished quality of life. Limited satisfaction with hearing aids, particularly in noisy listening conditions, suggests that central nervous system declines occur with presbyacusis and may limit the efficacy of interventions focused solely on improving audibility. This study of 49 older adults (M = 69.58, SD = 8.22 years; 29 female) was designed to examine the extent to which low and/or high frequency hearing loss was related to auditory cortex morphology. Low and high frequency hearing constructs were obtained from a factor analysis of audiograms from these older adults and 1,704 audiograms from an independent sample of older adults. Significant region of interest and voxel-wise gray matter volume associations were observed for the high frequency hearing construct. These effects occurred most robustly in a primary auditory cortex region (Te1.0) where there was also elevated cerebrospinal fluid with high frequency hearing loss, suggesting that auditory cortex atrophies with high frequency hearing loss. These results indicate that Te1.0 is particularly affected by high frequency hearing loss and may be a target for evaluating the efficacy of interventions for hearing loss.