Vision loss and depression are common conditions with major health implications. However, mechanisms of the association of visual health (across the full acuity spectrum) with depression remain unclear. To characterize the association between visual health and depression and investigate the association between depression and brain microstructure and macrostructure in subgroups divided by visual acuity. In the UK Biobank Study cohort, 114 583 volunteers were included at baseline from March to June 2006 to July 2010. Habitual distance visual acuity was examined using the logarithm of the minimum angle of resolution (LogMAR) characters. Depression was identified based on Patient Health Questionnaire (PHQ) or through an interview-based psychiatric diagnosis. Subgroup participants completed multimodal magnetic resonance imaging (MRI) of the brain and PHQ evaluation during the imaging visit after 2014. Data were analyzed from May 5 to August 9, 2022. Depression, depressive symptoms, and imaging-derived phenotypes from T1-weighted and diffusion MRI. Of the 114 583 participants from the UK Biobank Study, 62 401 (54.5%) were women, and the mean (SD) age was 56.8 (8.1) years (range, 39-72 years). A 1-line worse visual acuity (0.1 LogMAR increase) was associated with 5% higher odds of depression (odds ratio, 1.05 [95% CI, 1.04-1.07]) after adjustment for age, sex, race and ethnicity, Townsend index, educational qualifications, smoking, alcohol consumption, obesity, physical activity, history of hypertension, diabetes, hyperlipidemia, and family history of depression. Of the 7844 participants eligible for MRI analysis, there were linear associations between PHQ score and the left volume of gray matter in supracalcarine cortex (coefficient, 7.61 [95% CI, 3.90-11.31]) and mean isotropic volume fraction (ISOVF) in the right fornix (cres) and/or stria terminalis (coefficient, 0.003 [95% CI, 0.001-0.004]) after correction for multiple comparison. In addition, their association could be moderated by visual acuity, whereby increased PHQ score was associated with higher ISOVF levels only among those with poorer visual acuity (P = .02 for interaction). This study suggests an association between visual health and depression and that the diffusion characteristic of ISOVF in the fornix (cres) and/or stria terminalis is associated with depressive symptoms in participants with poorer visual acuity.