Aldehyde dehydrogenase 2 (ALDH2) catalyses aldehyde species, including alcohol metabolites, mainly in the liver. We recently observed that ALDH2 is also expressed in skeletal muscle mitochondria; thus, we hypothesize that rs671 polymorphism-promoted functional loss of ALDH2 may induce deleterious effects in human skeletal muscle. We aimed to clarify the association of the ALDH2 rs671 polymorphism with muscle phenotypes and athletic capacity in a large Japanese cohort. A total of 3,055 subjects, comprising 1,714 athletes and 1,341 healthy control subjects (non-athletes), participated in this study. Non-athletes completed a questionnaire regarding their exercise habits, and were subjected to grip strength, 30-s chair stand, and 8-ft walking tests to assess muscle function. The ALDH2 GG, GA, and AA genotypes were detected at a frequency of 56%, 37%, and 7% among athletes, and of 54%, 37%, and 9% among non-athletes, respectively. The minor allele frequency was 25% in athletes and 28% in controls. Notably, ALDH2 genotype frequencies differed significantly between athletes and non-athletes (genotype: p = 0.048, allele: p = 0.021), with the AA genotype occurring at a significantly lower frequency among mixed-event athletes compared to non-athletes (p = 0.010). Furthermore, non-athletes who harboured GG and GA genotypes exhibited better muscle strength than those who carried the AA genotype (after adjustments for age, sex, body mass index, and exercise habits). The AA genotype and A allele of the ALDH2 rs671 polymorphism were associated with a reduced athletic capacity and poorer muscle phenotypes in the analysed Japanese cohort; thus, impaired ALDH2 activity may attenuate muscle function.