The genetic risk of aggressive prostate cancer (PCa) is hard to be assessed due to the lack of aggressiveness-related single-nucleotide polymorphisms (SNPs). Prostate volume (PV) is a potential well-established risk factor for aggressive PCa, we hypothesize that polygenic risk score (PRS) based on benign prostate hyperplasia (BPH) or PV-related SNPs may also predict the risk of aggressive PCa or PCa death. We evaluated a PRS using 21 BPH/PV-associated SNPs, two established PCa risk-related PRS and 10 guideline-recommended hereditary cancer risk genes in the population-based UK Biobank cohort (N = 209,502). The BPH/PV PRS was significantly inversely associated with the incidence of lethal PCa as well as the natural progress in PCa patients (hazard ratio, HR = 0.92, 95% confidence interval [CI]: 0.87-0.98, P = 0.02; HR = 0.92, 95% CI 0.86-0.98, P = 0.01). Compared with men at the top 25th PRS, PCa patients with bottom 25th PRS would have a 1.41-fold (HR, 95% CI 1.16-1.69, P = 0.001) increased PCa fatal risk and shorter survival time at 0.37 yr (95% CI 0.14-0.61, P = 0.002). In addition, patients with BRCA2 or PALB2 pathogenic mutations would also have a high risk of PCa death (HR = 3.90, 95% CI 2.34-6.51, P = 1.79 × 10-7; HR = 4.29, 95% CI 1.36-13.50, P = 0.01, respectively). However, no interactive but independent effects were detected between this PRS and pathogenic mutations. Our findings provide a new measurement of PCa patients' natural disease outcomes via genetic risk ways.