The goal of this study was to assess whether a deep learning estimate of age from a chest radiograph image (CXR-Age) can predict longevity beyond chronological age. Chronological age is an imperfect measure of longevity. Biological age, a measure of overall health, may improve personalized care. This paper proposes a new way to estimate biological age using a convolutional neural network that takes as input a CXR image and outputs a chest x-ray age (in years) as a measure of long-term mortality risk. CXR-Age was developed using CXR from 116,035 individuals and validated in 2 held-out testing sets: 1) 75% of the CXR arm of PLCO (Prostate, Lung, Colorectal, and Ovarian Cancer Screening Trial) (N = 40,967); and 2) the CXR arm of NLST (National Lung Screening Trial) (N = 5,414). CXR-Age was compared to chronological age and a multivariable regression model of chronological age, risk factors, and radiograph findings to predict all-cause and cardiovascular mortality with a maximum 23 years and 13 years of follow-up, respectively. The primary outcome was observed mortality; results are provided for the testing datasets only. In the PLCO testing dataset, a 5-year increase in CXR-Age carried a higher risk of all-cause mortality than a 5-year increase in chronological age (CXR-Age hazard ratio [HR]: 2.26 [95% confidence interval (CI): 2.24 to 2.29] vs. chronological age HR: 1.77 [95% CI: 1.75 to 1.78]; p < 0.001). A similar pattern was found for cardiovascular mortality (CXR-Age cause-specific HR: 2.45 per 5 years [95% CI: 2.34 to 2.56] vs. chronological age HR: 1.82 per 5 years [95% CI: 1.74 to 1.90]). Similar results were seen for both outcomes in the NLST external testing dataset. Adding CXR-Age to the multivariable model resulted in significant improvements for predicting both outcomes in both testing datasets (p < 0.001 for all comparisons). Based on a CXR image, CXR-Age predicted long-term all-cause and cardiovascular mortality.