Recent studies have indicated that the incidence of esophageal cancer has declined in the past decade in the U.S. However, trends in the incidence and survival have not been thoroughly examined. Data from 46 063 patients with esophageal cancer between 1973 and 2015 were collected from the Surveillance, Epidemiology, and End Results database. The trends in the age-adjusted incidence and survival were analyzed using joinpoint regression models. The age-adjusted incidence of esophageal cancer increased from 5.55 to 7.44 per 100 000 person-years between 1973 and 2004. Later, it decreased at an annual percentage change of 1.23%. In the last 40 years, the strong male predominance increased slightly. Importantly, the percentage of patients with localized stage of squamous cell cancer decreased. It was observed that the incidence of esophageal squamous cell carcinoma declined since 1986, while the incidence of esophageal adenocarcinoma sharply increased since 1973 and surpassed the rate of squamous cell cancer, mainly due to the increase in the incidence among men. Consistently, the estimated 40-year limited-duration prevalence of esophageal adenocarcinoma was higher than that of esophageal squamous cell carcinoma. Additionally, we observed a modest but significant improvement in survival during the study period. The incidence of esophageal squamous cell carcinoma has decreased significantly over the past four decades in the U.S., while the incidence of adenocarcinoma has increased, particularly among men. Overall, the long-term survival of patients with esophageal cancer is poor but it has improved over the past decades, especially for the localized disease. Significant findings of the study The incidence of esophageal cancer has decreased at an annual percentage change of 1.23% since 2004. The incidence of esophageal adenocarcinoma has sharply increased since 1973 and surpassed the rate of squamous cell cancer, mainly due to the increase in the incidence among men. What this study adds There has been a shift in the prevalence of esophageal cancer histological subtypes over the past decades in the U.S. We found that the incidence of esophageal squamous cell carcinoma has continued to decrease, while the esophageal adenocarcinoma rate has continued to increase.