Ampullary adenomas (AAs), common in familial adenomatous polyposis (FAP), are precursors to ampullary carcinoma. We assessed the natural history of AAs and factors associated with clinically significant progression (CSP). Consecutive FAP patients with AAs and at least 2 EGDs were identified from a hereditary GI cancer registry. We assessed the incidence of CSP (increase in size to ≥10 mm and/or development of advanced histology) of AAs. Clinical, endoscopic, and pathologic features between patients with CSP and nonprogressors were compared. One hundred forty-three patients with AAs were included. Over a median follow-up of 7.8 years (interquartile range, 4.3-11.1), 41 patients (28.6%) developed CSP for an incidence of 35 per 1000 patient-years. Of 143 patients, 22 (15.6%) progressed to AAs ≥10 mm, 12 (8.5%) progressed to advanced histology, and 7 (4.9%) progressed both in size and histology. Two patients (1.4%) developed ampullary cancer. Male gender, abnormal appearance of the papilla at initial AA detection, prior cholecystectomy, and personal history of extracolonic malignancy were associated with CSP. Neither Spigelman stage nor the adenomatous polyposis coli gene pathogenic variants were associated with CSP. An intervention specifically for AA and not duodenal polyposis was performed in 24% of patients with AAs, including endoscopic papillectomy in 23 patients and duodenectomy in 3 patients at a median observation of 8.2 years. Most FAP patients with AAs did not experience CSP or require resection over 8 years of surveillance. Ampullary cancer was rare. Male gender, abnormal appearance of the papilla at AA detection, cholecystectomy, and history of extracolonic malignancy were associated with CSP. Our findings favor endoscopic surveillance of AAs over expedited resection for most patients with FAP.