Phase-shifting interferometry (PSI) requires accurate phase shifts between interferograms for realizing high-accuracy phase retrieval. This paper presents an adaptive PSI through synchronously capturing phase shifts measurement interferograms and phase measurement interferograms, in which the former is a series of spatial carrier frequency phase-shifting interferograms generated by an additional assembly and the phase shifts are calculated with the single-spectrum phase shifts measurement algorithm (SS-PSMA), the latter is employed for phase retrieval with an adaptive phase-shifting digital holography algorithm (PSDHA) based on complex amplitude recovery. In addition to exhibiting excellent reliability, high-accuracy phase retrieval (0.02 rad), and short calculation time (<25 ms), the proposed adaptive PSDHA is suitable for various interferograms with different fringe shapes and numbers. Importantly, both simulation analysis and experimental result demonstrate that this adaptive PSI based on PSDHA can effectively eliminate phase-shifting errors caused by phase shifter and external disturbance, ensuring high-accuracy phase shifts measurement and phase retrieval, meanwhile significantly reducing phase-shifting interferograms acquisition time and phase retrieval calculation time.