The speed of single-pixel imaging (SPI) is tied to its resolution, which is positively related to the number of modulation times. Therefore, efficient large-scale SPI is a serious challenge that impedes its wide applications. In this work, we report a novel, to the best of our knowledge, sparse SPI scheme and corresponding reconstruction algorithm to image target scenes at above 1 K resolution with reduced measurements. Specifically, we first analyze the statistical importance ranking of Fourier coefficients for natural images. Then the sparse sampling with a polynomially decending probability of the ranking is performed to cover a larger range of the Fourier spectrum than non-sparse sampling. The optimal sampling strategy with suitable sparsity is summarized for the best performance. Next, a lightweight deep distribution optimization (D2O) algorithm is introduced for large-scale SPI reconstruction from sparsely sampled measurements instead of a conventional inverse Fourier transform (IFT). The D2O algorithm empowers robustly recovering sharp scenes at 1 K resolution within 2 s. A series of experiments demonstrate the technique's superior accuracy and efficiency.