We demonstrate terahertz (THz) lens-free in-line holography on a chip in order to achieve 40 μm spatial resolution corresponding to ~0.7λ with a numerical aperture of ~0.87. We believe that this is the first time that sub-wavelength resolution in THz holography and the 40 μm resolution were both far better than what was already reported. The setup is based on a self-developed high-power continuous wave THz laser at 5.24 THz (λ = 57.25 μm) and a high-resolution microbolometer detector array (640 × 512 pixels) with a pitch of 17 μm. This on-chip in-line holography, however, suffers from the twin-image artifacts which obfuscate the reconstruction. To address this problem, we propose an iterative optimization framework, where the conventional object constraint and the L1 sparsity constraint can be combined to efficiently reconstruct the complex amplitude distribution of the sample. Note that the proposed framework and the sparsity-based algorithm can be applied to holography in other wavebands without limitation of wavelength. We demonstrate the success of this sparsity-based on-chip holography by imaging biological samples (i.e., a dragonfly wing and a bauhinia leaf).