Heparanase, an only endo-β-d-glucuronidase capable of cleaving heparan sulfate (HS) side chains at specific sites, contributes to remodeling of the extracellular matrix (ECM) and releasing of HS-linked growth factors, cytokines and signaling proteins. In addition, heparanase also plays an indispensable role in tumor angiogenesis, invasion and metastasis, indicating that it is a promising target for the development of antitumor drugs. Recent progress leads to three classes of heparanase inhibitors, including active analogs of endogenous substance, synthetic small molecule compounds and natural products. In this review, following an outline on the heparanase structure and function, an overview of the advancement of heparanase inhibitors as novel and potent anti-cancer agents will be given, especially introducing various existing heparanase inhibitors, as well as their inhibitory activities and mechanisms of action.