Effective and targeted identification of chemical components of the Chinese herbal medicine Sabia parviflora remains a major challenge. Herein, we used ultra-high-performance liquid chromatography coupled with quadrupole time-of-flight mass spectrometry to analyze the chemical composition of S.parviflora. Its chemical components were rapidly identified using the characteristic ion filtration method, which involves these steps: (1) summarize the characteristic ions based on similar skeletons and compounds with substitution patterns, and establish a database; (2) screen and classify different types of compounds in S. parviflora based on the characteristic ions; and (3) identify the compounds based on molecular weight, secondary fragments, and the database. In the present study, the characteristic ions in S. parviflora were grouped into five major classes. A total of 104 components were identified, including 12 potentially novel compounds. This rapid and accurate method provides an important basis for basic chemical research in S. parviflora.