Rhubarb is an important herbal medicine for the treatment of constipation, inflammation, and cancer. In this study, a facile method based on liquid chromatography coupled with electrospray ionization tandem mass spectrometry has been established for the analysis of bioactive phenolic compounds in rhubarbs. From six rhubarb species, official (Rheum officinale, R. palmatum, and R. tanguticum) and unofficial (R. franzenbachii, R. hotaoense, and R. emodi), a total of 107 phenolic compounds were identified or tentatively characterized based on their mass spectra. These compounds include sennosides, anthraquinones, stilbenes, glucose gallates, naphthalenes, and catechins. Ion chromatograms for the identified compounds of different rhubarbs were then compared. Consistent with previous reports, sennosides and rhein were only detected in official rhubarbs. Unexpectedly, we found that R. officinale contained very different phenolic compounds from the other two official species. Sennoside A, which has been considered as the major purgative component of rhubarb, was only detected in R. officinale, while its close isomers were observed in R. palmatum and R. tanguticum. In addition, the predominant anthraquinone glycosides in R. officinale were found to be rhein 8-O-glucoside and emodin 1-O-glucoside, whereas those in R. palmatum and R. tanguticum were rhein 1-O-glucoside and emodin 8-O-glucoside. Stilbenes, which are the major constituents of unofficial rhubarbs, were also different among the species. Our results clarify the chemical composition of rhubarbs comprehensively for the first time. Due to the significant differences in chemical components of rhubarbs, we suggest that different Rheum species be used separately in clinical practice.