Mesenchymal stem cells (MSCs) are adult stem cells harboring self-renewal and multilineage differentiation potential that are capable of differentiating into osteoblasts, adipocytes, or chondrocytes in vitro, and regulating the bone marrow microenvironment and adipose tissue remodeling in vivo. The process of fate determination is initiated by signaling molecules that drive MSCs into a specific lineage. Impairment of MSC fate determination leads to different bone and adipose tissue-related diseases, including aging, osteoporosis, and insulin resistance. Much progress has been made in recent years in discovering small molecules and their underlying mechanisms control the cell fate of MSCs both in vitro and in vivo. In this review, we summarize recent findings in applying small molecules to the trilineage commitment of MSCs, for instance, genistein, medicarpin, and icariin for the osteogenic cell fate commitment; isorhamnetin, risedronate, and arctigenin for pro-adipogenesis; and atractylenolides and dihydroartemisinin for chondrogenic fate determination. We highlight the underlying mechanisms, including direct regulation, epigenetic modification, and post-translational modification of signaling molecules in the AMPK, MAPK, Notch, PI3K/AKT, Hedgehog signaling pathways etc. and discuss the small molecules that are currently being studied in clinical trials. The target-based manipulation of lineage-specific commitment by small molecules offers substantial insights into bone marrow microenvironment regulation, adipose tissue homeostasis, and therapeutic strategies for MSC-related diseases.