Diabetes mellitus (DM) has been shown to be a clinical risk factor for bone diseases including osteoporosis and fragility. Bone metabolism is a complicated process that requires coordinated differentiation and proliferation of bone marrow mesenchymal stem cells (BMSCs). Owing to the regenerative properties, BMSCs have laid a robust foundation for their clinical application in various diseases. However, mounting evidence indicates that the osteogenic capability of BMSCs is impaired under high glucose conditions, which is responsible for diabetic bone diseases and greatly reduces the therapeutic efficiency of BMSCs. With the rapidly increasing incidence of DM, a better understanding of the impacts of hyperglycemia on BMSCs osteogenesis and the underlying mechanisms is needed. In this review, we aim to summarize the current knowledge of the osteogenesis of BMSCs in hyperglycemia, the underlying mechanisms, and the strategies to rescue the impaired BMSCs osteogenesis.