To assess the correlation between body components with insulin resistance (IR) and islet beta cell function in patients with newly diagnosed type 2 diabetes mellitus (T2DM) or pre-diabetes mellitus (pre-DM) and to explore whether this correlation differs in males and females. 111 newly diagnosed diabetic or pre-diabetic patients were recruited in this cross-sectional study. 75g oral glucose tolerance test was used to determine the diagnosis of DM or pre-DM. Homeostasis model assessment of insulin resistance (HOMA-IR) and glucose disposition index (DI30) was calculated to assess IR and islet beta cell function, respectively. Whole-body and regional lean mass (LM) and fat mass (FM) were obtained by dual-energy X-ray absorptiometry (DEXA). Partial correlation and multiple linear regression analyses were used to determine the associations between body composition, IR, and islet beta cell function. More body fat and appendicular fat was observed in female patients than in male, though with similar BMI. Legs fat % was negatively correlated with HOMA-IR, whereas legs lean % was positively associated with HOMA-IR in females (r = -0.673, p = 0.017; r =0.664, p = 0.018, respectively). The regression analysis showed that legs LM was positively correlated with HOMA-IR in females. However, in male patients, android FM was positively correlated with HOMA-IR (r = 0.462, p = 0.007), and trunk LM was negatively associated with DI30 (r = -0.458, p = 0.007). Nevertheless, no significant correlation was observed between body composition and islet beta cell function in female patients. Android FM was positively correlated with IR only in male patients but not in females. Besides, relative legs fat and LM were independently associated with IR in female patients but not in males. Further studies are needed to explore the underlying mechanism.