Background: Testosterone levels are differentially linked with diabetes risk in men and women: lower testosterone levels in men and higher testosterone levels in women are associated with type 2 diabetes, though, the mechanisms are not fully clear. We addressed sex-specific links between testosterone and major pathogenetic mechanisms of diabetes. Methods: We analyzed data of 623 subjects (202 male, 345 female without, and 76 female with oral contraceptive therapy [OCT]) for whom insulin sensitivity and insulin secretion were assessed by oral glucose tolerance test. Body fat percentage was assessed by bioelectrical impedance. Testosterone was measured by enzyme-linked immunoassay; free testosterone and Framingham risk score were calculated. Results: There were significant interactions between testosterone and sex for all tested metabolic traits. Increasing testosterone was associated with less body fat, elevated insulin sensitivity, and reduced glycemia, independent of adiposity in men. In women without OCT, testosterone correlated with more body fat, insulin resistance, and higher glucose concentrations. Testosterone was not associated with insulin secretion in either sex, but with lower Framingham risk score in men and higher Framingham risk score in women. Conclusions: Similar to diabetes risk, insulin resistance has different association directions with testosterone levels in males and females. Insulin resistance could therefore constitute the best biological candidate linking testosterone levels and diabetes prevalence. The question of antiandrogen therapy being able to improve metabolism, glucose tolerance and cardiovascular risk in women was not clarified in our study but should be reviewed with higher numbers in a carefully matched study to reduce the influence of confounding variables.