Physical exercise is one of the most relevant lifestyle choices for the prevention of diseases; however, participation in this type of activity remains low. Therefore, it is necessary to deepen the understanding of related psychological factors in men and women. To determine whether personality traits, emotional intelligence and negative affective are differentially related to physical exercise characteristics in men and women. A cross-sectional study was conducted with 579 physically active people (61.1% men) between 18 and 59 years of age. The Big Five Inventory (BFI-15p), Brief Emotional Intelligence Scale (EQ-i-M20), and Depression, Anxiety and Stress Scale (DASS-21) were used. Compared with men, women exercised fewer days and minutes per week, had fewer years of participation and performed fewer different physical exercises. On the emotional intelligence scale, compared with men, women showed less stress management, adaptability and general mood but greater interpersonal intelligence. With regard to personality traits, compared with men, women showed greater openness, conscientiousness, agreeableness, and neuroticism. In males, heightened levels of general mood and extraversion were associated to a lasting commitment to physical exercise over time. Conversely, in females, depression was negatively associated with the years dedicated to physical exercise. Distinct approaches are essential for men and women, acknowledging the varied ways psychological factors are linked to physical exercise based on gender.