Sex/gender differences in cognitive sciences are riddled by conflicting perspectives. At the center of debates are clinical, social, and political perspectives. Front and center, evolutionary and biological perspectives have often focused on 'nature' arguments, while feminist and constructivist views have often focused on 'nurture arguments regarding cognitive sex differences. In the current narrative review, we provide a comprehensive overview regarding the origins and historical advancement of these debates while providing a summary of the results in the field of sexually polymorphic cognition. In so doing, we attempt to highlight the importance of using transdisciplinary perspectives which help bridge disciplines together to provide a refined understanding the specific factors that drive sex differences a gender diversity in cognitive abilities. To summarize, biological sex (e.g., birth-assigned sex, sex hormones), socio-cultural gender (gender identity, gender roles), and sexual orientation each uniquely shape the cognitive abilities reviewed. To date, however, few studies integrate these sex and gender factors together to better understand individual differences in cognitive functioning. This has potential benefits if a broader understanding of sex and gender factors are systematically measured when researching and treating numerous conditions where cognition is altered.