Young adulthood has been recognized as an important period for the transition from adolescence to adulthood. The University Personality Inventory (UPI), a mental health questionnaire for young adulthood, is widely used to screen university students in East Asia. However, dichotomous systems do not allow respondent choose responses other than two options on each symptom. This study employed item response theory (IRT) to examine the properties and performance of UPI items for mental health problems. Japanese medical students (n = 1185) participated in this study, and the UPI was completed at the time of university admission. The two-parameter IRT model was used to assess the measurement characteristics of the UPI items. Among all participants, 35.4% (420/1185) had total UPI score of 21 or more, and 10.6% (126/1185) indicated that they had the idea of wanting to die (item 25). For further IRT analysis, unidimensionality was confirmed by exploratory factor analysis, in which the primary factor accounted for 39.6% of the variance. The scale has sufficient discrimination power. In the test characteristic curves, the rising slopes of the lines were between θ 0 and 2. The UPI is useful to assess mild or moderate mental health problems, while precision may decline among individuals experiencing both little and extremely high levels of stress. Our findings provide a basis for identifying people who have mental health concerns.