Individuals appear to infer others' psychological characteristics according to facial attractiveness and these psychological characteristics can be classified into two categories in social cognition, that is, warmth and competence. However, which category of psychological characteristic is more associated with face attractiveness and its neural mechanisms have not been explored. To address this, participants were asked to judge others' warmth and competence traits based on face attractiveness, while their brains were scanned using functional magnetic resonance imaging (fMRI). They also assessed the attractiveness of faces after scanning. Behavioral results showed that the correlation between face attractiveness and warmth ratings was significantly higher than that with competence ratings. fMRI results demonstrated that the dorsomedial prefrontal cortex (dmPFC), temporoparietal junction (TPJ), lateral prefrontal cortex, and lateral temporal lobe were more involved in the warmth task. Moreover, attractiveness ratings were negatively correlated with activation of the dmPFC and TPJ only in the warmth task. Furthermore, the attractiveness ratings were negatively correlated with the defined dmPFC, region related to attractiveness judgment, only in the warmth task. In conclusion, people are more inclined to infer others' warmth than competence characteristics from face attractiveness, that is, face attractiveness is more associated with warmth than with competence.