Health practitioners, policy-makers, and psychologists point to legitimate concerns about the negative impact of loneliness. To help resolve such negative impact, we need to better understand the psychometric structure of loneliness. Men’s and women’s differing social roles may mean that they experience different sources of loneliness. After matching via exact matching, we compared men and women’s scores (N = 273) on the abbreviated form of the Social and Emotional Loneliness Scale for Adults (SELSA) using confirmatory factor analysis and measurement invariance testing. We replicated the three-factor structure of the SELSA, thereby providing further evidence for differing etiologies of family, romantic, and social loneliness. We found no good evidence for gender differences in the structure of the questionnaire answers, indicating that the SELSA can be used to further illuminate the implications of loneliness for men and women.