Anal cancer screening has not been adopted by Taiwanese care providers. The study aim was to explore the differences of anal cytology and HPV detection among men with and without HIV. In this case-control study, men with HIV who attended one of the outpatient clinics of Taoyuan General Hospital were enrolled as cases. Men who had experienced condomless sex and tested HIV negative were enrolled as controls. Anal swabs were collected for thin-preparation anal cytology and HPV genotyping. A total of 288 men who had tested positive for HIV and 208 who had tested HIV negative were enrolled; 75% of subjects with HIV and 30.3% of those without HIV had tested positive for various types of HPV (P < 0.001). Anal cell dysplasia, including atypical squamous cells with undetermined significance (ASCUS), low-grade squamous intraepithelial lesions (LSILs), high-grade squamous intraepithelial lesions (HSILs), or atypical squamous cells cannot exclude HSIL (ASC-H), were noted in 20.8% of men with HIV and 4.8% of those without HIV (P < 0.001). In multivariate analysis, HIV serostatus, history of sexually transmitted infections, having male sexual partners, and practice of anal sex were correlated significantly with detection of any type of HPV. Additionally, both oncogenic and non-oncogenic HPV types were significantly associated with anal cytology dysplasia. We strongly suggest that there should be awareness of anal HPV infection and related anal cellular dysplasia in at-risk populations.