Gut microbiota plays crucial roles in maintaining host health. External factors, such as diet, medicines, and environmental toxins, influence the composition of gut microbiota. Ochratoxin A (OTA) is one of the most prevalent and relevant mycotoxins and is a highly abundant food and animal feed contaminant. In the present study, we aimed to investigate OTA gut microbiome toxicity in mice sub-chronically exposed to low doses of OTA (0.21, 0.5, and 1.5 mg/kg body weight) by daily oral gavage for 28 days. Fecal microbiota from control and OTA-treated mice was analyzed using 16S ribosomal RNA (rRNA) gene sequencing followed by metagenomics. OTA exposure caused marked changes in gut microbial community structure, including the decrease in the diversity of fecal microbiota and the relative abundance of Firmicutes, as well as the increase in the relative abundance of Bacteroidetes at the phylum level. At the family level, six bacterial families (unclassified Bacteroidales, Porphyromonadaceae, unclassified Cyanobacteria, Streptococcaceae, Enterobacteriaceae, Ruminococcaceae) were significantly altered by OTA exposure. Interestingly, OTA-induced changes were observed in the lower-dose OTA groups, while high-dose OTA group microbiota was similar to control group. Our results demonstrated that sub-chronic exposure at low doses of OTA alters the structure and diversity of the gut microbial community.