Infant gut microbiota composition is influenced by various factors early in life. Here, we investigate associations between infant gut microbiome development, infant age, breastfeeding duration, and human milk oligosaccharides (HMO) composition in breastmilk. A total of 94 mother-infant pairs were recruited as part of the Cambridge Baby Growth and Breastfeeding Study (CBGS-BF) (Cambridge, UK). Infant stool samples (n = 337) were collected at 2 week, 6 week, 3 month, and 6 month of age. The 16S rRNA V3-V4 rRNA region was sequenced using MiSeq Illumina to determine microbiota composition and diversity. Mother's hindmilk samples were collected at birth, 2 week, 6 week, 3 month, and 6 month postpartum. Concentrations of five neutral [2'FL, 3'FL, lacto-N-fucopentaose 1 (LNFP1), LNnT, LNT] and two acidic (3'SL, and 6'SL) HMOs were measured in all milk samples using High-Performance Anion-Exchange Chromatography with Pulsed Amperometric Detection (HPAEC-PAD). We explored the associations between infant gut microbiome parameters and age, duration of exclusive breastfeeding (EBF), and levels of individual HMOs. Bifidobacterium was the most abundant genus in infant stool at all-time points, irrespective of breastfeeding duration, with an overall mean relative abundance of 70%. The relative abundance of B. bifidum in stool from infants who were breastfed for longer than 6 months was significantly higher compared to the infant breastfed up to 3 months (p = 0.0285). Alpha-diversity (both Shannon and ASV-level Richness) of infant gut microbiota showed a biphasic change with infant age, decreasing from 2 weeks until 3 months and then increasing until 6 months of age. Bifidobacterium relative abundance was associated with higher concentrations of 2'FL and LNFP1 in breastmilk across all time-points (p = 0.049 and 0.017, respectively), with trends toward a higher abundance of B. longum species. No significant association with Bifidobacterium was found for breastmilk LNnT, 3'SL, and 6'SL levels. Our study is in line with previous data demonstrating that EBF duration in the first months of life impacts infant gut microbiota composition. The observed links between specific HMOs in breastmilk and bacteria in infant stool provide evidence of how mother's milk affects infant microbiome development.