Understanding how dietary nutrients modulate the gut microbiome is of great interest for the development of food products and eating patterns for combatting the global burden of non-communicable diseases. In this narrative review we assess scientific studies published from 2005 to 2019 that evaluated the effect of micro- and macro-nutrients on the composition of the gut microbiome using in vitro and in vivo models, and human clinical trials. The clinical evidence for micronutrients is less clear and generally lacking. However, preclinical evidence suggests that red wine- and tea-derived polyphenols and vitamin D can modulate potentially beneficial bacteria. Current research shows consistent clinical evidence that dietary fibers, including arabinoxylans, galacto-oligosaccharides, inulin, and oligofructose, promote a range of beneficial bacteria and suppress potentially detrimental species. The preclinical evidence suggests that both the quantity and type of fat modulate both beneficial and potentially detrimental microbes, as well as the Firmicutes/Bacteroides ratio in the gut. Clinical and preclinical studies suggest that the type and amount of proteins in the diet has substantial and differential effects on the gut microbiota. Further clinical investigation of the effect of micronutrients and macronutrients on the microbiome and metabolome is warranted, along with understanding how this influences host health.