The growth and morphogenesis of mammary parenchyma varies substantially between species and is regulated by an array of systemic and local factors. Central to this regulation is the mammary fat pad, a matrix of adipose and connective tissue capable of mediating hormone action and synthesizing an array of growth regulatory molecules. In this article we highlight differences between the morphological development of the mammary parenchyma in rodents, humans, and ruminant dairy animals, placing emphasis on differences in the cellular composition and structure of the mammary fat pad. While a great deal remains to be understood about the ability of stroma to locally regulate mammary development, the significance of its contribution is becoming increasingly apparent. The actions of several steroid and peptide hormones appear to be mediated by an array of growth factors, proteases and extracellular matrix components synthesized by constituents of the mammary fat pad. Further, mammary adipose tissue represents a significant store of lipid which, by itself and through its derivatives, could influence the growth of mammary epithelium in diverse ways. This review describes the integral role of the mammary fat pad during mammogenesis, emphasizing the point that species differences must be addressed if local growth and morphogenic mechanisms within the mammary gland are to be resolved.