Trained innate immunity has recently emerged as a novel concept of innate immune cells, such as myeloid cells, exhibiting immune memory, and nonspecific heterologous immunity to protect against infections. The memory and specificity are mediated by epigenetic, metabolic, and functional reprogramming of the myeloid cells and myeloid progenitors (and/or NK cells) in the bone marrow and peripheral tissues such as gut and lung mucosa. A variety of agents, such as BCG, viruses, and their components, as well as TLR agonists, and cytokines have been shown to be involved in the induction of trained immunity. Since these agents have been widely used in AIDS vaccine development as antigen delivery vectors or adjuvants, myeloid cell mediated trained immunity might also play an important role in protecting against mucosal AIDS virus transmission or in control of virus replication in the major gut mucosal reservoir. Here we review the trained innate immunity induced by these vectors/adjuvants that have been used in AIDS vaccine studies and discuss their role in mucosal vaccine efficacy and possible utilization in AIDS vaccine development. Delineating the protective effect of the trained innate immunity mediated by myeloid cells will guide the design of novel AIDS vaccines.