Humans have been infected with Mycobacterium tuberculosis (Mtb) for thousands of years. While tuberculosis (TB), one of the deadliest infectious diseases, is caused by uncontrolled Mtb infection, over 90% of presumed infected individuals remain asymptomatic and contain Mtb in a latent TB infection (LTBI) without ever developing disease, and some may clear the infection. A small number of heavily Mtb-exposed individuals appear to resist developing traditional LTBI. Because Mtb has mechanisms for intracellular survival and immune evasion, successful control involves all of the arms of the immune system. Here, we focus on immune responses to Mtb in humans and nonhuman primates and discuss new concepts and outline major knowledge gaps in our understanding of LTBI, ranging from the earliest events of exposure and infection to success or failure of Mtb control.