Immunosenescence comprises a set of dynamic changes occurring to both, the innate as well as the adaptive immune system that accompany human aging and result in complex manifestations of still poorly defined deficiencies in the elderly population. One of the most prominent alterations during aging is the continuous involution of the thymus gland which is almost complete by the age of 50. Consequently, the output of naïve T cells is greatly diminished in elderly individuals which puts pressure on homeostatic forces to maintain a steady T cell pool for most of adulthood. In a great proportion of the human population, this fragile balance is challenged by persistent viral infections, especially Cytomegalovirus (CMV), that oblige certain T cell clones to monoclonally expand repeatedly over a lifetime which then occupy space within the T cell pool. Eventually, these inflated memory T cell clones become exhausted and their extensive accumulation accelerates the age-dependent decline of the diversity of the T cell pool. As a consequence, infectious diseases are more frequent and severe in elderly persons and immunological protection following vaccination is reduced. This review therefore aims to shed light on how various types of persistent viral infections, especially CMV, influence the aging of the immune system and highlight potential measures to prevent the age-related decline in immune function.