Infections of the mammary gland remain a frequent disease of dairy ruminants that negatively affect animal welfare, milk quality, farmer serenity, and farming profitability and cause an increase in use of antimicrobials. There is a need for efficacious vaccines to alleviate the burden of mastitis in dairy farming, but this need has not been satisfactorily fulfilled despite decades of research. A careful appraisal of past and current research on mastitis vaccines reveals the peculiarities but also the commonalities among mammary gland infections associated with the major mastitis pathogens Escherichia coli, Staphylococcus aureus, Streptococcus uberis, Streptococcus agalactiae, or Streptococcus dysgalactiae. A major pitfall is that the immune mechanisms of effective protection have not been fully identified. Until now, vaccine development has been directed toward the generation of antibodies. In this review, we drew up an inventory of the main approaches used to design vaccines that aim at the major pathogens for the mammary gland, and we critically appraised the current and tentative vaccines. In particular, we sought to relate efficacy to vaccine-induced defense mechanisms to shed light on some possible reasons for current vaccine shortcomings. Based on the lessons learned from past attempts and the recent results of current research, the design of effective vaccines may take a new turn in the years to come.