The cell wall is a surface layer located outside the cell membrane of almost all bacteria; it protects cells from environmental stresses and gives them their typical shape. The cell wall is highly conserved in bacteria and is the target for some of our best antibiotics. Surprisingly, some bacteria are able to shed their wall under the influence of stress, yielding cells that are cell-wall-deficient. Notably, wall-deficient cells are flexible and are able to maneuver through narrow spaces, insensitive to wall-targeting antibiotics, and capable of taking up and exchanging DNA. Moreover, given that wall-associated epitopes are often recognized by host defense systems, wall deficiency provides a plausible explanation for how some bacteria may hide in their host. In this review we focus on this paradoxical stress response, which provides cells with unique opportunities that are unavailable to walled cells.