Retinal ganglion cells (RGCs) relay ~40 parallel and independent streams of visual information, each encoding a specific feature of a visual scene, to the brain for further processing. The polarity of a visual neuron's response to a change in contrast is generally the first characteristic used for functional classification: ON cells increase their spike rate to positive contrast; OFF cells increase their spike rate for negative contrast; ON-OFF cells increase their spike rate for both contrast polarities. Suppressed-by-Contrast (SbC) neurons represent a less well-known fourth category; they decrease firing below a baseline rate for both positive and negative contrasts. SbC RGCs were discovered over 50 years ago, and SbC visual neurons have now been found in the thalamus and primary visual cortex of several mammalian species, including primates. Recent discoveries of SbC RGCs in mice have provided new opportunities for tracing upstream circuits in the retina responsible for the SbC computation and downstream targets in the brain where this information is used. We review and clarify recent work on the circuit mechanism of the SbC computation in these RGCs. Studies of mechanism rely on precisely defined cell types, and we argue that, like ON, OFF, and ON-OFF RGCs, SbC RGCs consist of more than one type. A new appreciation of the diversity of SbC RGCs will help guide future work on their targets in the brain and their roles in visual perception and behavior.