Color is a key feature of natural environments that higher mammals routinely use to detect food, avoid predators, and interpret social signals. The distribution of color signals in natural scenes is widely variable, ranging from uniform patches to highly nonuniform regions in which different colors lie in close proximity. Whether individual neurons are tuned to this high degree of variability of color signals is unknown. Here, we identified a distinct population of cells in macaque visual cortex (area V4) that have a heterogeneous receptive field (RF) structure in which individual subfields are tuned to different colors even though the full RF is only weakly tuned. This spatial heterogeneity in color tuning indicates a higher degree of complexity of color-encoding mechanisms in visual cortex than previously believed to efficiently extract chromatic information from the environment.