It has been suggested that object colors in a colored environment are the result of combining in perception the (relative) brightness of each spectral component rather than of just mixing the spectral luminances. We tested this hypothesis with the following experiment: A pair of center-surround targets made of colored papers was illuminated with trichromatic white light. Two identical central color plates (test and match field, respectively) were surrounded by frames of different colors and thus looked different because of simultaneous color contrast. Observers were asked to match the colors by changing the illumination of the match field by means of a color-mixture projector (color match, CM). This color-matched reflectance was measured with a photometer, and its CIE coordinates were determined. We then illuminated the display with one of the three primaries that made up our trichromatic white light. The different reflectances of the different surrounds at each primary induced simultaneous brightness contrast. The brightnesses of the two central plates were therefore different. Observers were asked to change the intensity of the illumination of the match field at the respective primary so that it looked equally bright as the test field. This procedure was repeated for each primary (primary brightness match, PBM). Then the whole display except for the match field was illuminated with the trichromatic white as before, while the latter was illuminated with a trichromatic mixture consisting of the primaries at the intensities as set in the PBM experiment, and the CIE values were determined with the photometer. The CIE values of the match field after the CM and PBM procedures were nearly identical. This indicates that composite colors are composed in perception by combining the scaled (or relative) brightness of each spectral component and that this brightness scaling is largely restricted to interactions in the same spectral region. The results are compared with those of other models concerned with contrast colors as well as with neurophysiological data. Some limitations are mentioned.