With the development of wide color gamut displays, several recent studies investigated the performance of the CIE standard color matching functions (CMFs) in characterizing the color matches and observer metamerism between different primary sets. These studies, however, always employed different primaries to produce color stimuli, which failed to isolate the effect of the peak wavelength from that of the spectral shape. In this study, we carefully selected primaries with similar spectral shapes but different peak wavelengths. Human observers adjusted the intensities of the seven matching primary sets to match the color appearance of six stimuli, with a field of view around 5.7° × 5.7°, produced using a reference primary set, which was the same as one of the matching primary sets. The results clearly revealed the significant effect of the primary peak wavelength, and the failure of using chromaticities to characterize color matches using different primaries. The CIE 2006 2° CMFs were found to have the best performance in characterizing the color matches on average among the four CIE standard CMFs (i.e., the CIE 1931 2°, CIE 1964 2°, CIE 2006 2°, and CIE 2006 10° CMFs), which did not support the CIE's recommendation of using the 10° CMFs for stimuli beyond 4°. When the two categorical observer CMFs (i.e., Sarkar 2 set and Beijing Institute of Graphic Communication "BIGC" 17 set) were considered together, the BIGC 17 set had the best performance on average. More importantly, the performance of the CMFs varied with the primary sets. When the matching and reference sets were the same, the performance of all the CMFs were consistently good. In contrast, when the blue or green primary, or both of the two primaries, was shifted, the performance of all the CMFs became much worse. This clearly implies the importance of considering primary wavelengths when specifying color matches using different CMFs.