A number of psychophysical and physiological studies have suggested that first- and second-order motion signals are processed, at least initially, by independent pathways, and that the two pathways both consist of multiple motion-detecting channels that are each narrowly tuned to a different spatial scale (spatial frequency). However, the precise number and nature of the mechanisms that subserve first- and second-order motion perception in human vision remain both controversial and speculative. We sought to clarify this issue by conducting selective adaptation experiments, in which modulation-depth thresholds for identifying the direction of stimulus motion of first-order (luminance-defined) and second-order (contrast-defined) drifting gratings were measured both prior to and following adaptation to motion. The drift direction, spatial frequency and stimulus type (either first- or second-order) of the adaptation and test stimuli were systematically manipulated. When the adaptation and test stimuli were either both first-order gratings or both second-order gratings, robust elevations of direction-identification thresholds were found and, importantly, these aftereffects exhibited both direction-selectivity and spatial-frequency selectivity. Cross-over-adaptation effects between first- and second-order gratings were also sometimes observed, but were very weak and not spatial-frequency selective. These findings give direct support for the existence of multiple-scale processing for first- and second-order motion in the human visual system and provide additional evidence that the two varieties of motion are initially processed by independent pathways.