Neurons in cortical area MT or V5 of primates have a large, modulatory region surrounding the classical receptive field. This 'surround' has been suggested to be involved in motion segmentation, as well as in shape-from-motion processing. Our hypothesis is that it plays a functional role in both. We verify this by modeling the electrophysiological data obtained by Orban and co-workers in the macaque, and by developing a novel stimulus paradigm. Our results indicate an almost perfect dichotomy between both functionalities: our model neurons code for the object's edge if present, and for the first-order shape otherwise. We further show that small populations of model neurons can code linearly for the orientation-in-depth of translating planes.