There is evidence for different levels of visuospatial processing with their own frames of reference: viewer-centered, stimulus-centered, and object-centered. The neural locus of these levels can be explored by examining lesion location in subjects with unilateral spatial neglect (USN) manifest in these reference frames. Most studies regarding the neural locus of USN have treated it as a homogenous syndrome, resulting in conflicting results. In order to further explore the neural locus of visuospatial processes differentiated by frame of reference, we presented a battery of tests to 171 subjects within 48 hr after right supratentorial ischemic stroke before possible structural and/or functional reorganization. The battery included MR perfusion weighted imaging (which shows hypoperfused regions that may be dysfunctional), diffusion weighted imaging (which reveals areas of infarct or dense ischemia shortly after stroke onset), and tests designed to disambiguate between various types of neglect. Results were consistent with a dorsal/ventral stream distinction in egocentric/allocentric processing. We provide evidence that portions of the dorsal stream of visual processing, including the right supramarginal gyrus, are involved in spatial encoding in egocentric coordinates, whereas parts of the ventral stream (including the posterior inferior temporal gyrus) are involved in allocentric encoding.