Object positions are coded relative to their surroundings, presumably providing visual stability during eye movements. But when does this perceived stability arise? Here we used a visual illusion, the frame-induced position shift, and measured electrophysiological activity elicited by an object whose perceived position was either shifted because of a surrounding frame or not, thus dissociating perceived and physical locations. We found that visually evoked responses were sensitive to only physical location earlier in time (∼70 ms), but both physical and illusory location information was present at a later time point (∼140 ms). Furthermore, location information could be reliably decoded across physical and illusory locations during the later time interval but not during the earlier time interval, demonstrating that neural activity patterns are shared between the two processes at a later stage. These results suggest that visual stability of objects emerges relatively late and is thus dependent on recurrent feedback from higher processing stages.