Neural activation in the early visual cortex (EVC) reflects the perceived rather than retinal size of stimuli, suggesting that feedback possibly from extrastriate regions modulates retinal size information in EVC. Meanwhile, the lateral occipital cortex (LOC) has been suggested to be critically involved in object size processing. To test for the potential contributions of feedback modulations on size representations in EVC, we investigated the dynamics of relevant processes using transcranial magnetic stimulation (TMS). Specifically, we briefly disrupted the neural activity of EVC and LOC at early, intermediate, and late time windows while participants performed size judgment tasks in either an illusory or neutral context. TMS over EVC and LOC allowed determining whether these two brain regions are relevant for generating phenomenological size impressions. Furthermore, the temporal order of TMS effects allowed inferences on the dynamics of information exchange between the two areas. Particularly, if feedback signals from LOC to EVC are crucial for generating altered size representations in EVC, then TMS effects over EVC should be observed simultaneously or later than the effects following LOC stimulation. The data from 20 humans (13 females) revealed that TMS over both EVC and LOC impaired illusory size perception. However, the strongest effects of TMS applied over EVC occurred later than those of LOC, supporting a functionally relevant feedback modulation from LOC to EVC for scaling size information. Our results suggest that context integration and the concomitant change of perceived size require LOC and result in modulating representations in EVC via recurrent processing.SIGNIFICANCE STATEMENT How we perceive an object's size is not entirely determined by its physical size or the size of its retinal representation but also the spatial context. Using transcranial magnetic stimulation, we investigated the role of the early visual cortex (EVC) and the higher-level visual area, lateral occipital cortex (LOC), known to be critically involved in object processing, in transforming an initial retinal representation into one that reflects perceived size. Transcranial magnetic stimulation altered size perception earlier over LOC compared with EVC, suggesting that context integration and the concomitant change in perceived size representations in EVC rely on feedback from LOC.