The sense of body ownership, defined as the sensation that one's body belongs to oneself, is a fundamental component of bodily self-consciousness. Several studies have shown the importance of multisensory integration for the emergence of the sense of body ownership, together with the involvement of the parieto-premotor and extrastriate cortices in bodily awareness. However, whether the sense of body ownership elicited by different sources of signal, especially visuotactile and visuomotor inputs, is represented by common neural patterns remains to be elucidated. We used functional magnetic resonance imaging (fMRI) to investigate the existence of neural correlates of the sense of body ownership independent of the sensory modalities. Participants received tactile stimulation or executed finger movements while given synchronous and asynchronous visual feedback of their hand. We used multivoxel patterns analysis (MVPA) to decode the synchronous and asynchronous conditions with cross-classification between two modalities: the classifier was first trained in the visuotactile sessions and then tested in the visuomotor sessions, and vice versa. Regions of interest (ROIs)-based and searchlight analyses revealed significant above-chance cross-classification accuracies in the bilateral intraparietal sulcus (IPS), the bilateral ventral premotor cortex (PMv), and the left extrastriate body area (EBA). Moreover, we observed a significant positive correlation between the cross-classification accuracy in the left PMv and the difference in subjective ratings of the sense of body ownership between the synchronous and asynchronous conditions. Our findings revealed the neural representations of the sense of body ownership in the IPS, PMv, and EBA that is invariant to the sensory modalities.