Multisensory integration is known to occur at high neural levels, but there is also growing evidence that cross-modal signals can be integrated at the first stages of sensory processing. We investigated whether touch specifically affected vision during binocular rivalry, a particular type of visual bistability that engages neural competition in early visual cortices. We found that tactile signals interact with visual signals outside of awareness, when the visual stimulus congruent with the tactile one is perceptually suppressed during binocular rivalry and when the interaction is strictly tuned for matched visuo-tactile spatial frequencies. We also found that voluntary action does not play a leading role in mediating the effect, since the interaction was observed also when tactile stimulation was passively delivered to the finger. However, simultaneous presentation of visual and tactile stimuli is necessary to elicit the interaction, and an asynchronous priming touch stimulus is not affecting the onset of rivalry. These results point to a very early cross-modal interaction site, probably V1. By showing that spatial proximity between visual and tactile stimuli is a necessary condition for the interaction, we also suggest that the two sensory spatial maps are aligned according to retinotopic coordinates, corroborating the hypothesis of a very early interaction between visual and tactile signals during binocular rivalry.