How a sensory stimulus is processed and perceived depends on the surrounding sensory scene. In the visual cortex, contextual signals can be conveyed by an extensive network of intra- and inter-areal excitatory connections that link neurons representing stimulus features separated in visual space. However, the connectional logic of visual contextual inputs remains unknown; it is not clear what information individual neurons receive from different parts of the visual field, nor how this input relates to the visual features that a neuron encodes, defined by its spatial receptive field. Here we determine the organization of excitatory synaptic inputs responding to different locations in the visual scene by mapping spatial receptive fields in dendritic spines of mouse visual cortex neurons using two-photon calcium imaging. We find that neurons receive functionally diverse inputs from extended regions of visual space. Inputs representing similar visual features from the same location in visual space are more likely to cluster on neighbouring spines. Inputs from visual field regions beyond the receptive field of the postsynaptic neuron often synapse on higher-order dendritic branches. These putative long-range inputs are more frequent and more likely to share the preference for oriented edges with the postsynaptic neuron when the receptive field of the input is spatially displaced along the axis of the receptive field orientation of the postsynaptic neuron. Therefore, the connectivity between neurons with displaced receptive fields obeys a specific rule, whereby they connect preferentially when their receptive fields are co-oriented and co-axially aligned. This organization of synaptic connectivity is ideally suited for the amplification of elongated edges, which are enriched in the visual environment, and thus provides a potential substrate for contour integration and object grouping.