Whether the semantic representation of numbers is endowed with an intrinsic spatial component, so that smaller numbers are inherently represented to the left of larger ones on a Mental Number Line (MNL), is a central matter of debate in numerical cognition. To gain an insight into this issue, we investigated the performance of right brain damaged patients with left spatial neglect (N+) in a bimanual Magnitude Comparison SNARC task and in a uni-manual Magnitude Comparison Go/No-Go task (i.e. "is the number smaller or larger than 5?"). While the first task requires the use of contrasting left/right spatial codes for response selection, the second task does not require the use of these codes. In line with previous evidence, in the SNARC task N+ patients displayed a significant asymmetry in Reaction Times (RTs), with slower RTs to number "4", that was immediately precedent to the numerical reference "5", with respect to the number "6", that immediately followed the same reference. This RTs asymmetry was correlated with lesion of white matter tracts, i.e. Fronto-Occipital-Fasciculus, that allows prefrontal Ba 8 and 46 to regulate the distribution of attention on sensory and memory traces in posterior occipital, temporal and parietal areas. In contrast, no similar RTs asymmetry was found in the Go/No-Go task. These findings suggest that while in the SNARC task numbers get mentally organised from left-to-right as a function of their increasing magnitude, so that N+ patients display a delay in the processing of number-magnitudes that are immediately smaller than a given numerical reference, in the Go/No-Go task no left-to-right organization is activated. These results support the idea that it is the use of contrasting left/right spatial codes, whether motor or conceptual, that triggers the generation of a spatially left-to-right organised MNL and that the representation of number magnitude is not endowed with an inherent spatial component.