It only takes a few goal-directed hand movements to adapt one's movements to a prism-induced displacement of the visual scene. Adaptation to the displacement leads to errors in the opposite direction from the initial displacement when the prisms are removed. Such aftereffects are thought to arise from some form of spatial realignment of the senses or from motor learning. Here, we show that humans can simultaneously adapt the movements of the thumb and index finger of the same hand to opposing visual displacements. Neither the felt position of the hand nor the visually perceived direction can change in two opposite directions at the same time, ruling out an explanation based on realignment of the senses. It is conceivable that one could learn to adjust the movements differently for the two digits despite the fact that both adjustments would involve the same hand, but such motor learning should not transfer to matching the position of the unseen digit. As transfer was observed when visually matching the position of the unseen digit, motor learning cannot explain all of the results. An explanation involving supplementing proprioception with a memory-based visual estimate of the position of each unseen digit could explain all of the results. Irrespective of the mechanism, we can conclude that it is possible to adapt the perceived locations of the unseen digits without influencing proprioception.