Upon pollination, two sperm cells are transported inside the growing pollen tube toward the apex. One sperm cell fertilizes the egg cell to form the zygote, while the other fuses with the two polar nuclei to form the triploid endosperm. In Arabidopsis thaliana, the transport of the two sperm cells is characterized by sequential forward and backward movements with intermediate pauses. Until now, it is under debate which components of the plant cytoskeleton govern this mechanism. The sperm cells are interconnected and linked to the vegetative nucleus via a cytoplasmic projection, thus forming the male germ unit. This led to the common hypothesis that the vegetative nucleus is actively transported via myosin motors along actin cables while pulling along the sperm cells as passive cargo. In this study, however, we show that upon occasional germ unit disassembly, the sperm cells are transported independently and still follow the same bidirectional movement pattern. Moreover, we found that the net movement of sperm cells results from a combination of both longer and faster runs toward the pollen tube apex. We propose that the observed saltatory movement can be explained by the function of kinesins with calponin homology domain (KCH). This subgroup of the kinesin-14 family actively links actin filaments and microtubules. Based on KCH's specific properties derived from in vitro experiments, we built a tug-of-war model that could reproduce the characteristic sperm cell movement in pollen tubes.