Semaphorins form a family of secreted and membrane-bound molecules that were identified originally as axonal guidance factors during neuronal development. Given their wide distribution in many including mature tissues, semaphorin 4D (sema4D) and its main functional receptor plexin B1 (plxnB1) are expected to fulfill additional functions that remain to be uncovered. A main characteristic of the plexin receptor family is its ability to reorganize the cytoskeleton. PlxnB1 specifically may directly interact with Rho family GTPases to regulate F-actin driven pathways in cells in culture. Diurnal clearance phagocytosis by the retinal pigment epithelium (RPE) of photoreceptor outer segment fragments (POS) is critical for photoreceptor function and longevity. In this process, rearrangement of RPE cytoskeletal F-actin via activation of the Rho family GTPase Rac1 is essential for POS internalization. Here, we show a novel role in POS phagocytosis by RPE cells in culture and in vivo for plexin B1 and its ligand sema4D. Exogenous sema4D abolishes POS internalization (but not binding) by differentiated RPE cells in culture by decreasing the GTP load of Rac1. In the rat eye, sema4D localizes to retinal photoreceptors, while PlxnB1 is expressed by neighboring RPE cells. At the peak of diurnal retinal phagocytosis after light onset, plxnB1 phosphorylation and sema4D levels are reduced in wild-type rat retina in situ but not in mutant RCS rat retina in which the RPE lacks phagocytic activity. Finally, increased POS phagosome content after light onset is observed in the RPE in situ of mice with either plxnB1 or sema4D gene deletion. Altogether, our results demonstrate a novel physiological function for sema4D/plxnB1 signaling in RPE phagocytosis serving as attenuating brake prior to light onset whose release enables the diurnal phagocytic burst.