Despite intense efforts it has remained unresolved whether and how interfacial spin-orbit coupling (ISOC) affects spin transport across heavy-metal (HM)-ferromagnet (FM) interfaces. Here we report conclusive experiment evidence that the ISOC at HM/FM interfaces is the dominant mechanism for "spin memory loss". An increase in ISOC significantly reduces, in a linear manner, the dampinglike spin-orbit torque (SOT) exerted on the FM layer via degradation of the spin transparency of the interface for spin currents generated in the HM. In addition, the fieldlike SOT is also dominated by the spin Hall contribution of the HM and decreases with increasing ISOC. This work reveals that ISOC at HM/FM interfaces should be minimized to advance efficient SOT devices through atomic layer passivation of the HM/FM interface or other means.