During spermatogenesis, global nucleosome removal occurs where histones are initially replaced by transition proteins and subsequently by protamines. This chromatin reorganization is thought to facilitate the compaction of the paternal genome into the sperm head and to protect the DNA from damaging agents. Histone ubiquitination has been suggested to be important for sex chromosome inactivation during meiotic prophase and nucleosome removal at postmeiotic stages. However, the mechanisms regulating these ubiquitin-mediated processes are unknown. In this study, we investigate the role of the ubiquitin ligase RNF8 during spermatogenesis and find that RNF8-deficient mice are proficient in meiotic sex chromosome inactivation (MSCI) but deficient in global nucleosome removal. Moreover, we show that RNF8-dependent histone ubiquitination induces H4K16 acetylation, which may be an initial step in nucleosome removal. Thus, our results show that RNF8 plays an important role during spermatogenesis through histone ubiquitination, resulting in trans-histone acetylation and global nucleosome removal.