In this study, cellulose nanofibrils (CNFs), obtained from the chemical treatment of steam exploded rice straw fibers and the following oxidation by 2,2,6,6-tetramethylpiperidine-1-oxyl radical (TEMPO)-mediated system, were utilized to prepare the reinforced polyvinyl alcohol (PVA) composite films by solution casting method, and the morphology, transparency, thermal stability, mechanical properties and water absorption of these rice straw CNFs/PVA composite films were also investigated compared with the rice straw cellulose microfibrils (CMFs) reinforced PVA composite films. As compared with the rice straw CMFs/PVA composite films, the rice straw CNFs/PVA composite films exhibited more uniform fiber dispersion, better mechanical properties and transparency, and similar thermal stability. However, the rice straw CNFs/PVA composite films showed weaker water resistance than the rice straw CMFs/PVA composite films due to the existence of more polar groups in rice straw CNFs.