Cadmium (Cd) and arsenic (As) contamination in paddy soils poses serious health risks to humans. The accumulation of Cd and As in rice (Oryza sativa L.) depends on their bioavailability, which is affected by soil physicochemical properties and soil microbial activities. However, little is known about the intricate interplay between rice plants and their rhizosphere microbes during the uptake of Cd and As. In this study, different bacterial communities were established by sterilizing paddy soils with γ-radiation. A pot experiment using two paddy soils with different levels of contamination was conducted to explore how the bacterial community composition affects Cd and As accumulation in rice plants. The results showed that the sterilization treatment substantially changed the bacterial composition in the rhizosphere, and significantly increased the grain yield (by 33.5-38.3%). The sterilization treatment resulted in significantly decreased concentrations of Cd (by 18.2-38.7%) and As (by 20.3-36.7%) in the grain, straw, and root of rice plants. The accumulation of Cd and As in rice plants was negatively correlated with the relative abundance of sulfate-reducing bacteria and iron-oxidizing bacteria in the rhizosphere. Other specific taxa associated with the accumulation of Cd and As in rice plants were also identified. Our results suggest that regulating the composition of the rhizosphere bacterial community could simultaneously reduce Cd and As accumulation in rice grain and increase the grain yield. These results would be useful for developing strategies to cultivate safe rice crops in areas contaminated with Cd and As.