Sound absorbing composites with stratified structures, including double-layer and sandwich structures, were prepared through the combination of nitrile butadiene rubber (NBR) and polyurethane foam (PUFM). The effects of the thickness ratio of layers, different stratified structures and the variety of fillers on the sound absorption performance of the NBR-PUFM composites and the sound absorption mechanism were studied. The results show that the NBR-PUFM composite with a sandwich structure and thickness ratio of 1:8:1 displays good sound absorption, which could be improved further by adding fillers. Because the airflow resistivity, resonance absorption, interface dissipation and interface reflection were combined organically in the sandwich structure, the composites show excellent low-frequency sound absorption performance. Moreover, the composite also has advantages in cost and functionalization aspects.