Soil pollution with heavy metals has attracted increasing concern, which calls for the development of new remediation strategies. The combination of physical, chemical, and biological techniques can achieve more efficient remediation. However, few studies have focused on whether nanomaterials and beneficial microbes can be jointly used to facilitate phytoremediation. Therefore, we studied the role of nano-zero-valent iron (nZVI) and arbuscular mycorrhizal (AM) fungi in the phytoremediation of an acidic soil polluted with Cd, Pb and Zn, using sweet sorghum. X-ray diffraction (XRD), energy dispersive X-ray spectroscopy (EDS), and mapping analyses were conducted to explore the mechanisms of metal immobilization by nZVI. The results showed that although both bare nZVI (B-nZVI) and starch-stabilized nZVI (S-nZVI) inhibited root mycorrhizal colonization, Acaulospora mellea ZZ successfully colonized the plant roots. AM inoculation significantly reduced the concentrations of DTPA-Cd, -Pb, and -Zn in soil, and the concentrations of Cd, Pb, and Zn in plants, indicating that AM fungi substantially facilitated heavy metal immobilization. Both B-nZVI and S-nZVI, ranging from 50 mg/kg to 1000 mg/kg, did not impede plant growth, and generally enhanced the phytoextraction of heavy metals. XRD, EDS and mapping analyses showed that S-nZVI was more susceptible to oxidation than B-nZVI, and thus had more effective immobilization effects on heavy metals. Low concentrations of nZVI (e.g., 100 mg/kg) and AM inoculation had synergistic effects on heavy metal immobilization, reducing the concentrations of Pb and Cd in roots and enhancing root Zn accumulation. In conclusion, our results showed that AM inoculation was effective in immobilizing heavy metals, whereas nZVI had a low phytotoxicity, and they could jointly contribute to the phytoremediation of heavy metal-contaminated soils with sweet sorghum.