Phragmites australis, which is widely distributed throughout the world, is often used in the phytoremediation of acid mine drainage (AMD) due to its various mechanisms for survival under extremely harsh conditions. To explore the different responses of different aerial organs of P. australis to stress, soil and plant samples were collected from the AMD-polluted area of the Tongling mining area. The contents of manganese (Mn), iron (Fe), copper (Cu), zinc (Zn), arsenic (As), cadmium (Cd), and lead (Pb) in the soil and the leaf blades, leaf sheaths, and stems of P. australis as well as the contents/activities of cysteine synthase (CSase), superoxide dismutase (SOD), peroxidase (POD), glutathione (GSH), malondialdehyde (MDA), and proline (Pro) in the organs were determined. Our results revealed that the leaf sheath had the highest potential to store metals of all the organs. The highest translocation factor (TF) for Fe was observed from the stems to the leaf sheaths. A higher bioconcentration factor (BCF) for Mn was found in the leaf blades and leaf sheaths, while higher BCFs for Cd and Zn were observed in the stems. The content/activity of enzymatic and non-enzymatic stress-resistance substances varied from organ to organ. In general, the leaf sheaths remained almost as or slightly less stress-resistant than the leaf blades. It can be concluded that different plant organs play different roles in stress resistance, and understanding the tolerance mechanism of leaf sheaths to metals is essential for the application of phytoremediation procedures.