Hydrogels, one of the most important bioinspired materials, are receiving increasing attention because of their potential applications as scaffolds for artificial tissue engineering and vehicles for drug delivery, etc. However, these applications are always severely limited by their microstructure and mechanical behavior. Here we report the fabrication of a tough polyvinyl alcohol/graphene oxide (PVA/GO) nanocomposite hydrogel through a simple and effective directional freezing-thawing (DFT) technique. The resulting hydrogels show well-developed anisotropic microstructure and excellent mechanical properties with the assistance of DFT method and lamellar graphene. The hydrogels with anisotropic porous structures that consisted of micro-sized fibers and lamellas exhibit high tensile strengths, up to 1.85 MPa with a water content of 90%. More interestingly, the PVA/GO composite hydrogels exhibit the better thermostability, which can maintain the original shape when swollen in hot water (65 °C). In addition, the hydrogels with biocompatibility show good drug release efficiency due to the unique hierarchical structure. The successful synthesis of such hydrogel materials might pave the way to explore applications in biomedical and soft robotics fields.