To improve the graphene/copper interfacial bonding and the strength of the copper matrix, Cu-Cr-Mg alloy powder and graphene nanosheets (GNPs) have been used as raw materials in the preparation of a layered graphene/Cu-Cr-Mg composite through high-energy ball-milling and fast hot-pressing sintering. The microstructure of the composite after sintering, as well as the effect of graphene on the mechanical properties and conductivity of the composite, are also studied. The results show that the tensile strength of the composite material reached a value of 349 MPa, which is 46% higher than that of the copper matrix, and the reinforcement efficiency of graphene is as large as 136. Furthermore, the electrical conductivity of the composite material was 81.6% IACS, which is only 0.90% IACS lower than that of the copper matrix. The Cr and Mg elements are found to diffuse to the interface of the graphene/copper composite during sintering, and finely dispersed chromium carbide particles are found to significantly improve the interfacial bonding strength of the composite. Thus, graphene could effectively improve the mechanical properties of the composite while maintaining a high electrical conductivity.