Light is an easy acquired, effective and non-invasive external stimulus with great flexibility and focusability. Thus, light responsive hydrogels are of particular interests to researchers in developing accurate and controlled drug delivery systems. Light responsive hydrogels are obtained by incorporating photosensitive moieties into their polymeric structures. Drug release can be realized through three major mechanisms: photoisomerization, photochemical reaction and photothermal reaction. Recent advances in material science have resulted in great development of photosensitizers, such as rare metal nanostructures and black phosphorus nanoparticles, in order to respond to a variety of light sources. Hydrogels incorporated with photosensitizers are crucial for clinical applications, and the use of ultraviolet and near-infrared light as well as up-conversion nanoparticles has greatly increased the therapeutic effects. Existing light responsive drug delivery systems have been utilized in delivering drugs, proteins and genes for chemotherapy, immunotherapy, photodynamic therapy, gene therapy, wound healing and other applications. Principles associated with site-specific targeting, metabolism, and toxicity are used to optimize efficacy and safety, and to improve patient compliance and convenience. In view of the importance of this field, we review current development, challenges and future perspectives of light responsive hydrogels for controlled drug delivery.