Graphene-based photodetectors have shown responsivities up to 108 A/W and photoconductive gains up to 108 electrons per photon. These photodetectors rely on a highly absorbing layer in close proximity to graphene, which induces a shift of the graphene chemical potential upon absorption, hence modifying its channel resistance. However, due to the semimetallic nature of graphene, the readout requires dark currents of hundreds of microamperes up to milliamperes, leading to high power consumption needed for the device operation. Here, we propose a different approach for highly responsive graphene-based photodetectors with orders of magnitude lower dark-current levels. A shift of the graphene chemical potential caused by light absorption in a layer of colloidal quantum dots induces a variation of the current flowing across a metal-insulator-graphene diode structure. Owing to the low density of states of graphene near the neutrality point, the light-induced shift in chemical potential can be relatively large, dramatically changing the amount of current flowing across the insulating barrier and giving rise to an alternative gain mechanism. This readout requires dark currents of hundreds of nanoamperes up to a few microamperes, orders of magnitude lower than that of other graphene-based photodetectors, while keeping responsivities of ∼70 A/W in the infrared, almost 2 orders of magnitude higher than that of established germanium on silicon and indium gallium arsenide infrared photodetectors. This makes the device appealing for applications where high responsivity and low power consumption are required.