Differential dynamic microscopy (DDM) is an emerging technique to measure the ensemble dynamics of colloidal and complex fluid motion using optical microscopy in systems that would otherwise be difficult to measure using other methods. To date, DDM has successfully been applied to linear space invariant imaging modes including bright-field, fluorescence, confocal, polarised, and phase-contrast microscopy to study diverse dynamic phenomena. In this work, we show for the first time how DDM analysis can be extended to dark-field imaging, i.e. a linear space variant (LSV) imaging mode. Specifically, we present a particle-based framework for describing dynamic image correlations in DDM, and use it to derive a correction to the image structure function obtained by DDM that accounts for scatterers with non-homogeneous intensity distributions as they move within the imaging plane. To validate the analysis, we study the Brownian motion of gold nanoparticles, whose plasmonic structure allows for nanometer-scale particles to be imaged under dark-field illumination, in Newtonian liquids. We find that diffusion coefficients of the nanoparticles can be reliably measured by dark-field DDM, even under optically dense concentrations where analysis via multiple-particle tracking microrheology fails. These results demonstrate the potential for DDM analysis to be applied to linear space variant forms of microscopy, providing access to experimental systems unavailable to other imaging modes.