Point defects in wide-band-gap semiconductors are emerging as versatile resources for nanoscale sensing and quantum information science, but our understanding of the photoionization dynamics is presently incomplete. Here, we use two-color confocal microscopy to investigate the dynamics of charge in type 1b diamond hosting nitrogen-vacancy (NV) and silicon-vacancy (SiV) centers. By examining the nonlocal fluorescence patterns emerging from local laser excitation, we show that, in the simultaneous presence of photogenerated electrons and holes, SiV (NV) centers selectively transform into the negative (neutral) charge state. Unlike NVs, 532 nm illumination ionizes SiV^{-} via a single-photon process, thus hinting at a comparatively shallower ground state. In particular, slower ionization rates at longer wavelengths suggest the latter lies approximately ∼1.9  eV below the conduction band minimum. Building on the above observations, we demonstrate on-demand SiV and NV charge initialization over large areas via green laser illumination of variable intensity.