Inhibition in neuronal networks of the neocortex serves a multitude of functions, such as balancing excitation and structuring neuronal activity in space and time. Plasticity of inhibition is mediated by changes at both inhibitory synapses, as well as excitatory synapses on inhibitory neurons. Using slices from visual cortex of young male rats, we describe a novel form of plasticity of excitatory synapses on inhibitory neurons, weight-dependent heterosynaptic plasticity. Recordings from connected pyramid-to-interneuron pairs confirm that postsynaptic activity alone can induce long-term changes at synapses that were not presynaptically active during the induction, i.e., heterosynaptic plasticity. Moreover, heterosynaptic changes can accompany homosynaptic plasticity induced in inhibitory neurons by conventional spike-timing-dependent plasticity protocols. In both fast-spiking (FS) and non-FS neurons, heterosynaptic changes were weight-dependent, because they correlated with initial paired-pulse ratio (PPR), indicative of initial strength of a synapse. Synapses with initially high PPR, indicative of low release probability ("weak" synapses), had the tendency to be potentiated, while synapses with low initial PPR ("strong" synapses) tended to depress or did not change. Interestingly, the net outcome of heterosynaptic changes was different in FS and non-FS neurons. FS neurons expressed balanced changes, with gross average (n = 142) not different from control. Non-FS neurons (n = 66) exhibited net potentiation. This difference could be because of higher initial PPR in the non-FS neurons. We propose that weight-dependent heterosynaptic plasticity may counteract runaway dynamics of excitatory inputs imposed by Hebbian-type learning rules and contribute to fine-tuning of distinct aspects of inhibitory function mediated by FS and non-FS neurons in neocortical networks.SIGNIFICANCE STATEMENT Dynamic balance of excitation and inhibition is fundamental for operation of neuronal networks. Fine-tuning of such balance requires synaptic plasticity. Knowledge about diverse forms of plasticity operating in excitatory and inhibitory neurons is necessary for understanding normal function and causes of dysfunction of the nervous system. Here we show that excitatory inputs to major archetypal classes of neocortical inhibitory neurons, fast-spiking (FS) and non-fast-spiking (non-FS), express a novel type of plasticity, weight-dependent heterosynaptic plasticity, which accompanies the induction of Hebbian-type changes. This novel form of plasticity may counteract runaway dynamics at excitatory synapses to inhibitory neurons imposed by Hebbian-type learning rules and contribute to fine-tuning of diverse aspects of inhibitory function mediated by FS and non-FS neurons in neocortical networks.