Synaptic interactions between nearby excitatory and inhibitory neurons in the neocortex are thought to play fundamental roles in sensory processing. Here, we have combined optogenetic stimulation, whole cell recordings, and computational modeling to define key functional microcircuits within layer 2/3 of mouse primary somatosensory barrel cortex. In vitro optogenetic stimulation of excitatory layer 2/3 neurons expressing channelrhodopsin-2 evoked a rapid sequence of excitation followed by inhibition. Fast-spiking (FS) GABAergic neurons received large-amplitude, fast-rising depolarizing postsynaptic potentials, often driving action potentials. In contrast, the same optogenetic stimulus evoked small-amplitude, subthreshold postsynaptic potentials in excitatory and non-fast-spiking (NFS) GABAergic neurons. To understand the synaptic mechanisms underlying this network activity, we investigated unitary synaptic connectivity through multiple simultaneous whole cell recordings. FS GABAergic neurons received unitary excitatory postsynaptic potentials with higher probability, larger amplitudes, and faster kinetics compared with NFS GABAergic neurons and other excitatory neurons. Both FS and NFS GABAergic neurons evoked robust inhibition on postsynaptic layer 2/3 neurons. A simple computational model based on the experimentally determined electrophysiological properties of the different classes of layer 2/3 neurons and their unitary synaptic connectivity accounted for key aspects of the network activity evoked by optogenetic stimulation, including the strong recruitment of FS GABAergic neurons acting to suppress firing of excitatory neurons. We conclude that FS GABAergic neurons play an important role in neocortical microcircuit function through their strong local synaptic connectivity, which might contribute to driving sparse coding in excitatory layer 2/3 neurons of mouse barrel cortex in vivo.