This article investigates the synaptic weight distribution of a self-supervised, sparse, and randomly connected recurrent network inspired by hippocampal region CA3. This network solves nontrivial sequence prediction problems by creating, on a neuron-by-neuron basis, special patterns of cell firing called local context units. These specialized patterns of cell firing--possibly an analog of hippocampal place cells--allow accurate prediction of the statistical distribution of synaptic weights, and this distribution is not at all gaussian. Aside from the majority of synapses that are, at least functionally, lost due to synaptic depression, the distribution is approximately uniform. Unexpectedly, this result is relatively independent of the input environment, and the uniform distribution of synaptic weights can be approximately parameterized based solely on the average activity level. Next, the results are generalized to other cell firing types (frequency codes and stochastic firing) and place cell-like firing distributions. Finally, we note that our predictions concerning the synaptic strength distribution can be extended to the distribution of correlated cell firings. Recent published neurophysiological results are consistent with this extension.