Quantifying Information Conveyed by Large Neuronal Populations. 2019

John A Berkowitz, and Tatyana O Sharpee
Department of Physics, University of California San Diego, San Diego, CA 92093, U.S.A. jaberkow@ucsd.edu.

Quantifying mutual information between inputs and outputs of a large neural circuit is an important open problem in both machine learning and neuroscience. However, evaluation of the mutual information is known to be generally intractable for large systems due to the exponential growth in the number of terms that need to be evaluated. Here we show how information contained in the responses of large neural populations can be effectively computed provided the input-output functions of individual neurons can be measured and approximated by a logistic function applied to a potentially nonlinear function of the stimulus. Neural responses in this model can remain sensitive to multiple stimulus components. We show that the mutual information in this model can be effectively approximated as a sum of lower-dimensional conditional mutual information terms. The approximations become exact in the limit of large neural populations and for certain conditions on the distribution of receptive fields across the neural population. We empirically find that these approximations continue to work well even when the conditions on the receptive field distributions are not fulfilled. The computing cost for the proposed methods grows linearly in the dimension of the input and compares favorably with other approximations.

UI MeSH Term Description Entries
D008959 Models, Neurological Theoretical representations that simulate the behavior or activity of the neurological system, processes or phenomena; includes the use of mathematical equations, computers, and other electronic equipment. Neurologic Models,Model, Neurological,Neurologic Model,Neurological Model,Neurological Models,Model, Neurologic,Models, Neurologic
D009474 Neurons The basic cellular units of nervous tissue. Each neuron consists of a body, an axon, and dendrites. Their purpose is to receive, conduct, and transmit impulses in the NERVOUS SYSTEM. Nerve Cells,Cell, Nerve,Cells, Nerve,Nerve Cell,Neuron
D006801 Humans Members of the species Homo sapiens. Homo sapiens,Man (Taxonomy),Human,Man, Modern,Modern Man
D000069550 Machine Learning A type of ARTIFICIAL INTELLIGENCE that enable COMPUTERS to independently initiate and execute LEARNING when exposed to new data. Transfer Learning,Learning, Machine,Learning, Transfer
D016571 Neural Networks, Computer A computer architecture, implementable in either hardware or software, modeled after biological neural networks. Like the biological system in which the processing capability is a result of the interconnection strengths between arrays of nonlinear processing nodes, computerized neural networks, often called perceptrons or multilayer connectionist models, consist of neuron-like units. A homogeneous group of units makes up a layer. These networks are good at pattern recognition. They are adaptive, performing tasks by example, and thus are better for decision-making than are linear learning machines or cluster analysis. They do not require explicit programming. Computational Neural Networks,Connectionist Models,Models, Neural Network,Neural Network Models,Neural Networks (Computer),Perceptrons,Computational Neural Network,Computer Neural Network,Computer Neural Networks,Connectionist Model,Model, Connectionist,Model, Neural Network,Models, Connectionist,Network Model, Neural,Network Models, Neural,Network, Computational Neural,Network, Computer Neural,Network, Neural (Computer),Networks, Computational Neural,Networks, Computer Neural,Networks, Neural (Computer),Neural Network (Computer),Neural Network Model,Neural Network, Computational,Neural Network, Computer,Neural Networks, Computational,Perceptron

Related Publications

John A Berkowitz, and Tatyana O Sharpee
February 2024, The Journal of the Acoustical Society of America,
John A Berkowitz, and Tatyana O Sharpee
August 1979, Albrecht von Graefes Archiv fur klinische und experimentelle Ophthalmologie. Albrecht von Graefe's archive for clinical and experimental ophthalmology,
John A Berkowitz, and Tatyana O Sharpee
May 1989, The Journal of the Acoustical Society of America,
John A Berkowitz, and Tatyana O Sharpee
March 2003, Journal of psycholinguistic research,
John A Berkowitz, and Tatyana O Sharpee
January 2005, The Journal of pediatrics,
John A Berkowitz, and Tatyana O Sharpee
March 2009, Nature reviews. Neuroscience,
John A Berkowitz, and Tatyana O Sharpee
November 2013, Journal of neuroscience methods,
John A Berkowitz, and Tatyana O Sharpee
March 2022, Quarterly journal of experimental psychology (2006),
John A Berkowitz, and Tatyana O Sharpee
April 2012, Physical review. E, Statistical, nonlinear, and soft matter physics,
Copied contents to your clipboard!