Biomaterial Database

Entropy, mutual information, and systematic measures of structured spiking neural networks. 2020

Wenjie Li, and Yao Li

Department of Mathematics and Statistics, Washington University, St. Louis, MO 63130, USA. Electronic address: li.wenjie@wustl.edu.

The aim of this paper is to investigate various information-theoretic measures, including entropy, mutual information, and some systematic measures that are based on mutual information, for a class of structured spiking neuronal networks. In order to analyze and compute these information-theoretic measures for large networks, we coarse-grained the data by ignoring the order of spikes that fall into the same small time bin. The resultant coarse-grained entropy mainly captures the information contained in the rhythm produced by a local population of the network. We first show that these information theoretical measures are well-defined and computable by proving stochastic stability and the law of large numbers. Then we use three neuronal network examples, from simple to complex, to investigate these information-theoretic measures. Several analytical and computational results about properties of these information-theoretic measures are given.

UI	MeSH Term	Description	Entries
D009474	Neurons	The basic cellular units of nervous tissue. Each neuron consists of a body, an axon, and dendrites. Their purpose is to receive, conduct, and transmit impulses in the NERVOUS SYSTEM.	Nerve Cells,Cell, Nerve,Cells, Nerve,Nerve Cell,Neuron
D016571	Neural Networks, Computer	A computer architecture, implementable in either hardware or software, modeled after biological neural networks. Like the biological system in which the processing capability is a result of the interconnection strengths between arrays of nonlinear processing nodes, computerized neural networks, often called perceptrons or multilayer connectionist models, consist of neuron-like units. A homogeneous group of units makes up a layer. These networks are good at pattern recognition. They are adaptive, performing tasks by example, and thus are better for decision-making than are linear learning machines or cluster analysis. They do not require explicit programming.	Computational Neural Networks,Connectionist Models,Models, Neural Network,Neural Network Models,Neural Networks (Computer),Perceptrons,Computational Neural Network,Computer Neural Network,Computer Neural Networks,Connectionist Model,Model, Connectionist,Model, Neural Network,Models, Connectionist,Network Model, Neural,Network Models, Neural,Network, Computational Neural,Network, Computer Neural,Network, Neural (Computer),Networks, Computational Neural,Networks, Computer Neural,Networks, Neural (Computer),Neural Network (Computer),Neural Network Model,Neural Network, Computational,Neural Network, Computer,Neural Networks, Computational,Perceptron
D019277	Entropy	The measure of that part of the heat or energy of a system which is not available to perform work. Entropy increases in all natural (spontaneous and irreversible) processes. (From Dorland, 28th ed)	Entropies

Related Publications

Wenjie Li, and Yao Li

Mutual information measure of visual perception based on noisy spiking neural networks.

January 2023, Frontiers in neuroscience,

Wenjie Li, and Yao Li

Evolving spiking neural networks for audiovisual information processing.

September 2010, Neural networks : the official journal of the International Neural Network Society,

Wenjie Li, and Yao Li

Learning Spatiotemporally Encoded Pattern Transformations in Structured Spiking Neural Networks.

December 2015, Neural computation,

Wenjie Li, and Yao Li

Wireless transmission of neural signals using entropy and mutual information compression.

February 2011, IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society,

Wenjie Li, and Yao Li

Structured information in small-world neural networks.

February 2009, Physical review. E, Statistical, nonlinear, and soft matter physics,

Wenjie Li, and Yao Li

Spiking neural networks.

August 2009, International journal of neural systems,

Wenjie Li, and Yao Li

Introduction to spiking neural networks: Information processing, learning and applications.

January 2011, Acta neurobiologiae experimentalis,

Wenjie Li, and Yao Li

Mutual information of spin systems from autoregressive neural networks.

October 2023, Physical review. E,