Conditioning and time representation in long short-term memory networks. 2014

Francois Rivest, and John F Kalaska, and Yoshua Bengio
Department of Mathematics and Computer Science, Royal Military College of Canada, PO Box 17000, Station Forces, Kingston, ON, K7K 7B4, Canada, francois.rivest@rmc.ca.

Dopaminergic models based on the temporal-difference learning algorithm usually do not differentiate trace from delay conditioning. Instead, they use a fixed temporal representation of elapsed time since conditioned stimulus onset. Recently, a new model was proposed in which timing is learned within a long short-term memory (LSTM) artificial neural network representing the cerebral cortex (Rivest et al. in J Comput Neurosci 28(1):107-130, 2010). In this paper, that model's ability to reproduce and explain relevant data, as well as its ability to make interesting new predictions, are evaluated. The model reveals a strikingly different temporal representation between trace and delay conditioning since trace conditioning requires working memory to remember the past conditioned stimulus while delay conditioning does not. On the other hand, the model predicts no important difference in DA responses between those two conditions when trained on one conditioning paradigm and tested on the other. The model predicts that in trace conditioning, animal timing starts with the conditioned stimulus offset as opposed to its onset. In classical conditioning, it predicts that if the conditioned stimulus does not disappear after the reward, the animal may expect a second reward. Finally, the last simulation reveals that the buildup of activity of some units in the networks can adapt to new delays by adjusting their rate of integration. Most importantly, the paper shows that it is possible, with the proposed architecture, to acquire discharge patterns similar to those observed in dopaminergic neurons and in the cerebral cortex on those tasks simply by minimizing a predictive cost function.

UI MeSH Term Description Entries
D008570 Memory, Short-Term Remembrance of information for a few seconds to hours. Immediate Recall,Memory, Immediate,Working Memory,Memory, Shortterm,Immediate Memories,Immediate Memory,Immediate Recalls,Memories, Immediate,Memories, Short-Term,Memories, Shortterm,Memory, Short Term,Recall, Immediate,Recalls, Immediate,Short-Term Memories,Short-Term Memory,Shortterm Memories,Shortterm Memory,Working Memories
D008959 Models, Neurological Theoretical representations that simulate the behavior or activity of the neurological system, processes or phenomena; includes the use of mathematical equations, computers, and other electronic equipment. Neurologic Models,Model, Neurological,Neurologic Model,Neurological Model,Neurological Models,Model, Neurologic,Models, Neurologic
D001921 Brain The part of CENTRAL NERVOUS SYSTEM that is contained within the skull (CRANIUM). Arising from the NEURAL TUBE, the embryonic brain is comprised of three major parts including PROSENCEPHALON (the forebrain); MESENCEPHALON (the midbrain); and RHOMBENCEPHALON (the hindbrain). The developed brain consists of CEREBRUM; CEREBELLUM; and other structures in the BRAIN STEM. Encephalon
D000465 Algorithms A procedure consisting of a sequence of algebraic formulas and/or logical steps to calculate or determine a given task. Algorithm
D013997 Time Factors Elements of limited time intervals, contributing to particular results or situations. Time Series,Factor, Time,Time Factor
D016571 Neural Networks, Computer A computer architecture, implementable in either hardware or software, modeled after biological neural networks. Like the biological system in which the processing capability is a result of the interconnection strengths between arrays of nonlinear processing nodes, computerized neural networks, often called perceptrons or multilayer connectionist models, consist of neuron-like units. A homogeneous group of units makes up a layer. These networks are good at pattern recognition. They are adaptive, performing tasks by example, and thus are better for decision-making than are linear learning machines or cluster analysis. They do not require explicit programming. Computational Neural Networks,Connectionist Models,Models, Neural Network,Neural Network Models,Neural Networks (Computer),Perceptrons,Computational Neural Network,Computer Neural Network,Computer Neural Networks,Connectionist Model,Model, Connectionist,Model, Neural Network,Models, Connectionist,Network Model, Neural,Network Models, Neural,Network, Computational Neural,Network, Computer Neural,Network, Neural (Computer),Networks, Computational Neural,Networks, Computer Neural,Networks, Neural (Computer),Neural Network (Computer),Neural Network Model,Neural Network, Computational,Neural Network, Computer,Neural Networks, Computational,Perceptron
D057567 Memory, Long-Term Remembrance of information from 3 or more years previously. Memory, Longterm,Memory, Remote,Remote Memory,Long-Term Memories,Long-Term Memory,Longterm Memories,Longterm Memory,Memories, Long-Term,Memories, Longterm,Memories, Remote,Memory, Long Term,Remote Memories

Related Publications

Francois Rivest, and John F Kalaska, and Yoshua Bengio
October 2020, Medical image analysis,
Francois Rivest, and John F Kalaska, and Yoshua Bengio
August 1987, Acta psychologica,
Francois Rivest, and John F Kalaska, and Yoshua Bengio
October 1976, Journal of experimental psychology. Animal behavior processes,
Francois Rivest, and John F Kalaska, and Yoshua Bengio
July 2021, Scientific reports,
Francois Rivest, and John F Kalaska, and Yoshua Bengio
December 2021, Sensors (Basel, Switzerland),
Francois Rivest, and John F Kalaska, and Yoshua Bengio
April 2019, Physics in medicine and biology,
Francois Rivest, and John F Kalaska, and Yoshua Bengio
June 2020, Studies in health technology and informatics,
Francois Rivest, and John F Kalaska, and Yoshua Bengio
June 2020, Studies in health technology and informatics,
Francois Rivest, and John F Kalaska, and Yoshua Bengio
August 2018, International journal of neural systems,
Copied contents to your clipboard!