Self-control with spiking and non-spiking neural networks playing games. 2010

Chris Christodoulou, and Gaye Banfield, and Aristodemos Cleanthous
Department of Computer Science, University of Cyprus, 75 Kallipoleos Avenue, Nicosia, Cyprus. cchrist@cs.ucy.ac.cy

Self-control can be defined as choosing a large delayed reward over a small immediate reward, while precommitment is the making of a choice with the specific aim of denying oneself future choices. Humans recognise that they have self-control problems and attempt to overcome them by applying precommitment. Problems in exercising self-control, suggest a conflict between cognition and motivation, which has been linked to competition between higher and lower brain functions (representing the frontal lobes and the limbic system respectively). This premise of an internal process conflict, lead to a behavioural model being proposed, based on which, we implemented a computational model for studying and explaining self-control through precommitment behaviour. Our model consists of two neural networks, initially non-spiking and then spiking ones, representing the higher and lower brain systems viewed as cooperating for the benefit of the organism. The non-spiking neural networks are of simple feed forward multilayer type with reinforcement learning, one with selective bootstrap weight update rule, which is seen as myopic, representing the lower brain and the other with the temporal difference weight update rule, which is seen as far-sighted, representing the higher brain. The spiking neural networks are implemented with leaky integrate-and-fire neurons with learning based on stochastic synaptic transmission. The differentiating element between the two brain centres in this implementation is based on the memory of past actions determined by an eligibility trace time constant. As the structure of the self-control problem can be likened to the Iterated Prisoner's Dilemma (IPD) game in that cooperation is to defection what self-control is to impulsiveness or what compromising is to insisting, we implemented the neural networks as two players, learning simultaneously but independently, competing in the IPD game. With a technique resembling the precommitment effect, whereby the payoffs for the dilemma cases in the IPD payoff matrix are differentially biased (increased or decreased), it is shown that increasing the precommitment effect (through increasing the differential bias) increases the probability of cooperating with oneself in the future, irrespective of whether the implementation is with spiking or non-spiking neural networks.

UI MeSH Term Description Entries
D007858 Learning Relatively permanent change in behavior that is the result of past experience or practice. The concept includes the acquisition of knowledge. Phenomenography
D008959 Models, Neurological Theoretical representations that simulate the behavior or activity of the neurological system, processes or phenomena; includes the use of mathematical equations, computers, and other electronic equipment. Neurologic Models,Model, Neurological,Neurologic Model,Neurological Model,Neurological Models,Model, Neurologic,Models, Neurologic
D009435 Synaptic Transmission The communication from a NEURON to a target (neuron, muscle, or secretory cell) across a SYNAPSE. In chemical synaptic transmission, the presynaptic neuron releases a NEUROTRANSMITTER that diffuses across the synaptic cleft and binds to specific synaptic receptors, activating them. The activated receptors modulate specific ion channels and/or second-messenger systems in the postsynaptic cell. In electrical synaptic transmission, electrical signals are communicated as an ionic current flow across ELECTRICAL SYNAPSES. Neural Transmission,Neurotransmission,Transmission, Neural,Transmission, Synaptic
D009474 Neurons The basic cellular units of nervous tissue. Each neuron consists of a body, an axon, and dendrites. Their purpose is to receive, conduct, and transmit impulses in the NERVOUS SYSTEM. Nerve Cells,Cell, Nerve,Cells, Nerve,Nerve Cell,Neuron
D012054 Reinforcement, Psychology The strengthening of a conditioned response. Negative Reinforcement,Positive Reinforcement,Psychological Reinforcement,Reinforcement (Psychology),Negative Reinforcements,Positive Reinforcements,Psychological Reinforcements,Psychology Reinforcement,Psychology Reinforcements,Reinforcement, Negative,Reinforcement, Positive,Reinforcement, Psychological,Reinforcements (Psychology),Reinforcements, Negative,Reinforcements, Positive,Reinforcements, Psychological,Reinforcements, Psychology
D002755 Choice Behavior The act of making a selection among two or more alternatives, usually after a period of deliberation. Approach Behavior,Approach Behaviors,Behavior, Approach,Behavior, Choice,Behaviors, Approach,Behaviors, Choice,Choice Behaviors
D005716 Game Theory Theoretical construct used in applied mathematics to analyze certain situations in which there is an interplay between parties that may have similar, opposed, or mixed interests. In a typical game, decision-making "players," who each have their own goals, try to gain advantage over the other parties by anticipating each other's decisions; the game is finally resolved as a consequence of the players' decisions. Game Theories,Theories, Game,Theory, Game
D006801 Humans Members of the species Homo sapiens. Homo sapiens,Man (Taxonomy),Human,Man, Modern,Modern Man
D000200 Action Potentials Abrupt changes in the membrane potential that sweep along the CELL MEMBRANE of excitable cells in response to excitation stimuli. Spike Potentials,Nerve Impulses,Action Potential,Impulse, Nerve,Impulses, Nerve,Nerve Impulse,Potential, Action,Potential, Spike,Potentials, Action,Potentials, Spike,Spike Potential
D000818 Animals Unicellular or multicellular, heterotrophic organisms, that have sensation and the power of voluntary movement. Under the older five kingdom paradigm, Animalia was one of the kingdoms. Under the modern three domain model, Animalia represents one of the many groups in the domain EUKARYOTA. Animal,Metazoa,Animalia

Related Publications

Chris Christodoulou, and Gaye Banfield, and Aristodemos Cleanthous
September 2021, Science robotics,
Chris Christodoulou, and Gaye Banfield, and Aristodemos Cleanthous
January 2021, Frontiers in neuroscience,
Chris Christodoulou, and Gaye Banfield, and Aristodemos Cleanthous
August 2009, International journal of neural systems,
Chris Christodoulou, and Gaye Banfield, and Aristodemos Cleanthous
April 2024, Journal of mathematical biology,
Chris Christodoulou, and Gaye Banfield, and Aristodemos Cleanthous
March 2021, Scientific reports,
Chris Christodoulou, and Gaye Banfield, and Aristodemos Cleanthous
August 2023, IEEE transactions on pattern analysis and machine intelligence,
Chris Christodoulou, and Gaye Banfield, and Aristodemos Cleanthous
January 1984, Sheng li ke xue jin zhan [Progress in physiology],
Chris Christodoulou, and Gaye Banfield, and Aristodemos Cleanthous
January 2012, Frontiers in computational neuroscience,
Chris Christodoulou, and Gaye Banfield, and Aristodemos Cleanthous
October 1989, Physical review. A, General physics,
Chris Christodoulou, and Gaye Banfield, and Aristodemos Cleanthous
August 2018, IEEE transactions on cognitive communications and networking,
Copied contents to your clipboard!