Deep learning for hybrid EEG-fNIRS brain-computer interface: application to motor imagery classification. 2018

Antonio Maria Chiarelli, and Pierpaolo Croce, and Arcangelo Merla, and Filippo Zappasodi
Department of Neuroscience, Imaging and Clinical Sciences, 'G. d'Annunzio' University, Chieti, Italy. Institute of Advanced Biomedical Technologies, 'G. d'Annunzio' University, Chieti, Italy.

Brain-computer interface (BCI) refers to procedures that link the central nervous system to a device. BCI was historically performed using electroencephalography (EEG). In the last years, encouraging results were obtained by combining EEG with other neuroimaging technologies, such as functional near infrared spectroscopy (fNIRS). A crucial step of BCI is brain state classification from recorded signal features. Deep artificial neural networks (DNNs) recently reached unprecedented complex classification outcomes. These performances were achieved through increased computational power, efficient learning algorithms, valuable activation functions, and restricted or back-fed neurons connections. By expecting significant overall BCI performances, we investigated the capabilities of combining EEG and fNIRS recordings with state-of-the-art deep learning procedures. We performed a guided left and right hand motor imagery task on 15 subjects with a fixed classification response time of 1 s and overall experiment length of 10 min. Left versus right classification accuracy of a DNN in the multi-modal recording modality was estimated and it was compared to standalone EEG and fNIRS and other classifiers. At a group level we obtained significant increase in performance when considering multi-modal recordings and DNN classifier with synergistic effect. BCI performances can be significantly improved by employing multi-modal recordings that provide electrical and hemodynamic brain activity information, in combination with advanced non-linear deep learning classification procedures.

UI MeSH Term Description Entries
D007092 Imagination A new pattern of perceptual or ideational material derived from past experience. Imaginations
D008297 Male Males
D009044 Motor Cortex Area of the FRONTAL LOBE concerned with primary motor control located in the dorsal PRECENTRAL GYRUS immediately anterior to the central sulcus. It is comprised of three areas: the primary motor cortex located on the anterior paracentral lobule on the medial surface of the brain; the premotor cortex located anterior to the primary motor cortex; and the supplementary motor area located on the midline surface of the hemisphere anterior to the primary motor cortex. Brodmann Area 4,Brodmann Area 6,Brodmann's Area 4,Brodmann's Area 6,Premotor Cortex and Supplementary Motor Cortex,Premotor and Supplementary Motor Cortices,Anterior Central Gyrus,Gyrus Precentralis,Motor Area,Motor Strip,Precentral Gyrus,Precentral Motor Area,Precentral Motor Cortex,Premotor Area,Premotor Cortex,Primary Motor Area,Primary Motor Cortex,Secondary Motor Areas,Secondary Motor Cortex,Somatic Motor Areas,Somatomotor Areas,Supplementary Motor Area,Area 4, Brodmann,Area 4, Brodmann's,Area 6, Brodmann,Area 6, Brodmann's,Area, Motor,Area, Precentral Motor,Area, Premotor,Area, Primary Motor,Area, Secondary Motor,Area, Somatic Motor,Area, Somatomotor,Area, Supplementary Motor,Brodmann's Area 6s,Brodmanns Area 4,Brodmanns Area 6,Central Gyrus, Anterior,Cortex, Motor,Cortex, Precentral Motor,Cortex, Premotor,Cortex, Primary Motor,Cortex, Secondary Motor,Cortices, Secondary Motor,Gyrus, Anterior Central,Gyrus, Precentral,Motor Area, Precentral,Motor Area, Primary,Motor Area, Secondary,Motor Area, Somatic,Motor Areas,Motor Cortex, Precentral,Motor Cortex, Primary,Motor Cortex, Secondary,Motor Strips,Precentral Motor Areas,Precentral Motor Cortices,Premotor Areas,Primary Motor Areas,Primary Motor Cortices,Secondary Motor Area,Secondary Motor Cortices,Somatic Motor Area,Somatomotor Area,Supplementary Motor Areas
D009068 Movement The act, process, or result of passing from one place or position to another. It differs from LOCOMOTION in that locomotion is restricted to the passing of the whole body from one place to another, while movement encompasses both locomotion but also a change of the position of the whole body or any of its parts. Movement may be used with reference to humans, vertebrate and invertebrate animals, and microorganisms. Differentiate also from MOTOR ACTIVITY, movement associated with behavior. Movements
D011597 Psychomotor Performance The coordination of a sensory or ideational (cognitive) process and a motor activity. Perceptual Motor Performance,Sensory Motor Performance,Visual Motor Coordination,Coordination, Visual Motor,Coordinations, Visual Motor,Motor Coordination, Visual,Motor Coordinations, Visual,Motor Performance, Perceptual,Motor Performance, Sensory,Motor Performances, Perceptual,Motor Performances, Sensory,Perceptual Motor Performances,Performance, Perceptual Motor,Performance, Psychomotor,Performance, Sensory Motor,Performances, Perceptual Motor,Performances, Psychomotor,Performances, Sensory Motor,Psychomotor Performances,Sensory Motor Performances,Visual Motor Coordinations
D004569 Electroencephalography Recording of electric currents developed in the brain by means of electrodes applied to the scalp, to the surface of the brain, or placed within the substance of the brain. EEG,Electroencephalogram,Electroencephalograms
D006801 Humans Members of the species Homo sapiens. Homo sapiens,Man (Taxonomy),Human,Man, Modern,Modern Man
D000077321 Deep Learning Supervised or unsupervised machine learning methods that use multiple layers of data representations generated by nonlinear transformations, instead of individual task-specific ALGORITHMS, to build and train neural network models. Hierarchical Learning,Learning, Deep,Learning, Hierarchical
D000161 Acoustic Stimulation Use of sound to elicit a response in the nervous system. Auditory Stimulation,Stimulation, Acoustic,Stimulation, Auditory
D000328 Adult A person having attained full growth or maturity. Adults are of 19 through 44 years of age. For a person between 19 and 24 years of age, YOUNG ADULT is available. Adults

Related Publications

Antonio Maria Chiarelli, and Pierpaolo Croce, and Arcangelo Merla, and Filippo Zappasodi
July 2019, Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference,
Antonio Maria Chiarelli, and Pierpaolo Croce, and Arcangelo Merla, and Filippo Zappasodi
March 2022, Sensors (Basel, Switzerland),
Antonio Maria Chiarelli, and Pierpaolo Croce, and Arcangelo Merla, and Filippo Zappasodi
July 2020, Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference,
Antonio Maria Chiarelli, and Pierpaolo Croce, and Arcangelo Merla, and Filippo Zappasodi
January 2016, PloS one,
Antonio Maria Chiarelli, and Pierpaolo Croce, and Arcangelo Merla, and Filippo Zappasodi
January 2022, IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society,
Antonio Maria Chiarelli, and Pierpaolo Croce, and Arcangelo Merla, and Filippo Zappasodi
July 2017, GigaScience,
Antonio Maria Chiarelli, and Pierpaolo Croce, and Arcangelo Merla, and Filippo Zappasodi
February 2019, International journal of neural systems,
Antonio Maria Chiarelli, and Pierpaolo Croce, and Arcangelo Merla, and Filippo Zappasodi
January 2022, Journal of medical signals and sensors,
Antonio Maria Chiarelli, and Pierpaolo Croce, and Arcangelo Merla, and Filippo Zappasodi
January 2022, Journal of neural engineering,
Antonio Maria Chiarelli, and Pierpaolo Croce, and Arcangelo Merla, and Filippo Zappasodi
April 2015, Journal of neuroscience methods,
Copied contents to your clipboard!