Real-time decision fusion for multimodal neural prosthetic devices. 2010

James Robert White, and Todd Levy, and William Bishop, and James D Beaty
Applied Mathematics and Scientific Computation Program, University of Maryland-College Park, College Park, Maryland, United States of America. whitej@umd.edu

BACKGROUND The field of neural prosthetics aims to develop prosthetic limbs with a brain-computer interface (BCI) through which neural activity is decoded into movements. A natural extension of current research is the incorporation of neural activity from multiple modalities to more accurately estimate the user's intent. The challenge remains how to appropriately combine this information in real-time for a neural prosthetic device. RESULTS Here we propose a framework based on decision fusion, i.e., fusing predictions from several single-modality decoders to produce a more accurate device state estimate. We examine two algorithms for continuous variable decision fusion: the Kalman filter and artificial neural networks (ANNs). Using simulated cortical neural spike signals, we implemented several successful individual neural decoding algorithms, and tested the capabilities of each fusion method in the context of decoding 2-dimensional endpoint trajectories of a neural prosthetic arm. Extensively testing these methods on random trajectories, we find that on average both the Kalman filter and ANNs successfully fuse the individual decoder estimates to produce more accurate predictions. CONCLUSIONS Our results reveal that a fusion-based approach has the potential to improve prediction accuracy over individual decoders of varying quality, and we hope that this work will encourage multimodal neural prosthetics experiments in the future.

UI MeSH Term Description Entries
D008328 Man-Machine Systems A system in which the functions of the man and the machine are interrelated and necessary for the operation of the system. Man Machine Systems,Man-Machine System,System, Man-Machine,Systems, Man-Machine
D008959 Models, Neurological Theoretical representations that simulate the behavior or activity of the neurological system, processes or phenomena; includes the use of mathematical equations, computers, and other electronic equipment. Neurologic Models,Model, Neurological,Neurologic Model,Neurological Model,Neurological Models,Model, Neurologic,Models, Neurologic
D009044 Motor Cortex Area of the FRONTAL LOBE concerned with primary motor control located in the dorsal PRECENTRAL GYRUS immediately anterior to the central sulcus. It is comprised of three areas: the primary motor cortex located on the anterior paracentral lobule on the medial surface of the brain; the premotor cortex located anterior to the primary motor cortex; and the supplementary motor area located on the midline surface of the hemisphere anterior to the primary motor cortex. Brodmann Area 4,Brodmann Area 6,Brodmann's Area 4,Brodmann's Area 6,Premotor Cortex and Supplementary Motor Cortex,Premotor and Supplementary Motor Cortices,Anterior Central Gyrus,Gyrus Precentralis,Motor Area,Motor Strip,Precentral Gyrus,Precentral Motor Area,Precentral Motor Cortex,Premotor Area,Premotor Cortex,Primary Motor Area,Primary Motor Cortex,Secondary Motor Areas,Secondary Motor Cortex,Somatic Motor Areas,Somatomotor Areas,Supplementary Motor Area,Area 4, Brodmann,Area 4, Brodmann's,Area 6, Brodmann,Area 6, Brodmann's,Area, Motor,Area, Precentral Motor,Area, Premotor,Area, Primary Motor,Area, Secondary Motor,Area, Somatic Motor,Area, Somatomotor,Area, Supplementary Motor,Brodmann's Area 6s,Brodmanns Area 4,Brodmanns Area 6,Central Gyrus, Anterior,Cortex, Motor,Cortex, Precentral Motor,Cortex, Premotor,Cortex, Primary Motor,Cortex, Secondary Motor,Cortices, Secondary Motor,Gyrus, Anterior Central,Gyrus, Precentral,Motor Area, Precentral,Motor Area, Primary,Motor Area, Secondary,Motor Area, Somatic,Motor Areas,Motor Cortex, Precentral,Motor Cortex, Primary,Motor Cortex, Secondary,Motor Strips,Precentral Motor Areas,Precentral Motor Cortices,Premotor Areas,Primary Motor Areas,Primary Motor Cortices,Secondary Motor Area,Secondary Motor Cortices,Somatic Motor Area,Somatomotor Area,Supplementary Motor Areas
D009068 Movement The act, process, or result of passing from one place or position to another. It differs from LOCOMOTION in that locomotion is restricted to the passing of the whole body from one place to another, while movement encompasses both locomotion but also a change of the position of the whole body or any of its parts. Movement may be used with reference to humans, vertebrate and invertebrate animals, and microorganisms. Differentiate also from MOTOR ACTIVITY, movement associated with behavior. Movements
D011474 Prosthesis Design The plan and delineation of prostheses in general or a specific prosthesis. Design, Prosthesis,Designs, Prosthesis,Prosthesis Designs
D005121 Extremities The farthest or outermost projections of the body, such as the HAND and FOOT. Limbs,Extremity,Limb
D006801 Humans Members of the species Homo sapiens. Homo sapiens,Man (Taxonomy),Human,Man, Modern,Modern Man
D000465 Algorithms A procedure consisting of a sequence of algebraic formulas and/or logical steps to calculate or determine a given task. Algorithm
D012984 Software Sequential operating programs and data which instruct the functioning of a digital computer. Computer Programs,Computer Software,Open Source Software,Software Engineering,Software Tools,Computer Applications Software,Computer Programs and Programming,Computer Software Applications,Application, Computer Software,Applications Software, Computer,Applications Softwares, Computer,Applications, Computer Software,Computer Applications Softwares,Computer Program,Computer Software Application,Engineering, Software,Open Source Softwares,Program, Computer,Programs, Computer,Software Application, Computer,Software Applications, Computer,Software Tool,Software, Computer,Software, Computer Applications,Software, Open Source,Softwares, Computer Applications,Softwares, Open Source,Source Software, Open,Source Softwares, Open,Tool, Software,Tools, Software
D014584 User-Computer Interface The portion of an interactive computer program that issues messages to and receives commands from a user. Interface, User Computer,Virtual Systems,User Computer Interface,Interface, User-Computer,Interfaces, User Computer,Interfaces, User-Computer,System, Virtual,Systems, Virtual,User Computer Interfaces,User-Computer Interfaces,Virtual System

Related Publications

James Robert White, and Todd Levy, and William Bishop, and James D Beaty
December 2000, Current opinion in neurology,
James Robert White, and Todd Levy, and William Bishop, and James D Beaty
May 2017, IEEE journal of biomedical and health informatics,
James Robert White, and Todd Levy, and William Bishop, and James D Beaty
June 2024, Scientific reports,
James Robert White, and Todd Levy, and William Bishop, and James D Beaty
March 2023, IEEE transactions on visualization and computer graphics,
James Robert White, and Todd Levy, and William Bishop, and James D Beaty
October 2007, Journal of neurophysiology,
James Robert White, and Todd Levy, and William Bishop, and James D Beaty
December 2008, Journal of neurophysiology,
James Robert White, and Todd Levy, and William Bishop, and James D Beaty
February 2019, Journal of neural engineering,
James Robert White, and Todd Levy, and William Bishop, and James D Beaty
September 2023, Mathematical biosciences and engineering : MBE,
James Robert White, and Todd Levy, and William Bishop, and James D Beaty
January 1971, JAAMI : journal of the Association for the Advancement of Medical Instrumentation,
James Robert White, and Todd Levy, and William Bishop, and James D Beaty
July 2012, Nano letters,
Copied contents to your clipboard!