• Open Access

Collaborative Approach in the Development of High-Performance Brain–Computer Interfaces for a Neuroprosthetic Arm: Translation from Animal Models to Human Control

Authors

  • Jennifer L. Collinger Ph.D.,

    1. Department of Veterans Affairs Medical Center, Pittsburgh, Pennsylvania, USA
    2. Department of Physical Medicine & Rehabilitation
    3. Department of Bioengineering
    Search for more papers by this author
  • Michael A. Kryger M.D., M.S.,

    1. Department of Physical Medicine & Rehabilitation
    Search for more papers by this author
  • Richard Barbara Ph.D.,

    1. Department of Physical Medicine & Rehabilitation
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Timothy Betler M.A.,

    1. Schools of the Health Sciences Media Relations, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Kristen Bowsher Ph.D.,

    1. Office of Device Evaluation, Food and Drug Administration, Silver Spring, Maryland, USA
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Elke H. P. Brown M.D.,

    1. Department of Physical Medicine & Rehabilitation
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Samuel T. Clanton Ph.D.,

    1. Department of Physical Medicine & Rehabilitation
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Alan D. Degenhart B.S.,

    1. Department of Bioengineering
    2. Center for the Neural Basis of Cognition, Carnegie Mellon University and University of Pittsburgh, Pittsburgh, Pennsylvania, USA
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Stephen T. Foldes Ph.D.,

    1. Department of Veterans Affairs Medical Center, Pittsburgh, Pennsylvania, USA
    2. Department of Physical Medicine & Rehabilitation
    3. Center for the Neural Basis of Cognition, Carnegie Mellon University and University of Pittsburgh, Pittsburgh, Pennsylvania, USA
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Robert A. Gaunt Ph.D.,

    1. Department of Physical Medicine & Rehabilitation
    2. Department of Bioengineering
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Ferenc E. Gyulai M.D.,

    1. Department of Anesthesiology, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Elizabeth A. Harchick B.S.,

    1. Department of Physical Medicine & Rehabilitation
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Deborah Harrington B.S.,

    1. Department of Physical Medicine & Rehabilitation
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • John B. Helder M.S.,

    1. The Johns Hopkins University Applied Physics Laboratory, Laurel, Maryland, USA
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Timothy Hemmes,

    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Matthew S. Johannes Ph.D.,

    1. The Johns Hopkins University Applied Physics Laboratory, Laurel, Maryland, USA
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Kapil D. Katyal M.S.,

    1. The Johns Hopkins University Applied Physics Laboratory, Laurel, Maryland, USA
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Geoffrey S. F. Ling M.D., Ph.D., F.A.A.N.,

    1. Defense Advanced Research Projects Agency, Arlington, Virginia, USA
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Angus J. C. McMorland Ph.D.,

    1. Systems Neuroscience Institute
    2. Department of Neurobiology
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Karina Palko B.S.,

    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Matthew P. Para M.E.,

    1. The Johns Hopkins University Applied Physics Laboratory, Laurel, Maryland, USA
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Janet Scheuermann B.A.,

    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Andrew B. Schwartz Ph.D.,

    1. Department of Physical Medicine & Rehabilitation
    2. Department of Bioengineering
    3. Center for the Neural Basis of Cognition, Carnegie Mellon University and University of Pittsburgh, Pittsburgh, Pennsylvania, USA
    4. Systems Neuroscience Institute
    5. Department of Neurobiology
    6. McGowan Institute for Regenerative Medicine
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Elizabeth R. Skidmore Ph.D., O.T.R./L.,

    1. Department of Physical Medicine & Rehabilitation
    2. Department of Occupational Therapy, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Florian Solzbacher Ph.D.,

    1. Department of Electrical and Computer Engineering
    2. Department of Bioengineering
    3. Department of Materials Science and Engineering, University of Utah, Salt Lake City, Utah, USA
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Anita V. Srikameswaran M.D.,

    1. Schools of the Health Sciences Media Relations, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Dennis P. Swanson R.Ph., M.S.,

    1. Research Conduct and Compliance Office
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Scott Swetz M.S.,

    1. The Johns Hopkins University Applied Physics Laboratory, Laurel, Maryland, USA
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Elizabeth C. Tyler-Kabara M.D., Ph.D.,

    1. Department of Physical Medicine & Rehabilitation
    2. Department of Bioengineering
    3. McGowan Institute for Regenerative Medicine
    4. Department of Neurological Surgery
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Meel Velliste Ph.D.,

    1. Systems Neuroscience Institute
    2. Department of Neurobiology
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Wei Wang M.D., Ph.D.,

    1. Department of Physical Medicine & Rehabilitation
    2. Department of Bioengineering
    3. Center for the Neural Basis of Cognition, Carnegie Mellon University and University of Pittsburgh, Pittsburgh, Pennsylvania, USA
    4. Clinical and Translational Neuroscience Institute, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Douglas J. Weber Ph.D.,

    1. Department of Veterans Affairs Medical Center, Pittsburgh, Pennsylvania, USA
    2. Department of Physical Medicine & Rehabilitation
    3. Department of Bioengineering
    4. Center for the Neural Basis of Cognition, Carnegie Mellon University and University of Pittsburgh, Pittsburgh, Pennsylvania, USA
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Brian Wodlinger Ph.D.,

    1. Department of Physical Medicine & Rehabilitation
    2. Center for the Neural Basis of Cognition, Carnegie Mellon University and University of Pittsburgh, Pittsburgh, Pennsylvania, USA
    Search for more papers by this author
    • Equal contributors to this article, in alphabetical order.

  • Michael L. Boninger M.D.

    Corresponding author
    1. Department of Veterans Affairs Medical Center, Pittsburgh, Pennsylvania, USA
    2. Department of Physical Medicine & Rehabilitation
    3. Department of Bioengineering
    4. McGowan Institute for Regenerative Medicine
    Search for more papers by this author

Abstract

Our research group recently demonstrated that a person with tetraplegia could use a brain–computer interface (BCI) to control a sophisticated anthropomorphic robotic arm with skill and speed approaching that of an able-bodied person. This multiyear study exemplifies important principles in translating research from foundational theory and animal experiments into a clinical study. We present a roadmap that may serve as an example for other areas of clinical device research as well as an update on study results. Prior to conducting a multiyear clinical trial, years of animal research preceded BCI testing in an epilepsy monitoring unit, and then in a short-term (28 days) clinical investigation. Scientists and engineers developed the necessary robotic and surgical hardware, software environment, data analysis techniques, and training paradigms. Coordination among researchers, funding institutes, and regulatory bodies ensured that the study would provide valuable scientific information in a safe environment for the study participant. Finally, clinicians from neurosurgery, anesthesiology, physiatry, psychology, and occupational therapy all worked in a multidisciplinary team along with the other researchers to conduct a multiyear BCI clinical study. This teamwork and coordination can be used as a model for others attempting to translate basic science into real-world clinical situations.

Introduction

Brain–computer interfaces (BCIs) are systems that allow signals from the cerebral cortex to be recorded and utilized by a computer system for the purpose of controlling assistive technology. More than 40 years of basic science research by neurophysiologists and biomedical engineers have advanced the understanding of motor cortex physiology and the idea of neural control in animal models.[1-9] This has enabled nonhuman primates to control computer cursors,[10] functional electrical stimulation of forearm muscles for grasping,[11] and a robotic arm to perform self-feeding.[12] Motor BCIs have the potential to assist veterans arriving home with disabling injuries and lost limbs, as well as the thousands of individuals who acquire spinal cord injuries or lose limbs through accidents or neurologic disorders.[13-15] However it is a complex and expensive mission to bring all of the tools and professionals together to translate BCI technology to clinical application. As a result, there have been very few clinical trials with implanted BCIs in people with motor impairments.[16-22]

At the University of Pittsburgh, our focus is on upper limb motor neuroprostheses, which aim to restore movement and function. We are interested in developing a BCI to provide intuitive and natural control of a robotic upper limb. We have followed a multistep approach to move from basic science research to long-term clinical trials of implanted BCI technology. To date, we have demonstrated three degrees of freedom (DOF) control using electrocorticography (ECoG) in a short-term study (<30 days) as well as seven DOF control of an anthropomorphic sophisticated prosthetic arm using intracortical microelectrode arrays (MEAs).[19, 20] In this manuscript, we describe a roadmap of resources and procedures that we believe are pertinent to clinical trials of implanted neuroprosthetic devices. Larger, multisite investigations and improvements in this technology will be necessary in order to translate this technology to individuals with disabilities.

Summary of Translational Approach

In ECoG studies, initial human investigations, including real-time BCI and neuroscience experiments, were conducted in patients being observed in an epilepsy monitoring unit (EMU).[23-25] ECoG during epilepsy monitoring has been used by a number of research groups to enable the development of software and training techniques with human participants.[26-31] Following the studies in epilepsy patients, the next step in our BCI research pathway was to conduct a 28-day study of ECoG-based BCI in an individual with spinal cord injury.[20] This represented a critical translation of BCI technology into the targeted patient population. For the MEA investigation, we obtained Investigational Device Exemption (IDE) approval from the Food and Drug Administration (FDA) to conduct a long-term (<5 years) study in individuals with tetraplegia. This multiyear study required substantial collaboration with the government, that not only provided funding, but also launched a vast collaborative effort that brought together institutions across the country that have developed advanced robotic technology and neural recording devices.

This paper aims to describe key elements of our translational research project, specifically focused on the collaboration between various teams and disciplines. This will include a discussion of the foundational elements that led to the multiyear intracortical BCI study, including animal models, the Modular Prosthetic Limb (MPL), EMU research, and the 28-day ECoG Study. We will then describe the various components necessary in translating this work for the MEA BCI study, including funding, IDE preparation, hurdles in translation, and participant recruitment. Next we detail clinical aspects of the study, highlighting considerations in imaging, surgery, anesthesia, and psychiatry. The paper then summarizes the experimental trials, including the virtual environment, training, and outcome measurements. We conclude with the participants’ perspectives, and results from the MEA BCI study.

Foundational Studies

Nonhuman primate investigations of neural control

The science behind prosthetic control originated from the seminal findings of directional tuning[2] and population coding,[3] which elucidated the relationship between neuronal firing and the direction of limb movements. Since then, three decades of monkey experiments have paved the way for human trials, including studies on continuous trajectory decoding,[10, 32] brain-controlled reaching in a virtual environment,[4] and a self-feeding experiment[12] in which a monkey learned to reach and grasp using a robotic arm in four DOF. The specific methodology, including algorithms, real-time software, and training protocols culminated in 2010 with a PhD thesis[33] that demonstrated for the first time seven DOF robotic control in a nonhuman primate model by adding three DOF control of hand orientation. This work included a system for calibration and training of nonhuman primates using shared control, a training process by which the subject is progressively given more control of a robotic arm beginning with calibration from observed robotic arm movements.[19, 33] These animal studies demonstrated that intuitive control using cortical signals was possible in real time, and formed the foundation for an effort to translate this technology into human subjects.

ECoG studies in the EMU

Developing experience with implanted neural recording in human subjects was crucial to improving our scientific understanding and refining our protocols. An ideal patient population is one that is undergoing ECoG monitoring for clinical brain mapping in EMUs. Our EMU studies set the foundation for the BCI work in many ways. Because of its multidisciplinary nature, the EMU work accelerated the integration of team members with diverse backgrounds, allowing us to design experiments and solve problems by considering future research questions, technical capabilities and constraints, and clinical practicality. This work guided development of subsequent human study protocols and IDE applications. Finally, the EMU study generated critical pilot data[23-25, 34, 35] and initial funding from multiple agencies that made it possible for us to form and grow our clinical BCI research team with a demonstrated track record. The EMU research conducted in patients undergoing clinical brain mapping in itself has important scientific, engineering, and clinical value as exemplified by the ever-increasing number of research groups and publications based on this study setting.[23-31, 34-43]

Short-term ECoG study in spinal cord injury

ECoG studies in clinical patients,[39, 44] as well as nonhuman primate investigations of ECoG,[45, 46] provided a foundation for translating ECoG into the targeted patient population. In August 2011, with institutional support from UPMC and the University of Pittsburgh, and funding from National Institutes of Health (NIH) and the Craig Neilsen Foundation, a 30-year-old male with tetraplegia was implanted with a high-density ECoG grid over sensorimotor cortex for 28 days.[20] The participant was able to move a cursor toward a target in two dimensions with an 87% success rate. The participant also attempted 3D control, and steadily improved performance up to 80% after only a few days. While significant in its own right, this short-term ECoG study also allowed us to pilot and refine a number of the principles and technologies required for longer term investigations, including presurgical mapping, software development, and training strategies. This stepping stone also provided evidence to encourage further funding support and attract future study participants. Most importantly, it provided the team with experience working with a participant with paralysis, which required us to make accommodations related to accessibility, necessitated the use of training strategies that did not require overt movement, and allowed us to experience the amount of motivation and dedication required from successful study participants.

Regulatory and technical preparations

DARPA revolutionizing prosthetics program

The long-term MEA study was primarily funded by the Defense Advanced Research Projects Agency (DARPA), which through a primary contract with Johns Hopkins University Applied Physics Laboratory (JHU/APL), brought Blackrock Microsystems (Salt Lake City, UT, USA) and their relationship with the FDA to the team. As part of the Revolutionizing Prosthetics program, DARPA funded two research teams at the DEKA Research and Development Corporation and JHU/APL to design and build advanced upper extremity prostheses that enabled near-natural movement.[47, 48] The DARPA team brought together hundreds of professionals across the country. DARPA also assisted by encouraging support through the NIH and Veterans Administration, both of whom contributed to this study through resources and grants.

Modular prosthetic limb (MPL)

The training paradigm discovered in the animal work required observation of movement.[12] However, available prosthetics were not capable of producing high-dimensional, natural movements. The MPL (Figure 1) is an advanced upper-limb prosthetic designed to mimic the capabilities, form factor, and function of the human arm and hand.[49] The MPL emulates the weight, volumetric envelope, speed, torque, and range of motion abilities of its human counterpart, with 17-controllable DOF and 26 articulating joints. This anthropomorphic design and appearance of the MPL provides lifelike movement, and may convey a feeling of embodiment to users.

Figure 1.

JHU/APL modular prosthetic limb (MPL).

The MPL team at JHU/APL collaborated with the University of Pittsburgh team through various developmental planning, system modification and updating, prototyping, and support activities. Initial collaboration focused on defining the user specifications for the MPL as well as system integration within the experimental framework. This primarily focused on physical, electrical, and logical integration of the system logistically within the clinical environment. Software and control parameters of the MPL system were optimized to participant-decoded intent signals during the experiments. Much of this optimization was conducted through nonhuman primate testing, which facilitated quick implementation during the clinical trial. Safety features including limited velocity regions and customizable workspace exclusion were developed and rigorously tested to allow for experiments involving interaction between the participant and the MPL. Frequent practice sessions with the BCI are critical to experiment success, and thus minimizing hardware and software malfunctions was essential. This was achieved through quick turnaround repair efforts and hardware swapping with backup assemblies.

Virtual integration environment (VIE)

To allow offline training by the participant without the use of the physical MPL, a virtual reality framework called the VIE was developed by JHU/APL. The VIE visually and functionally simulates the MPL and provides a surrounding environment filled with everyday objects such as cups, books, and tables. This virtual MPL utilizes the same control interface as the physical MPL, allowing it to be used alongside or instead of the MPL during neural training.[50, 51] Because the VIE system is executed on a standard personal computer and requires minimal support, it was usable both in the laboratory and in the participant's home.

The VIE (Figure 2 and Video 1) was built using Unity3D (Unity Technologies, San Francisco, CA, USA), a game development environment, which allowed for rapid development of new scenarios that incorporated virtual objects with properties similar to their real counterparts. Custom objects that closely match those in real life were developed including those used in tests of upper limb function.[52] Participant-requested games that were both engaging to the participant and complementary to the training paradigm were also developed. The games allowed for longitudinal “scoring” and evaluation of the subject's performance using quantitative metrics.

Figure 2.

Example scene in the Virtual Integration Environment (VIE).

Regulatory process

Clinical trials of significant risk devices not cleared for marketing require the submission of an IDE and prospective approval by the FDA.[53] Institutional Review Board (IRB) approval must also be obtained for the corresponding clinical investigation. Many neural recording electrodes have premarket approval or 501 (k) clearance from the FDA with specific indications for use and allowable duration of use. While this process allows for leveraging existing FDA cleared devices without requiring as robust a statistical analysis of safety and efficacy data, an IDE must still be submitted in order to use devices for nonapproved indications. All IDEs have three main components: a report of prior research, an investigational plan, and a device description. Applications are evaluated to ensure that the device meets appropriate manufacturing specifications, the investigational plan is sound, the device has the potential to be effective, and that the risks of the study do not outweigh the potential benefit to participants or knowledge to be gained. In our case, prior research included laboratory, animal, and clinical testing of the intracortical MEA, which address safety and efficacy of the device. The investigational plan made up the majority of the document and included the purpose of the investigation, a detailed description of the device, the clinical protocol, informed consent documentation, and a risk analysis. Creating a clinical protocol and IDE that maximized the knowledge to be gained, while minimizing risks to participants required the team's diverse expertise. The device description detailed the safety measures associated with the manufacturing process, including methods, facilities, manufacturing, processing, packing, and storage. Detailing device information required close cooperation with an industry partner, Blackrock Microsystems, who supplied the 510k-cleared MEA (Figure 3). They provided written permission to the FDA to access their 510 (k) application for manufacturing information in support of our IDE application. Without cooperation from the device company, independent bench and preclinical testing of the device would have been required.

Figure 3.

Neuroport intracortical microelectrode array (MEA). Used with permission from Blackrock Microsystems.

Important to our success was the close working relationship between the FDA and DARPA that began in 2006. In 2011, the FDA selected the Revolutionizing Prosthetics program as the pilot program for the FDA Innovation Initiative. The FDA established a transparent and open communication channel that encouraged discussion of requirements to gain approval for technologies and processes related to human trials. As a result of this initiative, our team was able to obtain regulatory approvals efficiently without the need for multiple rounds of submission. Further, the University of Pittsburgh Office for Investigator-Sponsored Investigational New Drug and IDE Support provided presubmission reviews and assistance with the submission process. In parallel with FDA discussions, our team held regular meetings with the University of Pittsburgh IRB and members of an independent Data and Safety Monitoring Board. Thus, long before the documentation was submitted, the regulatory personnel provided feedback on the protocol, again reducing the number of submissions. This early involvement was facilitated by the Clinical and Translational Science Institute at the University of Pittsburgh, which is an NIH-funded effort to improve the efficiency with which biomedical advances are translated to patients.

Leveraging basic science resources

Nonhuman primate studies of a BCI-controlled MPL were conducted prior to beginning the clinical trial.[12, 33] As previously mentioned, these studies formed the scientific basis for the training paradigms used in the human trials and also provided an estimate of the number of neural units needed for high-dimensional control of the MPL. The human BCI software system utilizes the Real Time Messaging Architecture developed for earlier nonhuman primate experiments. The software was modified to provide extra flexibility in paradigm design allowing the participant to complete a larger variety of tasks with and without the MPL. This proved crucial as we routinely implemented new tests and training exercises throughout the study. We also used a flexible graphical user interface to tune computer-assist parameters[12, 19] throughout each test session in order to keep the participant motivated.

Translating this work to humans revealed several challenges not observed in the nonhuman primate work. First, it was discovered that the decoding algorithm needed to be more robust to provide good control over our longer testing sessions, especially after meals or other breaks. Most nonhuman primate experiments last 1–2 hours, while our human sessions were typically 4 hours long. Second, we found that BCI performance on the training task did not always generalize to good performance over a larger variety of tasks. These challenges were addressed by adding a regularization stage to the decoding algorithm[54] and modifying our training paradigm to include object interactions. The regularization stage helped prevent over-fitting to noise and other signals that improve performance only slightly but are likely to change over time. These changes not only improved the quality of our results but made testing sessions easier and more enjoyable for both participants and researchers.

Participant recruitment

At the core of any successful study are motivated individuals willing to serve as study participants. Given the very limited number of participants to be tested, their efforts have the power to determine the success of the study. Recruiting participants for clinical trials of neuroprosthetic devices can be challenging. Study participation requires a great commitment, as participants are asked to undergo voluntary neurosurgery and a demanding testing schedule. Strict inclusion/exclusion criteria as defined in the IDE, while necessary to ensure participant safety, limit the recruitment pool significantly. It is essential that the research team and potential participants have an open dialog and that the participants are fully informed about the potential risks associated with the study. For this particular study, participants were told that the BCI was a temporary implant and that they would derive no direct benefit from the study.

Three primary methods of recruitment were used. First, research registries allowed us to contact individuals who met our criteria and had expressed an interest in research. Second, clinicians were asked to refer participants. Finally, we encouraged media coverage of our studies, in particular the short-term ECoG study, to educate and inform the public about the science and technology behind the BCI projects. Media Relations chronicled study events with video of the testing procedures and interviews of the participants and the research team. That video was used by reporters from national and local media outlets and was also used in a story package that was posted with other press materials on the UPMC Website (www.upmc.com/bci). Media outreach conducted in collaboration with the journal press office and a Webcast press conference led to global coverage of the participant and research team's achievements. Since then, multiple potential candidates have contacted the trial coordinator for follow-up studies.

Clinical Considerations

Preoperative imaging

In order to account for functional reorganization secondary to chronic paralysis, it was important that electrode placement target cortical areas active for individual participants.[55] We utilized functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG). FMRI evaluates changes in cerebral perfusion with high spatial resolution. The blood-oxygenation level dependent (BOLD) response is an indirect measurement of neural activity that has been widely used for mapping motor activity. MEG records changes in magnetic fields resulting from neuronal firing, with high temporal resolution. MEG mapping complemented the more standard fMRI results as it is a direct measurement of neural activity. The two imaging modalities were used to identify cortical areas that were active while the participants attempted to mimic videos of repeated hand, wrist, elbow, and shoulder movements, even though they were unable to move their own paralyzed arm. The neuroimaging results were used to guide electrode placement as described later.

Surgical approach

Forming the surgical plan involved collaboration with the basic science team. The neurosurgeon responsible for the human surgery scrubbed in on nonhuman primate operations. This approach allowed the surgeon to learn strategies related to placing the MEA, bending the interconnect, and positioning pedestals within the constraints of the craniotomy and exposed skull. It also provided an opportunity to develop ideas for identifying standard human instruments that could interface with the special instruments used for placing the MEAs for the BCI surgeries. In preparing for the human surgery, special considerations were made involving craniotomy size, external pedestal location, and internal MEA location. We attempted to minimize the size of the craniotomy to prevent infection. The pedestals were secured to the skull, and exit through the scalp to provide the physical interface between the data acquisition cable and the MEA. Pedestal locations were chosen for multiple reasons. It was important to place the pedestals far enough away from the entry site to be able to secure the pedestals to the skull, and far enough away from each other to be able to accommodate the cables that connect the pedestals to the recording hardware. Importantly, the pedestals could not interfere with the participant's ability to position her head on her chair's headrest or a pillow for sleeping. Internal MEA position on the cortex was decided based on the preoperative imaging. Neuroimaging information was made available during the surgery and co-registered to the participant's anatomy in real-time using a surgical navigation system (Brainlab, Westchester, IL, USA) to help guide electrode placement.[56] The functional and anatomic images were reviewed by the entire team and targets were determined as a consensus. This consensus-driven approach for targeted electrode placement was further refined by identifying alternative sites preoperatively in case the initial target site was found to be suboptimal intraoperatively.

The surgery required general anesthesia, and thus we collaborated with an anesthesiologist early in the protocol development process. Pathologies such as tetraplegia, spastic, or flaccid paralysis, autonomic hyperreflexia, receptor up-or-down regulation at the neuromuscular junction, cervical fusions, and tracheostomy[57-65] were evaluated preoperatively and monitored closely intraoperatively, among other comorbidities that have a significant impact on anesthetic management. Collaboration with the anesthesiologist and surgeon allowed for the inclusion of appropriate risk descriptions and mitigation strategies in our clinical protocol.

Psychological support

An important component of the study process was providing the participant with support from a psychologist who specialized in spinal cord injury counseling. Prior to the surgery, the participant was screened using neuropsychological and psychosocial assessments. The goal of the screening was to ensure that the participant had normal cognitive function, realistic expectations about the study, and adequate family and caregiver support. Psychological support was made available during study participation with in-person or phone meetings approximately every 2 months, or whenever the participant thought it was necessary. The participant's mood and psychological well-being were assessed periodically, as well as how the participant handled the demands and expectations of the project. In the short term, the psychologist helped the participant adjust to the spotlight of a high-profile research study. In the long term, the role of the psychologist is to help the participant transition from a demanding research schedule back to every-day life.

BCI training and outcome measures

Frequent practice is key to facilitate learning of a BCI. For the intracortical study, the participant completed three 4-hour sessions per week. We attempted to balance purely target-based tasks that allowed for more quantitative assessments of progress, and activities of daily living (ADLs) tasks, such as feeding. If the participant was frustrated or tired, we could take a break, vary the difficulty of tasks, or switch from a target-based task to a more enjoyable ADL task or game. All tasks were designed to use the same endpoint-velocity control of the MPL or virtual MPL. We also kept track of success rates and scores to provide additional motivation.

An important component of the study was measurement of clinical outcomes associated with the BCI-controlled MPL. In order to use validated metrics, we used tests designed to measure upper limb function. In upper limb rehabilitation, clinical outcomes are characterized in terms of reach, grasp, and manipulation, as observed during standardized laboratory tasks (e.g., picking up and placing an object on a surface) or practical “real-world” tasks (e.g., brushing teeth, eating with a fork). Quality of movement (measured with ordinal scales or performance time) and the amount of use of the impaired limb (measured with ordinal scales or actigraphy) are frequent indicators of limb function. This study measured quality of movement with the Action Research Arm Test (ARAT),[52] an assessment that is traditionally used to measure unilateral upper limb function following a stroke.

Study methods and results

The MEA BCI study is registered at clinicaltrials.gov (http://clinicaltrials.gov/ct2/show/NCT01364480) and was conducted under an IDE granted by the FDA and with approval from the IRB at the University of Pittsburgh and the Space and Naval Warfare Systems Center Pacific. In February 2012, two intracortical MEAs (100 electrode shanks 1.5 mm in length, 4 × 4 mm footprint, Blackrock Microsystems) were implanted in the motor cortex of a 52-year-old woman with spinocerebellar degeneration. As previously published,[19] over 13 weeks of training, the participant learned to control the MPL in seven DOF. By modulating her neural activity, she was able to control the 3D endpoint velocity of the hand, 3D hand/wrist orientation, and 1D grasp, simultaneously. Steady improvements in performance were observed on tasks that required her to reach targets that sampled all seven dimensions. Performance was measured as success rate, completion time, and path efficiency. The participant also used the MPL to perform reach and grasp movements that resulted in clinically significant gains on the ARAT. Operation of the MPL under brain-control resulted in smooth and coordinated movements with speeds approaching that of able-bodied individuals.

Since the time of the original publication, we have worked to extend control to include a variety of hand shapes which would allow for different grasp types, such as a pinch or whole hand grasp (Video 2). The subject can play games involving several hand postures, such as rock-paper-scissors (Video 3). The participant has also used the MPL to accomplish a goal that she set at the beginning of the study: to feed herself chocolate. In the video, the participant is controlling the 3D endpoint velocity of the MPL and has continuous control over a single grasp dimension. Once she hits her target location, the MPL was paused to allow her to take a bite of the chocolate (Video 4). The patient has been implanted with the MEA for over 1 year, a testament to the robustness of the implantation and training system. No adverse events from MEA implantation have been observed to date.

Being a research participant: In their own words

Subjects require a strong support system from family, caregivers, and friends. In fact, such support was an important inclusion criterion of the study, and the neuropsychological assessment evaluated this support system. However, the traditional roles in the laboratory tended to shift; through many hours of cooperation between researchers and subjects, not only did the investigators become a part of the subjects’ support system, but it soon became clear that the research participants were as much team members as the study investigators. The participants offered their view of being collaborators in the studies below.

Tim Hemmes, short-term ECoG study participant

My role in this study was to be completely focused and give the team as clean a signal as possible. At times this was easy, but not always. I have not been able to move my arms for 8 years so the saying “use it or lose it” was really affecting me. It was hard to focus on “moving” my right arm and hand. In my opinion this was the most difficult obstacle to overcome and I compare it to a baby learning to walk. But when the electrodes started lighting up as I was thinking “up, down, left or right,” it made all of my “mental thinking” and exhausting focus worthwhile. (Note: the participant observed a measure of signal strength for each electrode on a computer display during the early phases of training.) My initial fear going into the study was being unable to create the thoughts that we needed. So to see these electrodes light up knowing it was working took a lot of pressure off me, and allowed me to focus even more. During this study, the team and I really started understanding each other and becoming closer on a personal level. Finding the words to explain to the researchers how I was able to overcome every obstacle they asked of me and what I was feeling, I very quickly felt like a team member and not just their “lab monkey.”

Jan Scheuermann, the MEA BCI study participant, and Karina Palko, Jan's attendant

After seeing Tim Hemmes in a video moving the robotic arm, I had no doubts about volunteering. I was fully informed of all the risks, and my attitude was “full speed ahead!” I feel blessed and honored to have the role of the human subject in this study. I was hooked up to the computer that sent signals to the robotic hand, 3 days a week for 4-hour sessions. Moving the hand was easier than I had thought it would be, and every success spurred me on for the next challenge. Karina, my attendant, went with me to feed me and take care of me. Not content to just sit there, she started taking notes that proved to be useful to the team. She also became an integral part of the process of cleaning my pedestals. We have both been very excited to see the triumphs of this project, and we are happy to be part of it.

Conclusions

The MEA BCI study brought together teams and individuals from multiple different disciplines including physiatry, bioengineering, neurosurgery, neurobiology, anesthesiology, psychology, and occupational therapy. The multidisciplinary effort and timeline is summarized in Figure 4. The team was able to work toward a common goal, and decide on achievable clinical research objectives and methods. Open communication and collaboration across groups was essential to the success of the project. Coordination between the University of Pittsburgh researchers, regulatory staff, Blackrock, JHU/APL, and DARPA allowed us to engage the FDA and IRB early, and thus quickly initiate the project. Technical and clinician teams worked with the participants allowing us to anticipate software and hardware problems early, resulting in long uninterrupted training periods. Finally, in working with the volunteers through our successes and failures, our participants became important members of the team. The BCI study can be an example of the multidisciplinary cooperation needed to bring a highly complex technology from animal models to a successful human trial.

Figure 4.

Roadmap toward the implementation of the multiyear MEA BCI clinical study.

Acknowledgments

This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) Revolutionizing Prosthetics program contract number N66001-10-C-4056, the National Institutes of Health (NIH) grant 8KL2TR000146-07, the Office of Research and Development, Rehabilitation Research & Development Service, Department of Veterans Affairs (Grants # B6789C, B7143R, and RX720), and the UPMC Rehabilitation Institute. The views expressed herein are those of the authors and do not reflect the official policy or position of the Department of Veterans Affairs, Department of Defense, or the United States government.

Ancillary