DOW-UAP-D130, AAWSAP DIRD, Technological Approaches to Controlling External Devices, March 2010

This document is a Defense Intelligence Reference Document (DIRD), a technical reference format used by the Defense Intelligence Agency (DIA) to capture baseline knowledge on a specific topic for later analytic use. DIRDs are best understood as reference and synthesis products rather than as original research. It is one of 38 DIRDs produced under the Advanced Aerospace Weapon System Applications Program (AAWSAP) between 2009 and 2011. Because AAWSAP’s scope permitted a broad range of supporting topics, not every DIRD in the series directly concerns aerospace systems or future threat assessment. The following summary reflects the DIRD’s scope and framing at the time of writing and should not be read as implying current validation of the concepts discussed. This DIRD surveys brain-machine interface technologies intended to allow users to control external devices without conventional manual controls, and it evaluates both noninvasive and invasive approaches for turning neural or related physiological signals into usable commands. The report reviews the underlying neural signals, distinguishes between open- and closed-loop control systems, and examines technologies including scalp-based electrical recording, magnetic and imaging-based methods, and implanted cortical interfaces, with particular attention to bandwidth, response time, signal quality, and practical usability. It concludes that, in the near term, the most practical systems are likely to be noninvasive electrical approaches that draw heavily on muscle and neural signals, while longer-term high-bandwidth control would likely require more advanced invasive interfaces capable of robust two-way communication with individual neurons. The document presents thought-based control of external devices as a research field with plausible assistive and specialized applications, while emphasizing that naturalistic, high-performance control remained constrained by major technical and physiological limits.

UNCLASSIFIED/f 1"8rt 8Ffl@ltltl ~!HI 8HL\f
Defense
Intelligence
Reference
Document
Acquisition Threat Support
23 March 2010
!COD: 1 December 2009
DIA-08-1003-012
Technological Approaches to
Controlling External Devices in
the Absence of Limb-Operated
Interfaces
UNCLASSIFIED//f8R 8FFI&I.t.k Wliili Qtlb¥

UNCLASSIFIED//F8R 8FFl6i,t.k Miili Ot!P X
Technological Approaches to Controlling External Devices
in the Absence of Limb-Operated Interfaces
Prepared by:
Acquisition Support Division (DW0-3)
Defense Warning Office
Directorate for Analysis
Defense Intelligence Agency
Author:
AAP Person 73
Administrative Note
COPYRIGHT WARN ING: Further dissemination of the photographs in this publication is not authorized.
This product is one in a series of advanced technology reports produced in FY 2009
under the Defense Intelligence Agency, Defense Warning Office's Advanced Aerospace
Weapon System Applications (AAWSA) Program. Comments or questions pertaining to
l
this document should be addressed to !AAP Person 1 AAWSA Program
Manager, Defense Intelligence Agency, ATTN: CLAR/DWO-3, Bldg 6000, Washington,
DC 20340-5100.
ii
UNCLASSIFIED//FQA &FFIGIAk Wiili ,u11a¥

UNCLASSIFIED//FOlil OFFI&I.t.k WIiii 8Plklf
Contents
Introduction ............................................................................................................v
Direct Neural Signals.............................................................................................. 1
Indirect Neuronal Signals - The BOLD Effect.......................................................... 3
Control of External Devices .................................................................................... 4
Noninvasive Technologies ...................................................................................... 6
EEG ..................................................................................................................... 7
MEG .................................................................................................................... 8
EMG .................................................................................................................... 9
MRI and fMRI ..................................................................................................... 9
NIRS.................................................................................................................... 10
Invasive Technologies.......................................................................................... 10
Open-Loop Direct Cortical Array Algorithm Modeling ....................................... 11
Closed-Loop Peripheral Arrays Utilizing Visual Feedback ................................. 13
MEMS, ECoG, and PSoC Circuitry ...................................................................... 14
Chronic Neural Implants and fMRI ................................................................... 16
New Electrode Designs..................................................................................... 18
Hybrid Neuro-Robotic Systems for True Closed-Loop BMI ................................ 20
Trials Using Human Subjects............................................................................ 22
Optical Stimulation of Action Potentials ........................................................... 23
Discussion ............................................................................................................ 23
Noninvasive Electrical Devices ......................................................................... 24
Noninvasive BOLD-Based devices..................................................................... 25
Noninvasive Magnetic Devices ......................................................................... 26
Invasive Technologies - General, Optical, and Ex-Vivo Engineering................. 26
Implantable Chips, Ladders, and Arrays........................................................... 27
iii
UNCLASSIFIED//rOR: orr1e1J1ct U91!! OHL¥

UNCLASSIFIED/,'FOR: 8ffl@IJllt t:191!! l>flt I
Conclusions .......................................................................................................... 27
Figures
Figure 1. Simplified Rendering of a Neuron ............................................................ 1
Figure 2. The General Layout of a Closed-Loop Control Interface........................... 5
Figure 3. Two Commercially Available EEG Sensors................................................ 8
Figure 4. Experimental Overview of Brain-Controlled Robot in a Closed Loop
With Visual Feedback Experiment ......................................................... 13
s.
Figure Schematic of the Neurochip Functional Blocks ...................................... 15
Figure 6. Implant Location ................................................................................... 16
Figure 7. Histology and Electrode Tracks ............................................................. 17
Figure 8. The Michigan Electrodes........................................................................ 18
Figure 9. Image Distortion and Custom Microwire Electrode Assembly to
Improve It............................................................................................. 19
Figure 10. Example of T2 Variability..................................................................... 19
Figure 11. T2 Value Analysis Summary of T2 Values in All the Image Slices That
Spanned the Electrode Arrays in All Animals ....................................... 20
Figure 12. A Hybrid Neuro-Robotic System .......................................................... 21
Figure 13. Experimental Arrangements ................................................................ 21
iv
UNCLASSIFIED//FOil Offl@IJllL YSIE 8Htlf

UNCLASSIFIED//P(Ht err1e1s1tt l!ISI!!! 9HL'I
Technological Approaches to Controlling External Devices
in the Absence of Limb-Operated Interfaces
Introduction
Since the advent of modern interactive control of computers, technologies
have been sought to directly connect the biological and the physical to form a
seamless entity. While science fiction explores the possibilities of shared
consciousness between brains and mainframes, real-life scientists work
toward an equally fantastic but more pedestrian goal of eliminating required
electromechanical human-machine interfaces (HMis). Such technologies
promise integration of thought and computer-controlled action, without the
need for a limb-operated device to translate intent from physiological
networks to physical circuits.
This paper first briefly reviews underlying neural structure, function, and
activity to provide background on the type of signals and the scale of temporal
changes that arise from conscious control within the nervous system. A short
primer on brain-machine interfaces (BMis) is included at the end of the
background material. This is followed by a survey of the state-of-the-art
detection and stimulation technologies available utilizing noninvasive and
invasive methods, including several studies that exemplify research paths
currently being undertaken. Discussion brings together elements of the
technology survey, application limitations, a timeline for useful commercial
deployment, and predicted future research directions. Two technology
research paths are highlighted as the most probable to produce functionally
useful devices (defined as nearly equal or even superior to the control
capabilities of current HMis) in the near and far term.
The technology endpoint this paper seeks is thought-based operation of
remote machinery during normal human activities without mechanical device
interaction-that is, control of external devices without the need to go to a
specified location, such as a shielded room; without the need to remain
perfectly motionless to reduce signal noise; and without the need for
interaction with a normal electro-mechanical device, such as an i-Phone or
other handheld device with buttons and trackballs. Data transfer rates are
sought to exceed 5-10 bits/second to be useful for operation of complex
devices; this rate range and above is referred to as high-bandwidth BMis.
Response time for a 1-of-N selection of commands is targeted at 300
milliseconds or less. These are the operational parameters assumed for final
implementation unless some limitation or supercapability is described.
The paper concludes that noninvasive electrical monitoring of neural activity,
primarily reading combined action of muscles and neurons, is the most
promising commercial technology in the near term. Inherent limitations in the
noninvasive electrical approach require future development of different
research paths. The paper argues, mainly by eliminating other approaches,
that the most probable technology in the long term involves invasive single­
neuron-based direct cortical connections to form a network of high-bandwidth
duplex communication pathways. Currently the most promising technologies
V
UNCLASSIFIED//FOR. OFFIGIAk Uii QPlk¥

UNCLASSIFIED//P91t 9Ffl@IAL YSIE 8HLY
for robust construction of such an interface are optical stimulation, gating, and
sensory devices, and chip-based electrode arrays that have been encased in
ex-vivo-engineered neural tissue.
vi
UNCLASSIFIED//FOR. orrlEIAL U.!~ Gilt I

UNCLASSIFIED//POR: err1e1At l:191!! er•t I
Direct Neural Signals
The human nervous system has two classes of cells, neurons and glia. Based on all
research to date, it is believed that signals within the network of neurons constitute the
whole of information processing, with glial cells playing a purely supporting role. This
neural doctrine has dominated research in BMI until recently and still constitutes the
only major research path in direct technologies. Furthermore, all technologies directly
measuring human neuronal action rely on detecting or influencing electrical activity of
these cells; no current in situ research selectively affects neurotransmitter activity
between local cells for the purpose of information exchange . Therefore, the focus for
the foreseeable future will be on the electrical activity of neurons as the primary target
of BMI.
Neurons consist of four parts: axon, dendrites, cell body or soma, and pre-synaptic
terminals. Electrical information is transmitted to the neuron through the dendrites,
proceeds through the cell body, and leaves the cell through the axon at one or more
pre-synaptic terminals. Neurons have one axon and from one to tens of thousands of
dendrites.
Figure 1. Simplified Rendering of a Neuron. The arrows indicate t he direction in which signals are
conveyed. The single axon conducts signals away from the cell body, while the multiple dendrites receive
signals from the axons of other neurons. The nerve terminals end on the dendrites or cell body of other
neurons or on other cell types, such as muscle or gland cells. (Reference 1)
Chemical details of how the action potentials travel through the cell or are transmitted
across the synapse are not important to the current treatise, other than the distinction
that in these biologically based electrical networks, ions of sodium, potassium, and
chlorine move through the cell membranes perpendicular to the propagation of the
action potential down the axon. This allows information to be transmitted faster than
ions could flow down the axon. The propagation of information is similar to a wave
traveling down a garden hose: quickly move one end of the hose back and forth with
sufficient force, and a wave will travel to the other end of the hose; however, any part
of the hose structure has only moved (nominally) perpendicular to the direction of wave
propagation. In a similar fashion, ions flow through channels across the axon's cell
membrane, changing the local membrane potential and thus propagating the electrical
signal down the axon.
1
UNCLASSIFIED/;'FOR OFFI&I.t.k Wliliii 8fslk¥

UNCLASSIFIED/;'FOR OFFl&I.t.k Wliliii &NL¥
The signal transmission down the axon of a neuron is an all-or-nothing process. When
the cell body is stimulated above threshold, the axon transmits the same action
potential at the same speed and in the same direction, regardless of the extent above
threshold or duration of the input. Action potentials have durations of 1-10
milliseconds. Input signals can result in transmission of multiple action potentials, and
thus the frequency and number of neuronal firings do vary with the input. Neurons
require some time to reset between firings, nominally the duration of the pulse for that
axon, yielding a typical maximum firing rate of between 100 hertz (Hz) and 1 kilohertz
(kHz).
It is instructional at this point to contrast this mechanism with propagation of signals
through a physical electrical circuit, the planned external portion of our BMI. In a
copper wire, electrons carry the signal. Electrons drift along the signal path, but the
signal itself moves as a compression wave rather than a transverse wave as in the
biological system. Going back to our garden hose example, consider the hose now filled
with small marbles: inserting a marble at one end will move each marble in the hose
just a little, but very rapidly the last marble in line will pop out of the far end of the
hose. As with the biological system, the signal is propagated to the far end of the hose
by local actors rather than physical motion of a single ion or electron moving the whole
distance. The underlying physics governing the signal transmission makes metallic and
semimetallic circuits about a million times faster than the biological system.
This difference in the carriers and underlying mechanisms of signal transmission
between biological and physical circuits has so far prevented the invention of a direct
connection between the two disparate systems. Instead, both noninvasive and invasive
direct-detection technologies rely on placing physical sensors or transmitters in close
proximity to the neurons of interest and utilizing classical electrodynamics to govern
signal jump between the systems. Additionally, given the maximum typical firing rate
for neurons of 1 kHz, sampling of action potentials at a few kHz will be fast enough to
detect firing of any individual neuron, though super-sampling above 10 kHz can be used
to reduce noise. Higher sampling frequencies also may be required if multiple neurons
are monitored and quantitative information about their relative firing sequence is
desired. Frequencies of a 1 kHz or below are sufficient to stimulate action potentials,
and again, higher system frequency may be required for multiple neuron sequential
stimulation. Finally, higher frequencies may be required if monitoring or stimulation of
some aspect of signal transmission other than action potentials is sought, such as
monitoring single ion channels. 1 There is currently no evidence that transduction
frequencies above 100 kHz have any advantage in BMis, thus the main challenge is the
connection dynamics, since even this ultra-maximum freq