DOW-UAP-D151, AAWSAP DIRD, Cognitive Limits on Simultaneous Control of Multiple Unmanned Spacecraft, December 2010

This document is a Defense Intelligence Reference Document (DIRD), a technical reference format used by the Defense Intelligence Agency (DIA) to capture baseline knowledge on a specific topic for later analytic use. DIRDs are best understood as reference and synthesis products rather than as original research. It is one of 38 DIRDs produced under the Advanced Aerospace Weapon System Applications Program (AAWSAP) between 2009 and 2011. Because AAWSAP’s scope permitted a broad range of supporting topics, not every DIRD in the series directly concerns aerospace systems or future threat assessment. The following summary reflects the DIRD’s scope and framing at the time of writing and should not be read as implying current validation of the concepts discussed. This DIRD examines how many unmanned spacecraft a single human operator could realistically supervise or control at once, using research from air traffic control and multi-vehicle remote piloting as rough analogs. The report argues that the practical limit depends heavily on task complexity: about 16 craft for simple monitoring or destination assignment, about 7 for moderately complex piloting or mission tasks, and about 4 for complex heterogeneous operations. It places particular emphasis on the operator’s ability to maintain a coherent mental “big picture” of multiple vehicles at once, and it suggests that automation and external displays can help by offloading working-memory demands, though not eliminating them. The document also highlights physiological workload measures as a possible way to detect or predict operator overload in real time. Overall, it presents multi-spacecraft control as a human-factors and systems-integration problem in which progress depends on managing cognitive limits through interface design, automation, and workload monitoring.

UNCLASSIFIED/,<FOR QFFI&il.t.k WSli 8,.kY
Defense
Intelligence
Reference
Document
Defense Futures
15 December 2010
ICOD: 8 September 2010
DIA-08-1101-001
Cognitive Limits on
Simultaneous Control of
Multiple Unmanned Spacecraft
UNCLASSIFIED/ /FOA OFFl&IAL 148! 9HtY



UNCLASSIFIED//F&R &FFIOIAl l118E 8Hl¥
Cognitive Limits on Simultaneous Control of Multiple
Unmanned Spacecraft
The Defense Intelligence Reference Document provides non-substantive but
authoritative reference information related to intelli ence to ics or methodolo ies.
Prepared by:
Technology Warning Division (DW0-4)
Defense Warning Office
Directorate for Analysis
Defense Intelligence Agency
Author:
AAP Person 73
COPYRIGHT WARNING: Further dissemination of the photographs in this publication is not authorized.
This product is one of a series of advanced technology reports produced in FY 2010
under the Defense Intelligence Agency, Defense Warning Office's Advanced Aerospace
~m=
Weapons System Applications (AAWSA) P~ra!i o:auestions pertaining to
this document should be addressed to jAAP Person 1 ~ AAWSA Program
Manager, Defense Intelligence Agency, Al : J A - u 0- , Bldg 6000,
Washington D.C. 20340-5100
ii
UNCLASSIFIED//liQlil OliliiliCiliil,l Wili Qtlb¥

UNCLASSIFIED//FOR OFFl@IAL Y!H! 8HL'I
Contents
Summary................................................................................................................iv
Chapter 1: Introduction......................................................................................... 1
Chapter 2: Measurement of Mental Workload ........................................................ 3
Subjective Measurements .................................................................................. 3
Performance Measures....................................................................................... 4
Physiological Measures ...................................................................................... 4
Cardiac Function ............................................................................................ 5
CNS Measurements ........................................................................................ 6
Ocular Measurements..................................................................................... 6
Skin Measurements ........................................................................................ 7
Serum Levels of Hormones............................................................................. 7
Chapter 3: Studies in Cognitive Workload for Air Traffic Controllers ..................... 8
Modeling the Air Traffic Control Task ............................................................... 15
Chapter 4: Studies in Command of Multiple Semi-Automated Vehicles ................ 18
Chapter 5: Discussion.......................................................................................... 22
Chapter 6: Conclusions........................................................................................ 23
References ........................................................................................................... 24
Figures
Figure 1. One-dimensional Representation of Changes in Performance as Workload
Varies ..................................................................................................................... 2
Figure 2. Typical EKG Signal for a Normal Heartbeat.............................................. 5
Figure 3. Air Traffic Control .................................................................................. 10
Figure 4. Representation of Performance Results from Brookings Study.............. 13
Figure 5. Information Processing Model for a Human Operator............................ 16
Figure 6. Examples of Unmanned Military Vehicles .............................................. 19
Tables
Table 1. Variables used in Determining Complexity of the Traffic.................. 11
Table 2. Correlations among Physiological Variables ..................................... 14
iii
UNCLASSIFIED/ j FOR OPPICI.AL YSlii &PU,¥

UNCLASSIFIED//FOR OFFl@IAL Y!H! 8HL'I
Cognitive Limits on Simultaneous Control of Multiple
Unmanned Spacecraft
Summary
Space exploration 40 years into the future may include manned missions to parts of the
outer solar system. A possible scenario may include sending a small fleet of craft with
different primary missions. For example, a trailing spacecraft of nuclear powered
electromagnets designed to shield the manned part of the fleet from solar radiation;
halo spacecraft with powerful radars to scout for incoming objects; exploration and
mining craft, etc. The fleet could regularly travel out of unaided visual range of each
other, joining up when necessary for maintenance, exchange of materials such as fuel,
or other necessities. Piloting these multiple craft could be economically accomplished if
only one remote pilot on station at a time was necessary.
The cognitive limitation of a human astronaut and his ability to perform the multiple­
vehicle piloting task is the focus of this paper as little work has been done in this
specific area. However, a large body of cognitive research on the limitations of object
supervision and tracking for the task of air traffic control (ATC) exists. Additionally,
there is an emerging body of research concerned with multiple unmanned vehicle
piloting for heterogeneous missions. These are the two areas reviewed in detail as they
relate to possible spacecraft missions.
Pilots develop an internal mental representation of the identity, position, mission, and
current direction of relevant objects. This is referred to colloquially as "the big picture."
The primary research question we seek to answer is whether there is a cognitive limit
to the number of objects that can be monitored and tracked within the big picture.
Secondarily, we seek to find whether this maximum number is limited by the
complexity of interaction; how those limiting factors are described, whether there is a
real-time objective measure that indicates when a pilot is approaching his maximum
capacity, and whether that capacity has been exceeded. The maximum number of
tracked objects is highly dependent on the complexity of the piloting and mission tasks
at hand.
Research is lacking in the area of cognitive limits on the number of spacecraft one pilot
could control given any mission scenario. Currently, two models are being used to
examine similar activities in air traffic control and remote piloting of multiple unmanned
vehicles. In both areas it has been shown the cognitive limits on the number of craft
capable of simultaneous control is 16 for simple destination selection, 7 for moderately
complex piloting and/or mission task completion, and 4 for complex heterogeneous
craft. While additional future research may help to increase the automation component
of aircraft and mission control, no current evidence exists to show that a complete
mental picture can be maintained for more than about 16 objects at one time, even
with external working memory augmentation. However, it has also been demonstrated
that physiological variables can be objectively employed to indicate overload. Nominal
success has been achieved in classifying physiological states near high workload thus
enabling both prediction and possibly prevention of overload.
iv
UNCLASSIFIED/;<FOA: OFFICiI.\k W&li 8Ptl¥

UNCLASSIFIED//FOR QFFI&iIAk WS& 8,.klf
Chapter 1: Introduction
Due to the complexity, duration and numerous support requirements of future manned
deep-space missions involving exploration, mineral exploitation, and possible
colonization, a likely scenario will be the inclusion of unmanned fleets of support craft.
Coupled with other requirements, an intensive research program is needed to
investigate the cognitive limits on pilots and other operators responsible for the
simultaneous control of multiple unmanned spacecraft making up the support fleet; a
"fleet' approach is proposed in an effort to optimize safety and exploratory reach. This
research effort would also aim at maximizing the functional efficiency of the mission
and reducing the operation costs of unmanned vehicle fleets.
In this scenario there is much about the ancillary craft that are automated in both
navigation and mission. Many of them will not require full-time piloting but given that
they could be hundreds of miles from each other at any instant of time, they need
monitoring to prevent unseen system failure or collision from letting them just
disappear one day during the mission like a Martian probe. Accomplishing this
monitoring task and the occasional piloting task for multiple craft in the fleet could be
economically accomplished if only one remote pilot on station at a time was necessary.
We will focus here on the cognitive limitation of a human astronaut to perform the
multiple-vehicle piloting task. It is not a surprise that there is little work in this specific
area - in fact there were zero peer-reviewed articles in the major journals concerning
remote piloting of multiple spacecraft (published in the last 30 years). There is
however, a large body of cognitive research on the limitations of object supervision and
tracking for the task of air traffic control (ATC). There is additionally an emerging body
of research concerned with multiple unmanned vehicle piloting for heterogeneous
missions. These are the two areas reviewed in detail as they relate to possible
spacecraft missions.
When a pilot or ATC operator is in control of several craft they have developed an
internal mental representation of the identity, position, mission, and current direction of
each object tracked. This is referred to colloquially as "the big picture." This mental
representation is also called situational awareness. Keeping all of the information about
all of the objects straight as long as they are in scope is the goal.
The primary research question we seek to answer is whether there is a cognitive limit
to the number of moving objects that can be maintained in the big picture. Secondary
questions are whether this maximum number is limited by complexity, how those
limitations might be described, whether there is a real-time objective measure that will
indicate when a pilot is approaching their maximum capacity, and whether that capacity
has been overloaded.
For the current treatise we will consider only traditional humans as pilots. Cyborg­
enhanced astrobots are a topic for another tome.
In discussing cognitive limitations, it is useful to introduce the concepts of task demand,
mental workload, and a simplistic model of multiple resource theory. For a given task,
the gross level of neural activity required is a representation of the mental demands of
the task. Of course, a complex task can demand varied resources such as visual and
1
UNCLASSIFIED// FOR OFFICIJl!tt tl!H!! 8,.klf

UNCLASSIFIED/ /FOR OSSICIAl WSE 8HLY
audio processing, and this is where multiple resource theory enters: different resources
can be considered independent if demands on one resource do not tax the availability of
capacity of the other. Performance of a task can be high or low: in general if spare
capacity is available, performance is high, and if capacity is limited or exceeded,
performance is low. Moreover, task demand enforces a fixed theoretical relationship
between mental workload and task performance: this is shown in Figure l.
Relationship between Workload and Task Performance
High
Performance
Workload
Low
D Al A2 A3 B C
Task Demand
Figure 1. One-dimensional Representation of Changes in Performance
as Workload Varies. Task demand increases to the right. In the three A
regions, performance remains unchanged : A2 is optimal, where a trained
operator exerts minimal effort to maintain a set level of performance. At times
of low demand the operator exerts effort to maintain vigilance (Al) until
demand is so low, the subject disengages from the task (D). The workload curve
is not well defined at very low levels of demand (dotted lines). At the other end,
as demand increases, the subject can exert effort to keep up with demand (A3).
Additional effort maintains performance until degradation begins (B), and in the
overload condition, region C, performance is degraded beyond acceptable levels.
While effort is maintained in overload, some low level of performance exists.
2
UNCLASSIFIED/ /FOR OFFI@IAI: WSE OHl:l/

UNCLASSIFIED/ j POI\ OFFICIAL ttSl!! 8HLY
Chapter 2: Measurement of Mental Workload
There are three approaches to measuring mental workload in a subject performing a
primary task. The first is subjective evaluation, either post-hoc self-report
questionnaires concentrating on how "busy" one may have felt, or an experimental
observation of activity. The second is performance measures, an objective evaluation of
how well the subject completes the primary task, a secondary task, or an
experimentally inserted reference task. The final approach is to record real-time
physiological measures, with the assumption that increased workload increases anxiety
and this will be exhibited by changes in the autonomic nervous system (ANS).
SUBJECTIVE MEASUREMENTS
Self-report measures are appealing because they get inside the mind that was
performing the task. There is no absolute objective scale to measure one person's "fully
occupied" from another person's view of the same state; however, through rating
scales and self-drawn graphs, one can obtain an accurate picture of how perceived
workload evolved during the experiment. Perceived workload is important because it is
what needs to be maintained between a subjective minimum, where attention may
wander, and a subjective maximum, where increased emotion may decrease
performance capacity.
The most frequently used standardized self-report tools are the NASA Task Load Index
(NASA-TLX or just TLX) and the Subjective Workload Assessment Technique
(SWAT) .1,2,3.4 The TLX is a subjective workload assessment based on a multi­
dimensional rating questionnaire. An overall workload score is derived based on a
weighted average of ratings on six subscales: mental demands, physical demands,
temporal demands, own performance, effort, and frustration. SWAT is a two-step
assessment of three workload factors: time load, mental effort load, and psychological
stress load. In the first step, hypothetical activities are ranked according to perceived
workload. In the second step, the experimental task is evaluated post-hoc, using a 1-3
rating scale for each of the three dimensions. An interval scale of workload is derived,
from 0-100, based on the reference data collected for each subject in the first step, and
the evaluation of the experimental task. A custom self-report can also be designed by
the experimenter to specifically focus on the research questions in a given experiment.
The second type of subjective measure is evaluation by an expert observer. In this type
of measurement, assumptions are made on the mental act