Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
INTRODUCTION
From the inception of Tactical Decision Aids in the battle space has grown more complex, technology is
military, there has been great debate over whether com- necessary to enable the human to understand the battle
mand and control (C2) is an art or a science. We have space, plan successful COAs to achieve desired military
no desire to engage further in this debate, and we readily goals, monitor the tactical/operational environment, and
stipulate that whenever a human has ultimate control react to any situation or threat that may interfere with
of the final decision on an appropriate course of action the achievement of the commander’s intent and guid-
(COA), a certain art form is involved. But along with ance. Furthermore, we find it axiomatic, on the basis of
this stipulation, this article postulates that, because the years of observation, that commanders and their staffs
JOHNS HOPKINS APL TECHNICAL DIGEST, VOLUME 31, NUMBER 1 (© 2012) 31
J. G. COOLEY AND J. A. B. MCKNEELY
working with well designed C2 systems are more effec- Significant work has been undertaken to model the
tive at achieving their desired purposes; inversely, com- necessary tasks performed by both human and machine
manders who are forced to operate with poorly designed functions for effective C2. One groundbreaking effort
C2 systems are at times challenged to meet their objec- was that of Colonel John R. Boyd, U.S. Air Force
tives by the very systems designed to help them. (retired),2 who created the OODA (observe, orient,
An unfortunate consequence of the art-versus- decide, and act) loop model (Fig. 1) for air-to-air con-
science debate for C2 is the lack of standardized systems flict. This model describes C2 as the need for the pilot
engineering practices for systems developed to provide to observe, orient, decide, and act.
decision-making assistance to warfighters. Every engi- Later models such as MAPE (monitor, assess, plan,
neer dedicated to providing systems to support C2 wants and execute) and MAAPPER (monitor, analyze, assess,
to build the appropriate tool set to enable warfighters to predict, plan, execute, and report) were subsequently
maximize their effectiveness. With the same vehemence developed to more specifically apply a C2 model to a
as those who debate C2 systems’ usefulness, the engi- larger theater of warfare and to more accurately describe
neering community has itself debated the methods to that theater’s processes of operation.
gauge and test the effectiveness of these systems. Two Paul North and Steven Forsythe3 compared these
distinct camps squared off in these debates. On one models in their paper, “A Process Decomposition
hand, the human factors engineering community has Approach for Evaluating Command and Control (C2)
worked diligently to dissect the concepts of command Functional Performance,” and ultimately determined
and to determine what content and presentation of that three key elements (KEs) can be synthesized from
information is necessary to facilitate good decisions. This each of these models. These KEs are (i) the need for
research has involved both physical and cognitive stud-
the commander to maintain situational awareness, (ii)
ies of commanders in controlled environments. On the
the need for the command staff to plan, and (iii) the
other hand, many C2 developers have employed rapid
need to execute or prosecute the warfighting mission.
prototyping in an “expert systems” approach, develop-
To fully account for the factors of C2, these KEs should
ing knowledge and heuristics by interviewing recognized
be expanded to include a fourth element: (iv) the need
experts, then codifying these rules into their systems
to perform an accurate intelligence preparation of the
and rapidly getting those systems into operational envi-
ronments to determine how to best serve the warfighter. battle space, as shown in Fig. 2. An accurate intelligence
Each of these two camps desires the same end result: a preparation of the battle space is what initiates the
better science of control to aid the art of command. This shared perspectives of the other three KEs and lays the
article is an attempt by two authors, each representing foundation for all of the commander’s decisions. Clearly,
one of these camps, to show how the combination of these four elements or tasks can be decomposed in many
these varied approaches may provide a key to good sys- different ways, depending on the size of the warfight-
tems engineering practices in C2 systems. ing theater and the specific mission being addressed by
As a way of describing the combined C2 systems engi- the command staff. However, if a commander is able
neering methodology, we first address and familiarize
the reader with each approach used in isolation. These
discussions include the systems engineering activities
Observe
involved, as well as the associated strengths and weak-
nesses of the individual approach. Then we turn to a
case study of a combined rapid prototyping and human
factors approach. We hope to show the benefits of a com-
bined human factors engineering and rapid prototyping
approach to provide a more holistic systems engineering
methodology for C2. Act Orient
WHAT IS C2?
Joint Publication 1-02 defines C2 as follows:
The exercise of authority and direction by a properly des-
ignated commander over assigned and attached forces in
the accomplishment of the mission. Command and con- Decide
trol functions are performed through an arrangement of
personnel, equipment, communications, facilities, and pro-
cedures employed by a commander in planning, directing,
coordinating, and controlling forces and operations in the
accomplishment of the mission.1 Figure 1. Boyd’s2 OODA loop model for C2.
JOHNS HOPKINS APL TECHNICAL DIGEST, VOLUME 31, NUMBER 1 (© 2012) 33
J. G. COOLEY AND J. A. B. MCKNEELY
developed capability has indeed closed the previously may shift, unintended warfighter performance chal-
identified gaps and if so, whether closure of those gaps lenges may emerge, or the engineering team may find
uncovers further gaps in capability that reduce the effec- another critical need that may have been obfuscated by
tiveness of the C2 process. earlier challenges. As the engineering team continues
A deeper investigation of this model of systems engi- to develop and the prototype team builds and evalu-
neering and its applicability to C2 systems reveals that ates these capabilities with representative end users, the
this approach also supports incremental development of teams not only provide better systems for the warfighter’s
multiple systems that will necessarily be integrated into use, they also become more aware of the warfighter’s
a holistic C2 system. This incremental approach is pro- challenges. The Battle Group Anti-Air Warfare (AAW)
vided through the up-front needs definition and capabil- Coordination (BGAAWC) Program is an excellent
ity improvement activity. The need sets the stage for the example of this phenomenon. Over a period of a dozen
objective C2 system, and the capability improvement plan years, these engineers worked on a series of systems
defines how it will get there. Each component (system for designed to improve the AAW capabilities in the fleet.
the system) then undergoes the engineering cycle with Each system undertaken was derived from the experi-
a common foundation of requirements. This approach ences gathered while under way. An excellent example
helps to ensure that when the parts are integrated, they was the genesis of the Dual Net Multi-Frequency Link
result in the planned whole, as evidenced through a ver- Program. The BGAAWC team was working with ship’s
ification process. The addition of cognitive engineering/ crew, supporting the Automatic Identification Program,
human-systems integration (HSI) also helps to ensure and noticed that while the identification was improving,
that these cycles of development meet the user needs the improvement was stymied by reduced radio connec-
and expectations supporting C2 warfighter decision tivity during certain weather phenomena. This identi-
making. This type of approach is represented in Fig. 5. fication of a critical challenge spawned the capability
that became the Multi-Frequency Link Program, which
in turn spawned the Dual Net Multi-Frequency Link
RAPID PROTOTYPING AND KNOWLEDGE Program, a program that not only significantly improved
connectivity, but also allowed the proper connectivity
ENGINEERING for the Automatic Identification Program to maximize
The cyclical model of systems engineering includes its effectiveness.
spiral development, rapid prototyping, and knowledge The task of rapid prototyping for C2 systems engi-
engineering to develop and field C2 systems. The spiral neering often involves significant knowledge engineer-
development methodology lends itself well to the con- ing. The knowledge engineering field was made popular
cept of rapid prototyping. As the engineering team by the artificial intelligence movement of the late 1980s.
peels back the layers of the critical needs, capabilities The concept of artificially intelligent “expert systems”
can be developed to close the most difficult challenges. included a methodology and scientific approach to
As these capabilities come online, the critical need gather system heuristics that expert humans employed.
These heuristics would then
– Operational Data Collection be incorporated into a series
– Lessons Learned of inductive and deductive
– Engineering Measurement Programs
– Threat Assessment logic statements that could
– Operational Needs – Test and Evaluation be run exhaustively to find
– Capabilities Improvements – Product Development
and Production the “intelligent” solution.
Most C2 systems do not
– Data Collection attempt to replace humans
– Mission Performance
Analysis and their associated knowl-
Iterate edge. However, the use of
Capability
development
these gathered heuristics
plan – Prototype Development does allow the C2 systems
– Laboratory Demonstration
– Enabling Science and Technology – At-Sea Experiments engineers to identify the dif-
– Hypothesis, Concept Development, ferences between processes
Trade-offs, and Critical Experiments
– Modeling and Simulations that can be more readily
Integration
and verification accomplished by the power
of the computer and pro-
cesses that require the inter-
vention of a human to sort
out the best answer. In this
Figure 5. C2 system-of-systems engineering model. way, the developed C2 sys-
JOHNS HOPKINS APL TECHNICAL DIGEST, VOLUME 31, NUMBER 1 (© 2012) 35
J. G. COOLEY AND J. A. B. MCKNEELY
tems can augment the human processing by providing anomalies that occur while under way. The simulations
the “heavy lifting” in terms of computation, thus allow- do provide an excellent environment for analysis, mod-
ing the commander to focus on the decisions. eling, and early testing, but to truly gain the benefits
An example of the separation of tasks between the of spiral development with rapid prototyping, the engi-
commander and the automation can be seen in C2 tools neers must be embedded with the warfighting staffs.
such as weapons allocation processors. In such systems, This is particularly true in the area of C2. Although
the computer models the entire set of possible blue-on- small team simulators (i.e., aircraft simulators) have been
red intercepts, determines which set of intercepts provides developed to effectively test the C2 decision processes, it
the highest probability of raid annihilation, considers the is very difficult to expand that encapsulated simulation
spatial and temporal requirements associated with the environment to a globally distributed C2 environment.
rules of engagement, and presents the options to the com- Furthermore, it is difficult with large groups of people
mander for final decision. In this manner the computer to fully simulate the “fear factor” associated with the
is doing the billions if not trillions of mathematical com- stressors and fog of warfare. Understanding these facts,
putations that humans cannot do in a timely manner. the Navy regularly conducts full-scale at-sea exercises to
However, the automated tool does not make the decision; provide the appropriate environment for training and
it only provides information to the commander. The evaluation. The use of the rapidly prototyped capabili-
human then contemplates considerations that cannot be ties during these exercises is invaluable for the systems
readily quantified into computer logic to make the final engineer. In fact, many systems have significant data-
decision. In extreme cases, the human may, after a long gathering and -reduction tools to capture data from
period of building trust in the system, decide to remove these events because of the unmatched value associated
himself from the process, but this is normally done in a with these exercises.
highly defined and constricted manner. Unfortunately the results from rapid prototyping
The challenge with knowledge engineering is find- experiments are often disregarded by some in the engi-
ing the experts. Many of the C2 system challenges the neering community. The term “anecdotal engineering”
community faces today are understood by very few com- is used far too often to discount the fact that certain
manders. The Force AAW Coordination Technology systems are being employed with great success. Rapid
(FACT) Program, circa 1997–2002, provides an excel- prototyping is not designed to conduct controlled exper-
lent example of this challenge. For years the program iments. It is designed to provide capabilities quickly and
had been working on ship and battle group systems, but to morph those capabilities through spiral development
in the late 1990s the Navy asked the team to develop by interacting with the warfighters in the operating
a theater capability. As the project attacked this latest environments. The results of these efforts can be quanti-
challenge it soon became evident that ship and battle fied more qualitatively than quantitatively. Hence, some
group knowledge and tactics could not necessarily be engineers would abandon this methodology and would
extrapolated to theater and combatant command tac- instead choose experimentation to determine the best
tics. Hence, the program manager sought out experts solution to a critical challenge. This is unfortunate, as
who could provide the knowledge necessary to build the concept of rapid prototyping is used in almost every
the Area Air Defense Commanders Command Support field of endeavor. (Think focus groups for television and
Capability. Three flag officers ultimately formed the advertising, beta releases for software, and car shows for
expert team that advised the development team on this the automobile industry.) None of these efforts truly mea-
project. Without their expertise and their commitment sures anything more than anecdotal evidence of what is
to working with the engineering team for several years, good, yet companies regularly use these methodologies
the system would not have met the needs of the war to determine their future COAs. Furthermore, there are
fighter. Culling the flag officers’ knowledge and experi- significant capabilities with reams of associated experi-
ences in various roles of theater command was crucial mental data that have missed the mark completely. That
to developing a tool that would provide the information is because these experiments worked extremely well
the commanders need to operate successfully. in a controlled environment, which unfortunately was
There are challenges associated with the spiral devel- not the environment in which the warfighter happens
opment and rapid prototyping methodology. The most to operate. As previously stated, creating a C2 environ-
immediate of these challenges is gaining access to the ment with the appropriate depth, scope, and fear factor
environments that will enable the system developers that approximate warfare is extremely rare.
to work closely with the warfighting customers. There
is no substitute for actual at-sea experience. The best
constructive simulations are just that, simulations.
COGNITIVE ANALYSIS AND DECISION
Although the Navy has spent considerable effort and ANALYSIS APPROACHES
dollars constructing hardware-in-the-loop simulation Achieving the desired impact with new C2 systems
capabilities, these capabilities cannot replicate the requires an understanding of human information pro-
JOHNS HOPKINS APL TECHNICAL DIGEST, VOLUME 31, NUMBER 1 (© 2012) 37
J. G. COOLEY AND J. A. B. MCKNEELY
In addition to the goals (i.e., desired effects) of war (SMEs) with semistructured and structured techniques.
fighters and the decisions required to meet those goals, The collected knowledge of the cognitive processes and
the nature of the operating environment is a major com- the environment in which these processes are performed
ponent to mission success. An ecological perspective is a is considered when the two paths converge, where repre-
key common aspect of the different cognitive engineer- sentations of the knowledge elicited are produced. The
ing approaches. The context in which the warfighter analysis and the resulting representations lead directly
executes C2 is considered, forming an understanding to DSS design concepts. The Office of Naval Research-
of cognition “in the wild.” The mental processes of the sponsored Exceptional Expertise for Submarine Com-
warfighter and the impact of environment on decision mand Team Decision Making (E2SCDM) Project
making are addressed: system objectives, people, arti- applied this knowledge elicitation approach.7 The team
facts, human goals, and the environment in which the observed submarine command team training, inter-
goals are applicable are considered collectively and simul- viewed more than 70 submarine operational command
taneously. Cognitive task analysis (CTA) provides for team officers and trainers, and reviewed submarine inci-
the documentation and analysis of these considerations, dent reports. This data collection and analysis supported
where CTA captures people’s tasks and goals within their insights into operational team practices and system
operational context. Methods for systematically investi- design that are being researched to improve submarine
gating the user’s tasks, organizing the results of observa- command team performance.
tions, and using this information to drive system design The ability to meet the DSS design goals must also be
and evaluation have become foundations for HSI. considered. As a result of maturity, cost, and/or schedule
The methods of CTA, however, do not fully account constraints, the technology may not always support the
for the complexity of C2. The recognition of this short- realization of the concept. An assessment of technology
fall led to the development of the effects-based decision maturity and feasibility is conducted and then combined
analysis methodology (EDAM), which employs the best with the DSS design concepts to create a total system
practices from current cognitive engineering processes. design concept and, ultimately, a prototyped demonstra-
EDAM (see Fig. 7) begins with defining and docu- tor. It is important to note that, although these steps
menting the context: scenario design and articulation are listed as a sequence, a great amount of concurrent
and an initial work domain analysis. This context is then and iterative work will take place between the steps; fur-
used throughout the process to support development of thermore, there is a critical need for a multidisciplinary
user profiles and to provide the foundation to consider team that provides systems, human performance, soft-
design decisions that need to be made and demands of ware, hardware, and operational/domain views into the
the environment and the system. design effort. This iterative, incremental nature lends
User profiles are critical products that provide the itself to the investigation of the software development
development team with the understanding of roles, rapid prototyping approach, and synergies between the
responsibilities, information requirements, collabora- two approaches (cognitive engineering and rapid proto-
tion needs, and systems used. An example user profile is typing) can be leveraged to support better design and
shown in Fig. 8 for the Undersea Warfare (USW) com- analysis at a more affordable cost.
munity.6 User profiles can aid the developer in ensuring This combined rapid prototyping and cognitive engi-
that different types of users’ specific needs are supported. neering/human factors approach also lends itself to a
This information is particularly important in C2 envi- key component of EDAM, which is human performance
ronments, where many people
need different types of informa-
tion in different forms and at dif- SME Participation
ferent speeds of response. Scenario Decision
Decision
design support
Knowledge elicitation follows knowledge Decision Demonstrator
systems
elicitation and work
the scenario development and concept
Work environment
should be conducted along two Work environment analysis System
parallel paths addressing cogni- domain and knowledge design
control task elicitation concept
tive performance, one focusing analysis Technology
on decisions and the other on trade-off
study
the work environment. Knowl-
edge elicitation methods include
attending courses related to Human
function; visiting training sites, performance
assessment
command centers, and other
C2-intensive sites; and inter-
viewing subject-matter experts Figure 7. EDAM process diagram.
assessment, at all stages to the degree possible. Human warfighters to elicit decision and work environment
performance assessment can be conducted at varying requirements, and SMEs from both pools will participate
levels of prototype fidelity and maturity and can run in design evaluations.
from task walkthroughs up to full-scale, full-functional- Systems engineering aims to provide complete,
ity human-in-the-loop measures of total system perfor- detailed, and verifiable requirements as early as pos-
mance. A critical component to this testing is having sible in the development process. Due to the complex
the right metrics at the right level (Fig. 9). C2 systems interaction of humans and the context in which they
span levels of performance; therefore, metrics must also operate, the cognitive/human factors team requires
span the levels of performance. Measuring human per- analysis of the candidate system architecture before
formance and diagnosing the impact on mission effec- they can fully develop valid requirements. To overcome
tiveness is important; unfortunately, doing so remains the challenge faced by a C2 program (of incorporating
difficult, particularly for decision making, because mul- early system-level human-performance-related require-
tiple measures, both objective and subjective in nature, ments and detailing the design), we recommend that
are required.8 Internal APL research has contributed an incremental system development approach be taken,
to advancing metrics and maturing testing protocols; one in which requirements are identified, prototyped,
however, these metrics (i.e., for cognitive performance: evaluated, and further specified to mitigate system and
neural correlates of human cognitive state, situational operational risk. This approach was recommended by
awareness, etc.) remain relatively immature and require the National Research Council Committee on Human-
more investigation. System Design Support for Changing Technology.9
Throughout the design and development process, Key is documentation of analysis in such a way that
participation of operational SMEs is absolutely critical. systems engineering can incorporate the technical input.
Domain-knowledgeable individuals assist in developing All too often input is not integrated into the design effort.
the scenarios that are used for interviews with current To facilitate integration, cognitive engineering products
JOHNS HOPKINS APL TECHNICAL DIGEST, VOLUME 31, NUMBER 1 (© 2012) 39
J. G. COOLEY AND J. A. B. MCKNEELY
With the advent of recent cognitive experimenta- Section Task Analysis (WSTA) (Fig. 11), calls for col-
tion designed to map the physiological human response lecting comprehensive performance measurements of a
(even brainwave reaction) to information and recogni- submarine watch section that is using Acoustic Rapid
tion, the rapid prototypers are empowered with unprec- COTS Insertion (A-RCI) and the BYG-1 Combat Con-
edented insight into system design. This gathering of trol Systems using the Navy’s shore-based Submarine
physiological data goes beyond the consciousness of the Multi-Mission Team Trainer (SMMTT). The focus of
warfighter and defies the biases that tradition and status WSTA is to analyze and understand the decision-mak-
quo bring to more anecdotal evidence. Rapid prototyp- ing processes involved in executing a simulated mission
ing also provides the cognitive engineering team with scenario, to understand information flow between mem-
early opportunities to validate function allocation, task bers of the watch section, and to identify the strengths
design, workload estimates, and operational concepts. and weaknesses of fleet-delivered sonar and combat sys-
The duality of benefits a rapid prototyping approach tems in support of watch information flow.
provides (both to the software developers and to the Many of the WSTA process activities involved
HSI teams) highlights the advantages of taking such components of EDAM. The program is scenario based
an approach. Furthermore, the systems engineers can and involves SMEs throughout. Additionally, the in-
gain valuable system-level insights from human-in-the- depth understanding of the decision-making processes
loop rapid prototyping. Quantitative metrics can pro- is a result of conducting task analysis, developing user
vide insights into the operational effectiveness of the profiles, and conducting and analyzing human–system
concept, and qualitative metrics can give indications of performance testing in SMMTT by using stressing sce-
user buy-in. narios. This understanding resulted in a display concept
that integrates real-time sonar waterfall data with active
contact solutions to provide the commanding officer and
AN EXAMPLE OF A HYBRID APPROACH the officer of the deck with a more intuitive, actionable
tactical picture. System developers then transformed the
The integrative method of build–test–build is inher-
concept to a working prototype, and the result is IBAL
ent in the PEO IWS5 Advanced Processor Build (APB)
(“eyeball”), a 360° plan position indicator-type display
Program. The APB Program is designed to bring con- specifically designed for the ship driver.
tinuous improvement and technology innovation into The integrated full system (hardware, software, and
the submarine combat system. Historically, this pro- human performance) testing with fleet officers of the
gram has implemented a thorough testing and analysis deck, commanding officers, and tactical teams from
method combined with fleet feedback and operational both Atlantic and Pacific fleets enabled the Devel-
effectiveness analysis to determine system improvement opment Squadron to target a 2008 installation of the
opportunities. Beginning in 2006 (and still continuing), IBAL capability, with employment guidance and train-
the program officially integrated the human element of ing. WSTA introduces formal cognitive engineering
the system as a component to a total system evaluation approaches and products along with systems engineering
methodology.10 This analysis approach, called Watch as an embedded part of the planning, analysis, develop-
ment, and testing of an APB. Coupled with engineer-
ing measurements, postevent analysis provides objective,
statistically based feedback to the APB development and
production system improvement process, system employ-
ment guidance, and submarine training.
The success of the PEO IWS5 APB Program has
spawned similar initiatives in other Navy programs. The
C2 Engineering Measurement Program (C2 EMP) for the
USW-DSS was established to provide qualitative data to
support a system design process fully coupled with real-
world operational experience [Moundalexis, M., Ocker-
man, J., Croucher, A., and Dean, M., “Command and
Control Engineering Measurement Program (C2EMP):
Initial C2 Survey Findings from SHAREM 163,” presen-
tation given at the Joint Undersea Warfare Technology
Spring Conference, San Diego, CA (8–11 March 2010)].
The focus of this evaluation was on USW-DSS capabili-
Figure 11. WSTA using SMMTT.10 [Contact management displays ties and their impact on C2, as well as on the quality
on USS Virginia (SSN 774). Photo taken by Petty Officer 1st Class (timeliness and accuracy) of detect-to-engage execution.
James Pinsky, U.S. Navy.] This USW-DSS Program has incorporated a warfighter-
JOHNS HOPKINS APL TECHNICAL DIGEST, VOLUME 31, NUMBER 1 (© 2012) 41
J. G. COOLEY AND J. A. B. MCKNEELY
ing capabilities can be improved. for Evaluating Command and Control (C2) Functional Performance,”
in Proc. 2006 Command and Control Research and Technology Symp.,
San Diego, CA, pp. 1–14 (2006).
4McKneely, J. A., Ockerman, J., and Koterba, N., “A Hybrid Approach
The Authors
Jeffrey G. Cooley is a member of the Principal Professional Staff in APL’s
Air and Missile Defense Department. He has supported a wide variety of
C2 programs, gaining expertise in net-centric C2, modeling and simulation,
case-based reasoning, and planning tools for both air and ballistic missile
defense for Naval and Joint Operations. He currently serves as the Assistant
Program Area Manger for Integrated Warfare Systems and Technologies and
is responsible for the Laboratory’s Surface Combatant Combat Systems Pro-
Jeffrey G. Cooley Jennifer A. B. grams. Jennifer A. B. McKneely is a member of the Principal Professional
McKneely Staff in APL’s Force Projection Department. She has supported a wide vari-
ety of system design development and research projects, gaining experience in C2, human-systems engineering processes
and tools development, situational awareness, augmented cognition, and autonomy. She currently serves as the Project
Manager for the Intelligence Advanced Research Projects Activity (IARPA) Sirius Program and supports human-systems
engineering in cyber operations. For further information on the work reported here, contact Jeff Cooley. His e-mail address
is jeffrey.cooley@jhuapl.edu.
The Johns Hopkins APL Technical Digest can be accessed electronically at www.jhuapl.edu/techdigest.