User:SA Scholar/Sandbox

From Wikipedia, the free encyclopedia

Is it Situation Awareness or Situational Awareness?

Grammatically, situational awareness refers to awareness that only happens sometimes in certain situations. As the term implies, situation awareness refers to ‘awareness’ of the ‘situation.’

Situation awareness is the correct term for the field of study that concerns the knowledge and understanding of the environment that is critical to those who need to make decisions in complex areas such as aviation, air traffic control, driving, power plant operations, and military command and control.

Situation awareness has been formally defined as "the perception of elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future" (Endsley, 1988, 1995b, 2000).

Contents 1 INTRODUCTION 2 ORIGINS 3 DEFINITIONS OF SITUATION AWARENESS 4 RELATED CONCEPTS 4.1 Situational Understanding 4.2 Situation Assessment 4.3 Mental Models 4.4 Sensemaking 5 THEORETICAL MODEL OF SITUATION AWARENESS 6 SITUATION AWARENESS IN TEAM OPERATIONS 6.1 Team SA 6.2 Shared SA 6.3 Team SA Model 7 MEASUREMENT OF SITUATION AWARENESS 7.1 Objective Measures of SA 7.2 Subjective Measures of SA 7.3 Performance and Behavioral Measures of SA 7.4 Process Indices of SA 7.5 Multi-faceted Approach to SA Measurement 8 REFERENCES

[edit] INTRODUCTION

At its core, Situation awareness (SA) involves being aware of what is happening around you to understand how information, events, and your own actions will impact your goals and objectives, both now and in the near future. Lacking SA or having inadequate SA has consistently been identified as one of the primary factors in accidents attributed to human error (e.g., Hartel, Smith, & Prince, 1991; Merket, Bergondy, & Cuevas-Mesa, 1997; Nullmeyer, Stella, Montijo, & Harden, 2005). Thus, SA is especially important in work domains where the information flow can be quite high and poor decisions may lead to serious consequences (e.g., piloting an airplane, functioning as a soldier, or treating critically ill or injured patients).

Having complete, accurate and up-to-the-minute SA is considered to be essential in any domain where the effects of ever-increasing technological and situational complexity on the human decision-maker are a concern. Indeed, SA has been recognized as a critical, yet often elusive, foundation for successful decision-making across a broad range of complex and dynamic systems, including aviation and air traffic control (e.g., Nullmeyer, Stella, Montijo, & Harden 2005), emergency response and military command and control operations (e.g., Blandford & Wong 2004; Gorman, Cooke, & Winner 2006), and offshore oil and nuclear power plant management (e.g., Flin & O’Connor, 2001).

[edit] ORIGINS

Although the term itself is fairly recent, the concept appears to go back a long way in the history of military thinking—it is recognizable in Sun Tzu's Art of War, for instance. The term itself, can be traced back to World War I, where it was recognized as a crucial component for crews in military aircraft (Press, 1986).

Before being widely adopted by human factors scientists in the 1990s, the term was first used by United States Air Force (USAF) fighter aircrew returning from war in Korea and Vietnam (see Watts, 2004). They identified having good SA as the decisive factor in air-to-air combat engagements—the "ace factor" (Spick, 1988). Survival in a dogfight was typically a matter of observing the opponent's current move and anticipating his next move a fraction of a second before he could observe and anticipate one's own. USAF pilots also came to equate SA with the "observe" and "orient" phases of the famous observe-orient-decide-act loop (OODA Loop) or Boyd cycle, as described by the USAF fighter ace and war theorist Col. John Boyd. In combat, the winning strategy is to "get inside" your opponent’s OODA loop, not just by making your own decisions quicker, but also by having better SA than the opponent, and even changing the situation in ways that the opponent cannot monitor or even comprehend. Losing one's own SA, in contrast, equates to being "out of the loop."

Clearly, SA has far reaching application as it is needed for individuals and teams to function effectively in their environment. Thus, we are beginning to see SA going beyond the field of aviation and work being conducted in a wide variety of domains. Currently, the study of SA is now being examined in such diverse areas as air-traffic control, nuclear power plant operation, vehicle operation and anesthesiology (Endsley, 1995b; Gaba, Howard & Small, 1995; Collier & Follesf, 1995; Bolstad, 2000, Sollenberger & Stein, 1995).

[edit] DEFINITIONS OF SITUATION AWARENESS

Although numerous definitions of SA have been proposed, Endsley’s definition (1995b), "the perception of elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future," is firmly established and widely accepted. While some definitions are specific to the environment from which they were adapted, Endsley’s definition is applicable across multiple task domains. Several other definitions of SA have been suggested, generally restating the same themes:

• “accessibility of a comprehensive and coherent situation representation which is continuously being updated in accordance with the results of recurrent situation assessments” (Sarter & Woods, 1991)

• "the combining of new information with existing knowledge in working memory and the development of a composite picture of the situation along with projections of future status and subsequent decisions as to appropriate courses of action to take" (Fracker, 1991b)

• "the continuous extraction of environmental information along with integration of this information with previous knowledge to form a coherent mental picture, and the end use of that mental picture in directing further perception and anticipating future need" (Dominguez, Vidulich, Vogel, & McMillan, 1994)

• "adaptive, externally-directed consciousness that has as its products knowledge about a dynamic task environment and directed action within that environment” (Smith & Hancock, 1995)

• "the ability to maintain a constant, clear mental picture of relevant information and the tactical situation including friendly and threat situations as well as terrain" (Dostal, 2007)

• SA is simply “knowing what is going on so you can figure out what to do” (Adam, 1993)

• “what you need to know not to be surprised” (Jeannot, Kelly, & Thompson, 2003)

[edit] RELATED CONCEPTS

Several cognitive processes related to situation awareness are briefly described in this section. The matrix shown below (see Table 1) attempts to illustrate the relationship among some of these concepts (S. M. Fiore, personal communication, November 6, 2007). Note that situation awareness and situation assessment are more commonly discussed in complex domains such as aviation and military operations and relate more to achieving immediate tactical objectives. Sensemaking and achieving understanding are more commonly found in industry and the organizational psychology literature and often relate to achieving long-term strategic objectives.

Table 1

Related Cognitive Processes Organized by Objective and Phase

		Phase
		Process	Outcome
Objective	Tactical (short-term)	situation assessment	situation awareness
	Strategic (long-term)	sensemaking	understanding

[edit] Situational Understanding

Situation awareness is sometimes confused with the term “situational understanding.” In the context of military command and control applications, situational understanding refers to the "product of applying analysis and judgment to the unit's situational awareness to determine the relationships of the factors present and form logical conclusions concerning threats to the force or mission accomplishment, opportunities for mission accomplishment, and gaps in information" (Dostal, 2007). Situational understanding is the same as Level 2 SA in the Endsley model — the comprehension of the meaning of the information as integrated with each other and in terms of the individual’s goals. It is the “so what” of the data that is perceived.

[edit] Situation Assessment

Endsley (1995b, p. 36) argues that “it is important to distinguish the term situation awareness, as a state of knowledge, from the processes used to achieve that state. These processes, which may vary widely among individuals and contexts, will be referred to as situation assessment or the process of achieving, acquiring, or maintaining SA.” Thus, in brief, situation awareness is viewed as "a state of knowledge," and situation assessment as "the processes” used to achieve that knowledge. Note that SA is not only produced by the processes of situation assessment, it also drives those same processes in a recurrent fashion. For example, one’s current awareness can determine what one pays attention to next and how one interprets the information perceived (Endsley, 2000).

[edit] Mental Models

Accurate mental models are one of the prerequisites for achieving SA (Endsley & Jones, 1997; Sarter & Woods, 1991). Mental models can be described as well-defined, highly-organized yet dynamic knowledge structures developed over time from experience (Glaser, 1989; Kozlowski, 1998). The volume of available data inherent in complex operational environments can overwhelm the capability of novice decision makers to attend, process, and integrate this information efficiently, resulting in information overload and negatively impacting their SA (Endsley, 1997). In contrast, experienced decision makers assess and interpret the current situation (Level 1 and 2 SA) and select an appropriate action based on conceptual patterns stored in their long-term memory as ‘mental models’ (Serfaty, MacMillan, Entin, & Entin, 1997). Cues in the environment activate these mental models, which in turn guide their decision making process.

[edit] Sensemaking

Klein, Moon, and Hoffman (2006) distinguish between situation awareness and sensemaking as follows:

“...situation awareness is about the knowledge state that’s achieved—either knowledge of current data elements, or inferences drawn from these data, or predictions that can be made using these inferences (Endsley, 1995b). In contrast, sensemaking is about the process of achieving these kinds of outcomes, the strategies, and the barriers encountered.” (p. 71)

In brief, sensemaking is viewed more as “a motivated, continuous effort to understand connections (which can be among people, places, and events) in order to anticipate their trajectories and act effectively” (Klein et al., 2006, p. 71) rather than the state of knowledge underlying situation awareness. Endsley (2004) points out that as an effortful process, sensemaking is actually considering a subset of the processes used to maintain situation awareness. In the vast majority of the cases, SA is instantaneous and effortless, proceeding from pattern recognition of key factors in the environment – “The speed of operations in activities such as sports, driving, flying and air traffic control practically prohibits such conscious deliberation in the majority of cases, but rather reserves it for the exceptions.” Endsley (2004) also points out that sensemaking is backward focused, forming reasons for past events, while situation awareness is typically forward looking, projecting what is likely to happen in order to inform effective decision processes.

[edit] THEORETICAL MODEL OF SITUATION AWARENESS

The most common theoretical framework of SA is provided by Dr. Mica Endsley (1995b). Endsley’s model illustrates three stages or steps of SA formation: perception, comprehension and projection.

Perception (Level 1 SA): The first step in achieving SA is to perceive the status, attributes, and dynamics of relevant elements in the environment. Thus, Level 1 SA, the most basic level of SA, involves the processes of monitoring, cue detection, and simple recognition, which lead to an awareness of multiple situational elements (objects, events, people, systems, environmental factors) and their current states (locations, conditions, modes, actions).

Comprehension (Level 2 SA): The next step in SA formation involves a synthesis of disjointed Level 1 SA elements through the processes of pattern recognition, interpretation, and evaluation. Level 2 SA requires integrating this information to understand how it will impact upon the individual’s goals and objectives. This includes developing a comprehensive picture of the world, or of that portion of the world of concern to the individual.

Projection (Level 3): The third and highest level of SA involves the ability to project the future actions of the elements in the environment. Level 3 SA is achieved through knowledge of the status and dynamics of the elements and comprehension of the situation (Levels 1 and 2 SA), and then extrapolating this information forward in time to determine how it will affect future states of the operational environment.

Endsley’s model of SA (see Figure 1 below) also illustrates several variables that can influence the development and maintenance of SA, including individual, task, and environmental factors. For example, individuals vary in their ability to acquire SA; thus, simply providing the same system and training will not ensure similar SA across different individuals. Endsley’s model shows how SA “provides the primary basis for subsequent decision making and performance in the operation of complex, dynamic systems” (Endsley, 1995a, p. 65). Although alone it cannot guarantee successful decision making, SA does support the necessary input processes (e.g., cue recognition, situation assessment, prediction) upon which good decisions are based (Artman, 2000).

SA also involves both a temporal and a spatial component. Time is an important concept in SA, as SA is a dynamic construct, changing at a tempo dictated by the actions of individuals, task characteristics, and the surrounding environment. As new inputs enter the system, the individual incorporates them into this mental representation, making changes as necessary in plans and actions in order to achieve the desired goals. SA also involves spatial knowledge about the activities and events occurring in a specific location of interest to the individual. Thus, the concept of SA includes perception, comprehension, and projection of situational information, as well as temporal and spatial components.

Figure 1. Endsley's model of situation awareness (adapted from Endsley, 1995b).

[edit] SITUATION AWARENESS IN TEAM OPERATIONS

In many systems, people work not just as individuals, but as members of a team. Thus, it is necessary to consider the SA of not just individual team members, but also the SA of the team as a whole. To begin to understand what is needed for SA within teams, it is first necessary to clearly define what constitutes a team. A team is not just any group of individuals; rather teams have a few defining characteristics. As defined by Salas et al (1992), a team is:

"a distinguishable set of two or more people who interact dynamically, interdependently and adaptively toward a common and valued goal/objective/mission, who have each been assigned specific roles or functions to perform, and who have a limited life span of membership."

[edit] Team SA

Team SA is defined as “the degree to which every team member possesses the SA required for his or her responsibilities” (Endsley, 1995b, p. 39; see also Endsley, 1989). The success or failure of a team depends on the success or failure of each of its team members. If any one of the team members has poor SA, it can lead to a critical error in performance that can undermine the success of the entire team. By this definition, each team member needs to have a high level of SA on those factors that are relevant for his or her job. It is not sufficient for one member of the team to be aware of critical information if the team member who needs that information is not aware.

In a team, each member has a subgoal pertinent to his/her specific role that feeds into the overall team goal. Associated with each member's subgoal are a set of SA elements about which he/she is concerned. Team SA, therefore, can be represented as shown in Figure 2. As the members of a team are essentially interdependent in meeting the overall team goal, some overlap between each member’s subgoal and their SA requirements will be present. It is this subset of information that constitutes much of team coordination. That coordination may occur as a verbal exchange, a duplication of displayed information, or by some other means.

Figure 2. Team SA can be determined by examining the goals and SA requirements of all team members (adapted from Endsley & Jones, 1997, 2001).

[edit] Shared SA

Shared situation awareness can be defined as "the degree to which team members possess the same SA on shared SA requirements" (Endsley & Jones, 1997, p. 47; 2001, p. 48). As implied by this definition, there are information requirements that are relevant to multiple team members. A major part of teamwork involves the area where these SA requirements overlap — the shared SA requirements that exist as a function of the essential interdependency of the team members. In a poorly functioning team, two or more members may have different assessments on these shared SA requirements and thus behave in an uncoordinated or even counter-productive fashion. Yet in a smoothly functioning team, each team member shares a common understanding of what is happening on those SA elements that are common — shared SA. Thus, shared SA refers to the overlap between the SA requirements of the team members, as presented in Figure 3. As depicted by the clear areas of the figure, not all information needs to be shared. Clearly, each team member is aware of much that is not pertinent to the others on the team. Sharing every detail of each person's job would only create a great deal of “noise” to sort through to get needed information. It is only that information which is relevant to the SA requirements of each team member that is needed.

Figure 3. Shared SA Requirements (adapted from Endsley & Jones, 1997; 2001).

[edit] Team SA Model

The situation awareness of the team as a whole, therefore, is dependent upon both (1) a high level of SA among individual team members for the aspects of the situation necessary for their job; and (2) a high level of shared SA between team members, providing an accurate common operating picture of those aspects of the situation common to the needs of each member (Endsley & Jones, 2001). Endsley and Jones (1997; 2001) describe a model of team situation awareness as a means of conceptualizing how teams develop high levels of shared SA across members (see Figure 4): Each of these four factors — requirements, devices, mechanisms and processes — act to help build team and shared SA.

Figure 4. Model of team situation awareness (adapted from Endsley & Jones, 2001).

1. Team SA Requirements - the degree to which the team members know which information needs to be shared, including their higher level assessments and projections (which are usually not otherwise available to fellow team members), and information on team members' task status and current capabilities.

2. Team SA Devices - the devices available for sharing this information, which can include direct communication (both verbal and non-verbal), shared displays (e.g., visual or audio displays, or tactile devices), or a shared environment. As non-verbal communication, such as gestures and display of local artifacts, and a shared environment are usually not available in distributed teams, this places far more emphasis on verbal communication and communication technologies for creating shared information displays.

3. Team SA Mechanisms - the degree to which team members possess mechanisms, such as shared mental models, which support their ability to interpret information in the same way and make accurate projections regarding each other's actions. The possession of shared mental models can greatly facilitate communication and coordination in team settings.

4. Team SA Processes - the degree to which team members engage in effective processes for sharing SA information which may include a group norm of questioning assumptions, checking each other for conflicting information or perceptions, setting up coordination and prioritization of tasks, and establishing contingency planning among others.

[edit] MEASUREMENT OF SITUATION AWARENESS

While the SA construct has been widely researched, the multivariate nature of SA poses a considerable challenge to its quantification and measurement (for a detailed discussion on SA measurement, see Endsley & Garland, 2000; Fracker, 1991a; 1991b). In general, techniques vary in terms of direct measurement of SA (e.g., objective real-time probes or subjective questionnaires assessing perceived SA) or methods that infer SA based on operator behavior or performance. Direct measures are typically considered to be ‘product-oriented’ in that these techniques assess an SA outcome; inferred measures are considered to be ‘process-oriented,’ focusing on the underlying processes or mechanisms required to achieve SA (Graham & Matthews, 2000). These SA measurement approaches are further described next.

[edit] Objective Measures of SA

Objective measures directly assess SA by comparing an individual’s perceptions of the situation or environment to some ‘ground truth’ reality. Specifically, objective measures collect data from the individual on his or her perceptions of the situation and compare them to what is actually happening to score the accuracy of their SA at a given moment in time. Thus, this type of assessment provides a direct measure of SA and does not require operators or observers to make judgments about situational knowledge on the basis of incomplete information. Objective measures can be gathered in one of three ways: real-time as the task is completed (e.g., real-time probes – Jones & Endsley, 2000), during an interruption in task performance (e.g., Situation Awareness Global Assessment Technique (SAGAT) – Endsley, 1995a), or post-test following completion of the task.

[edit] Subjective Measures of SA

Subjective measures directly assess SA by asking individuals to rate their own or the observed SA of individuals on an anchored scale (e.g., Post-Trial Participant Subjective Situation Awareness Questionnaire (PSAQ) – Strater, Endsley, Pleban, & Matthews, 2001; Situation Awareness Rating Technique the (SART) – Taylor, 1989). Subjective measures of SA are attractive in that they are relatively straightforward and easy to administer. However, several limitations should be noted. Individuals making subjective assessments of their own SA are often unaware of information they do not know. Subjective measures also tend to be global in nature, and, as such, do not fully exploit the multivariate nature of SA to provide the detailed diagnostics available with objective measures. Nevertheless, self-ratings may be useful in that they can provide an assessment of operators’ degree of confidence in their SA and their own performance.

Subjective estimates of an individual’s SA may also be made by experienced observers (e.g., peers, commanders, or trained external experts). These observer ratings may be somewhat superior to self-ratings of SA because more information about the true state of the environment is usually available to the observer than to the operator, who may be focused on performing the task (i.e., trained observers may have more complete knowledge of the situation). However, observers have only limited knowledge about the operator’s concept of the situation and cannot have complete insight into the mental state of the individual being evaluated. Thus, observers are forced to rely more on operators’ observable actions and verbalizations in order to infer their level of SA. In this case, such actions and verbalizations are best assessed using performance and behavioral measures of SA, as described next.

[edit] Performance and Behavioral Measures of SA

Performance measures ‘infer’ SA from the end result (i.e., task performance outcomes), based on the assumption that better performance indicates better SA. Common performance metrics include quantity of output or productivity level, time to perform the task or respond to an event, and the accuracy of the response or, conversely, the number of errors committed. The main advantage of performance measures is that these can be collected objectively and without disrupting task performance. However, although evidence exists to suggest a positive relation between SA and performance, this connection is probabilistic and not always direct and unequivocal (Endsley, 1995b). In other words, good SA does not always lead to good performance and poor SA does not always lead to poor performance (Endsley, 1990). Thus, performance measures should be used in conjunction with others measures of SA that directly assess this construct.

Behavioral measures also ‘infer’ SA from the actions that individuals choose to take, based on the assumption that good actions will follow from good SA and vice-versa. Behavioral measures rely primarily on observer ratings, and are, thus, somewhat subjective in nature. To address this limitation, observers can be asked to evaluate the degree to which individuals are carrying out actions and exhibiting behaviors that would be expected to promote the achievement of higher levels of SA (see, for example, the Situation Awareness Behaviorally Anchored Rating Scale (SABARS) – Matthews, Pleban, Endsley, & Strater, 2000; Strater et al, 2001). This approach removes some of the subjectivity associated with making judgments about an individual’s internal state of knowledge by allowing them to make judgments about SA indicators that are more readily observable.

[edit] Process Indices of SA

Process indices examine how individuals process information in their environment, such as by analyzing communication patterns between team members or using eye tracking devices. Team communication (particularly verbal communication) supports the knowledge building and information processing that leads to SA construction (Endsley & Jones, 1997). Thus, since SA may be distributed via communication, computational linguistics and machine learning techniques can be combined with natural language analytical techniques (e.g., Latent Semantic Analysis) to create models that draw on the verbal expressions of the team to predict SA and task performance (Bolstad, Cuevas, Gonzalez, & Schneider, 2005; Bolstad, Foltz, Franzke, Cuevas, Rosenstein, & Costello, 2007). Although evidence exists to support the utility of communication analysis for predicting team SA (Foltz, Bolstad, Cuevas, Franzke, Rosenstein, & Costello, in press), time constraints and technological limitations (e.g., cost and availability of speech recording systems and speech-to-text translation software) may make this approach less practical and viable in time-pressured, fast paced operations.

Psycho-physiological measures also serve as process indices of operator SA by providing an assessment of the relationship between human performance and a corrected change in the operator’s physiology. In other words, cognitive activity is associated with changes in the operator’s physiological states. For example, the operator’s overall functional state (as assessed using psycho-physiological measures, such as electroencephalographic (EEG) data, eyeblinks, and cardiac activity) may provide an indication as to whether the operator is sleep fatigued at one end of the continuum, or mentally overloaded at the other end (Wilson, 2000). Other psycho-physiological measures, such as event related potentials (ERP), event related desynchronization (ERD), transient heart rate (HR), and electrodermal activity (EDA), may be useful for evaluating an operator’s perception of critical environmental cues, that is, to determine if the operator has detected and perceived a task-relevant stimulus (Wilson, 2000). In addition, it is also possible to use psycho-physiological measures to monitor operators’ environmental expectancies, that is, their physiological responses to upcoming events, as a measure of their current level of SA (Wilson, 2000).

[edit] Multi-faceted Approach to SA Measurement

The multivariate nature of SA significantly complicates its quantification and measurement, as it is conceivable that a metric may only tap into one aspect of the operator’s SA. Further, studies have shown that different types of SA measures do not always correlate strongly with each other (cf. Durso, Truitt, Hackworth, Crutchfield, Nikolic, Moertl, Ohrt, & Manning, 1995; Endsley, Selcon, Hardiman, & Croft, 1998; Vidulich, 2000). Accordingly, rather than rely on a single approach or metric, valid and reliable measurement of SA should utilize a battery of distinct yet related measures that complement each other. Such a multi-faced approach to SA measurement capitalizes on the strengths of each measure while minimizing the limitations inherent in each.

[edit] REFERENCES

• Adam, E. C. (1993). Fighter cockpits of the future. Proceedings of 12th IEEE/AIAA Digital Avionics Systems Conference (DASC), 318-323.
• Artman, H. (2000). Team situation assessment and information distribution. Ergonomics, 43(8), 1111-1128.
• Blandford, A. & Wong, W. (2004). Situation awareness in emergency medical dispatch. International Journal of Human–Computer Studies, 61, 421–452.
• Bolstad, C. A. (2000). Age-related factors affecting the perception of essential information during risky driving situations. Paper presented at the Human Performance Situation Awareness and Automation: User-Centered Design for the New Millennium Conference, Savannah, GA.
• Bolstad, C. A., Cuevas H. M., Gonzalez, C., & Schneider, M. (2005). Modeling shared situation awareness. Proceedings of the 14th Conference on Behavior Representation in Modeling and Simulation (BRIMS). Los Angeles, CA.
• Bolstad, C. A., Foltz, P., Franzke, M., Cuevas, H. M., Rosenstein, M., & Costello, A. M. (2007). Predicting situation awareness from team communications. Proceedings of the 51st Annual Meeting of the Human Factors and Ergonomics Society. Santa Monica, CA: HFES.
• Collier, S. G. & Follesf, K. (1995). SACRI: A measure of situation awareness for nuclear power plant control rooms. Proceedings of an International Conference: Experimental Analysis and Measurement of Situation Awareness (pp. 115-122). Daytona Beach, FL.
• Dominguez, C., Vidulich, M., Vogel, E. & McMillan, G. (1994). Situation awareness: Papers and annotated bibliography. Armstrong Laboratory, Human System Center, ref. AL/CF-TR-1994-0085.
• Dostal, B. C. (2007). Enhancing situational understanding through the employment of unmanned aerial vehicles. Army Transformation Taking Shape ...Interim Brigade Combat Team Newsletter, No. 01-18. Retrieved November 7, 2007, from http://www.globalsecurity.org/military/library/report/call/call_01-18_ch6.htm
• Durso, F. T., Truitt, T. R., Hackworth, C. A., Crutchfield, J. M., Nikolic, D., Moertl, P. M., Ohrt, D., & Manning, C. A. (1995). Expertise and chess: A pilot study comparing situation awareness methodologies. In D.J. Garland & M.R. Endsley (Eds.), Experimental analysis and measurement of situation awareness (pp. 295-303). Daytona Beach, FL: Embry-Riddle Aeronautical University Press.
• Endsley, M. R. (1988). Situation awareness global assessment technique (SAGAT). Proceedings of the National Aerospace and Electronics Conference (NAECON), 789-795. New York: IEEE.
• Endsley, M. R. (1989). Final report: Situation awareness in an advanced strategic mission (No. NOR DOC 89-32). Hawthorne, CA: Northrop Corporation.
• Endsley, M. R. (1990). Predictive utility of an objective measure of situation awareness. Proceedings of the Human Factors Society 34th Annual Meeting (pp. 41-45). Santa Monica, CA: Human Factors Society.
• Endsley, M. R. (1995a). Measurement of situation awareness in dynamic systems. Human Factors, 37(1), 65-84.
• Endsley, M. R. (1995b). Toward a theory of situation awareness in dynamic systems. Human Factors 37(1), 32-64.
• Endsley, M. R. (1997). The role of situation awareness in naturalistic decision making. In Zsambok, C. E. & G. Klein (Eds.), Naturalistic decision making (pp. 269-283). Mahwah, NJ: LEA.
• Endsley, M. R. (2000). Theoretical underpinnings of situation awareness: A critical review. In M. R. Endsley & D. J. Garland (Eds.), Situation awareness analysis and measurement. Mahwah, NJ: LEA
• Endsley, M. R. (2004). Situation awareness: Progress and directions. In S. Banbury & S. Tremblay (Eds.), A cognitive approach to situation awareness: Theory, measurement and application (pp. 317-341). Aldershot, UK: Ashgate Publishing.
• Endsley, M. R. & Garland, D. J. (Eds.) (2000). Situation awareness analysis and measurement. Mahwah, NJ: Lawrence Erlbaum Associates.
• Endsley, M. R. & Jones, W. M. (1997). Situation awareness, information dominance, and information warfare (No. AL/CF-TR-1997-0156). Wright-Patterson AFB, OH: United States Air Force Armstrong Laboratory.
• Endsley, M. R., & Jones, W. M. (2001). A model of inter- and intrateam situation awareness: Implications for design, training and measurement. In M. McNeese, E. Salas & M. Endsley (Eds.), New trends in cooperative activities: Understanding system dynamics in complex environments. Santa Monica, CA: Human Factors and Ergonomics Society.
• Endsley, M. R., Selcon, S. J., Hardiman, T. D., & Croft, D. G. (1998). A comparative evaluation of SAGAT and SART for evaluations of situation awareness. Proceedings of the Human Factors and Ergonomics Society Annual Meeting (pp. 82-86). Santa Monica, CA: Human Factors and Ergonomics Society.
• Flin, R. & O’Connor, P. (2001). Applying crew resource management in offshore oil platforms. In E. Salas, C.A. Bowers, & E. Edens (Eds.), Improving teamwork in organization: Applications of resource management training (pp. 217-233). Hillsdale, NJ: Erlbaum.
• Foltz, P. W., Bolstad, C. A., Cuevas, H. M., Franzke, M., Rosenstein, M., & Costello, A. M. (in press). Measuring situation awareness through automated communication analysis. To appear in M. Letsky, N. Warner, S. M. Fiore, & C. Smith (Eds.), Macrocognition in teams. Aldershot, England: Ashgate.
• Fracker, M. L. (1991a). Measures of situation awareness: An experimental evaluation (Report No. AL-TR-1991-0127). Wright-Patterson Air Force Base, OH: Armstrong Laboratories.
• Fracker, M. L. (1991b). Measures of situation awareness: Review and future directions (Report No. AL-TR-1991-0128). Wright-Patterson Air Force Base, OH: Armstrong Laboratories.
• Gaba, D. M., Howard, S. K., & Small, S. D. (1995). Situation awareness in anesthesiology. Human Factors, 37, 20-31.
• Glaser, R. (1989). Expertise and learning: How do we think about instructional processes now that we have discovered knowledge structures? In D. Klahr & K. Kotovsky (Eds.), Complex information processing: The impact of Herbert A. Simon (pp. 269-282). Hillsdale, NJ: LEA.
• Gorman, J. C. Cooke, N. J., & Winner, J. L. (2006). Measuring team situation awareness in decentralized command and control environments. Ergonomics, 49 (12-13), 1312–1325.
• Graham, S. E. & Matthews, M. D. (2000). Modeling and measuring situation awareness. In J.H. Hiller & R. L. Wampler (Eds.), Workshop on assessing and measuring training performance effectiveness (Tech. Rep. 1116) (pp. 14-24). Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social Sciences.
• Hartel, C. E. J., Smith, K., & Prince, C. (1991, April). Defining aircrew coordination: Searching mishaps for meaning. Paper presented at the 6th International Symposium on Aviation Psychology, Columbus, OH.
• Jeannot, E., Kelly, C. & Thompson, D. (2003). The development of situation awareness measures in ATM systems. Brussels: Eurocontrol.
• Jones, D. G. & Endsley, M. R. (2000). Examining the validity of real-time probes as a metric of situation awareness. Proceedings of the 14th Triennial Congress of the International Ergonomics Association and the 44th Annual Meeting of the Human Factors and Ergonomics Society. Santa Monica, CA: Human Factors and Ergonomics Society.
• Klein, G., Moon, B, & Hoffman, R. R. (2006). Making sense of sensemaking 1: Alternative perspectives. IEEE Intelligent Systems, 21 (4), 70-73.
• Kozlowski, S. W. J. (1998). Training and developing adaptive teams: Theory, principles, and research. In J. A. Cannon-Bowers, & E. Salas, (Eds.), Making decisions under stress: Implications for individual and team training (pp. 115-153). Washington, DC: American Psychological Association.
• Matthews, M. D., Pleban, R. J., Endsley, M. R., & Strater, L. G. (2000). Measures of infantry situation awareness for a virtual MOUT environment. Proceedings of the Human Performance, Situation Awareness and Automation: User-Centered Design for the New Millennium. Savannah, GA: SA Technologies, Inc.
• Merket, D. C., Bergondy, M., & Cuevas-Mesa, H. (1997, March). Making sense out of teamwork errors in complex environments. Paper presented at the 18th Annual Industrial/Organizational-Organizational Behavior Graduate Student Conference, Roanoke, VA.
• Nullmeyer, R. T., Stella, D., Montijo, G. A., & Harden, S. W. (2005). Human factors in Air Force flight mishaps: Implications for change. Proceedings of the 27th Annual Interservice/Industry Training, Simulation, and Education Conference (paper no. 2260). Arlington, VA: National Training Systems Association.
• Press, M. (1986). Situation awareness: Let's get serious about the clue-bird. Unpublished manuscript.
• Salas, E., Dickinson, T. L., Converse, S., & Tannenbaum, S. I. (1992). Toward an understanding of team performance and training. In R. W. Swezey & E. Salas (Eds.), Teams: their training and performance (pp. 3-29). Norwood, NJ: Ablex.
• Sarter, N. B. & Woods, D. D. (1991). Situation awareness: A critical but ill-defined phenomenon. International Journal of Aviation Psychology, 1, 45-57.
• Serfaty, D., MacMillan, J., Entin, E. E., & Entin, E. B. (1997). The decision-making expertise of battle commanders. In C. E. Zsambok & G. Klein (Eds.), Naturalistic decision making (pp. 233-246). Mahwah, NJ: LEA.
• Smith, K., & Hancock, P. A., (1995). Situation awareness is adaptive, externally directed consciousness. Human Factors, 37 (1), 137-148.
• Sollenberger, R. L., & Stein, E. S. (1995). A simulation study of air traffic controllers’ situation awareness. Proceedings of an International Conference: Experimental Analysis and Measurement of Situation Awareness (pp. 211-217). Daytona Beach, FL.
• Spick, M. (1988). The Ace Factor: Air Combat and the Role of Situational Awareness. Annapolis, MD: Naval Institute Press.
• Strater, L. D., Endsley, M. R., Pleban, R. J., & Matthews, M. D. (2001). Measures of platoon leader situation awareness in virtual decision making exercises (No. Research Report 1770). Alexandria, VA: Army Research Institute.
• Taylor, R. M. (1989). Situational awareness rating technique (SART): The development of a tool for aircrew systems design. Proceedings of the AGARD AMP Symposium on Situational Awareness in Aerospace Operations, CP478. Seuilly-sur Seine: NATO AGARD.
• Vidulich, M. A. (2000). Testing the sensitivity of situation awareness metrics in interface evaluations. In M. R. Endsley & D. J. Garland, (Eds.), Situation awareness analysis and measurement (pp. 227-246). Mahwah, NJ: Lawrence Erlbaum Associates.
• Watts, B. D. (2004). "Situation awareness" in air-to-air combat and friction. Chapter 9 in Clausewitzian Friction and Future War, McNair Paper no. 68 (revised edition; originally published in 1996 as McNair Paper no. 52). Institute of National Strategic Studies, National Defense University
• Wilson, G. F. (2000). Strategies for psychophysiological assessment of situation awareness. In M. R. Endsley & D. J. Garland, (Eds.), Situation awareness analysis and measurement (pp. 175-188). Mahwah, NJ: Lawrence Erlbaum Associates.