Human Factors, their Interactions and their Influence on ATC Performance

Table of Contents

Methods

Research approach

Generalization potential

Edwards, Sharples, Wilson and Kirwan (2012) researched the need for a human factors model of performance which integrated the interactions between factors, including identifying which factors critically impact air traffic controller (ATCO) performance[1]. They conducted a literature review of peer-reviewed journal articles to ascertain previous factor interaction research, analysed incident reports to determine the influence of multiple factors in incidents, and conducted an online survey of air traffic management (ATM) professionals to identify and further refine the human performance factors crucial for investigation into ATCO performance. This qualitative meta-analysis compares the findings from the three studies to highlight the differences between the results of the literature review and the findings from incident reports and the online survey.

Results

The authors carried out a literature review of 83 peer-reviewed journal articles and conference papers searching for nine human factors identified as influential in ATC incidents. The review found mental workload (MWL), stress and fatigue to be the most commonly researched factors. Trust and situation awareness (SA) were found the least.

After analysing 275 incident reports from a Eurocontrol database, the authors found the most frequently reported contributory factors included SA, communication, attention and teamwork. Fewer reports featured trust, fatigue and stress as contributory factors.

Illustration 1 compares the researched factors and reported factors. The factors are ranked from most frequent to least frequent. Stress and fatigue were among the most researched factors yet were some of the least reported contributory factors in the incident reports. Similarly, SA was often considered a contributory factor in incident reports yet the literature review found little research on it.

Illustration 1 - Comparison of Human Factors Ranking between Research and Incident Reports
Researched Factors found in Literature Review	Reported Factors from Incident Reports
MWL	Attention
Stress	SA
Fatigue	Communications
Attention	Teamwork
Communications	Workload
Vigilance	Vigilance
Teamwork	Perception
SA	Memory
Trust	Trust
Not given	Fatigue
Not given	Stress

Illustration 2 below shows the differences in factor pair ranking between three groups: the incident reports, ATCOs completing the survey, and investigators completing the survey. The incident report factor pairs show in brackets the categories they would be named as had they been in the survey. High workload is abbreviated to HWL, and underload to UWL. The factor pairs are ranked from most frequent to least frequent.

Further information regarding how factors were renamed is in the methods section.

Illustration 2 - Variances in Factor Pair Ranking - Top Ten Results
Ranking	Incident Reports - Factor Pairs	Survey Results - ATCOs	Survey Results - Investigators
1	Attention/vigilance & communications (becomes SA & communications)	HWL & inadequate teamwork	Inadequate communications & HWL
2	SA & attention/vigilance (becomes SA & SA)	Inadequate SA & HWL	HWL & inadequate teamwork
3	MWL & attention/vigilance (becomes HWL/UWL & SA)	HWL & stress	Inadequate SA & HWL
4	MWL & communications (becomes HWL/UWL & communications)	Inadequate communications & HWL	HWL & UWL
5	MWL & SA (becomes HWL/UWL & SA)	Fatigue & inadequate teamwork	HWL & stress
6	SA & communications	Inadequate SA & inadequate communications	Inadequate SA & inadequate communications
7	SA & memory (becomes SA & SA)	HWL & UWL	Inadequate communications & Inadequate teamwork
8	Teamwork & communications	Stress & inadequate SA	Stress & inadequate SA
9	Attention/vigilance & teamwork (becomes SA & teamwork)	Fatigue & HWL	Inadequate teamwork & inadequate SA
10	SA & teamwork	Stress & inadequate communications	Not given

Only one of the factor pairs is equally ranked with all three groups agreeing on the 6th ranked factor pair of SA and communications. The highest position with a match between factor pairs occurs in third place, with both the incident reports and investigators agreeing on HWL and inadequate SA; and HWL and communication is ranked fourth in incident reports and by ATCOs. Stress and inadequate SA was agreed to the 8th most likely to contribute to an accident by both ATCOS and investigators, whereas stress did not feature at all in the ten highest pairs from incident reports.

ATCOs tended to focus on HWL which featured 6 times in the top 10 (places 1, 2, 3, 4, 7, & 9). ATCOs also placed two pairs featuring fatigue in their top ten, at 5th and 9th. Fatigue was not present in any of the top 9 factor pairs for investigators, who also had HWL in 5 factor pairs.

ATCOs rated HWL & Inadequate Teamwork highest while the Investigators placed Inadequate Communication & HWL as the top factor pair, with HWL & Inadequate Teamwork second. These rankings show the high emphasis placed on effective cooperation and collaboration with colleagues, especially when there is a high workload.

The survey responses showed a statistically significant difference between the ATCO responses and those from incident investigators (including those still active as ATCOs), using a Mann-Whitney U test. The different subsets of the sample show different perceptions are held by each group regarding the contribution of each factor pair in incidents. ATCOs tended to give lower ratings to all of the factor pairs with means ranging from 1.73 – 2.59, whereas those respondents trained in incident investigation were more likely to select “frequently” or “very frequently” from the scale when rating the factor pairs, resulting in means that ranged from 2.00 to 3.43. Given that the rating tool was a 5-point Likert scale, the means from both groups demonstrate a belief that no single factor is solely responsible for any incident. The standard deviations for the survey factor pairs listed in Illustration 2 had a range of 0.75-1.01 for ATCOs and 0.77-1.17 for investigators. These standard deviations are relatively small and indicate that most participants answered in a similar fashion.

Comparing Illustrations 1 and 2, it is apparent that the factors that have been most researched are not reflected as highly contributory to incidents in either formal incident reports or the online survey. SA was the second least common factor found during the literature review yet featured in the top 3 factor pairs in Illustration 2. Stress is one of the most common factors to appear in the articles reviewed by the authors, yet is not particularly prevalent in Illustration 2, appearing only five times in the survey responses and never in the incident reports column. Illustration 1’s third most researched factor of fatigue only appears twice in Illustration 2 in the survey responses by ATCOs.

The differences between factors/factor pairs in Illustrations 1 and 2 show agreement over the human factors affecting ATC performance and the need to consider the interactions between such factors when considering incident causes. The data highlights the contrast between the published research and the opinions and experience of those working in ATC (gathered through incident reports and survey responses) with regards to the frequency and influence of each factor and factor pair.

Methods

Research approach

The research was an exploratory study to identify the relevance and need to create an incident investigation model that considers multiple human performance factors acting in synergy. It also sought to identify which factors would be most useful in such a model.

Sample

The sample was selected by snowball sampling from the target population of active ATCOs and incident investigators.
Over two months the survey received 65 responses with respondents from 24 European countries.
The median respondent was an ATCO with no incident investigation role, working in a European country.
80% of the respondents were active as ATCOs and almost half of the sample acted as incident investigators.

Design

The online survey was designed in collaboration with five ex-controllers to ensure the questions were relevant and meaningful to the respondents.
The online survey offered seven factors for consideration by participants with the wording of some factors altered to be less confusing and more accessible for answering. Mental workload was differentiated into high workload and underload; SA was deemed to encompass vigilance, memory, attention and perception; and trust was incorporated into teamwork, resulting in seven factors.
Each factor was paired once with each other factor, resulting in a total of 28 items.
Counterbalancing of the factor pairs was used to reduce order effects from respondents.
A five point Likert scale was employed to measure answers, ranging from “very rarely” to “very frequently”.

Variables

Sampling method – Snowball sampling relies on the initial respondents encouraging those in their network to also respond (and so on). This could be reflected by the high percentage (43.1%) of respondents having incident investigation training. It is unlikely that this percentage accurately reflects the target population.
Dependent variable - The respondents' work experiences and possible involvement in any incident.

Procedure

The online survey was available for answering for two months.

Data analysis

The article provided graphic results for the literature review, percentages for the incident reports and means and standard deviations for the survey results.
The data from the survey was analysed using a Mann-Whitney U test to determine between group differences for ATCOs and investigators.
The data was then analysed using inferential statistics - Levene's test and Kolmogorov-Smirnov analysis.
The data was found to not have a normal distribution so non-parametric statistics were used - Friedman's ANOVA, Wilcoxon analysis and bonferroni correction.
This meta-analysis compares the results from each study to highlight the differences in a qualitative manner.

Generalization potential

This study could be generalised to the wider Western-ised ATC population. Although the incident reports came from a single ATC provider, the nature of that provider (Eurocontrol) means that many nationalities would be included, and the survey respondents came from 24 European countries. The cultural differences between wider nationalities such as European vs Asian, or South American vs African would indicate that certain human factors may carry more or less influence in other populations from other cultures.

References

1. EDWARDS Tamsyn, Sarah SHARPLES, John R WILSON & Barry KIRWAN (2012). Factor interaction influences on human performance in air traffic control: the need for a multifactorial model. Work, 2012, volume 41, pages 159-166.