Are pilots at risk of accidents due to fatigue?
In 2002, Goode1 carried out research to assess the potential effect of duty hours (as a proxy for fatigue) on human-factors-related accidents in commercial aviation. Although the study kept relatively well to a quasiscientific approach in its methods, it also made some unwarranted pseudoscientific statements as well as misleading ones. Both are summarized in the next section.
- The title ("Are pilots at risk of accidents due to fatigue?") and parts of the abstract ("[…] there is likely to be a reduction in the risk of commercial aviation accidents due to pilot fatigue") suggested a correlation between fatigue and accidents in the study (and both the research problem and the discussion sustained an interpretive bias in favor of such relationship). However, the study did not actually measure fatigue, only duty-hours; thus, it misleads the reader towards a correlation that is unwarranted given the research methods used.
- The author was ambivalent in his use of inferences. On the one hand, he stated that "identifying fatigue in the flight crew exposure data can be done only by inference" (page 312) while also posing the assumption that if results were not significant then "one could infer that pilot human factor accidents are not affected by work schedule parameters" (pages 309, 310, 311). On the other hand, however, when results turned out significant, he stopped inferring, being instead positive that "there is a discernible pattern of increased probability of an accident the greater the hours of duty time for pilots" (pages 311, 312). In reality, inferential analysis is applicable for inferring either way, not just when results are not significant.
- Furthermore, the assumption that if distributions were as expected under the null hypothesis, then "one could infer that pilot human factor accidents are not affected by work schedule parameters" (pages 309, 310, 311) was also incorrect: not achieving statistical significance is not proof in favor of the null hypothesis.
- The author stated that chi-square test results exceeding the 5% significance threshold were highly significant, and that because of that "there is a discernible pattern of increased probability of an accident the greater the hours of duty time for pilots" (pages 311, 312). Such statement seemed to suggest two incorrect conclusions if based only on such statistical significance: that the results definitely showed that the human factors accidents sampled were affected by the work schedule parameters measured, and that the results had practical importance. Both are incorrect insofar such decisions are not warranted by any mathematical result but are made by the researcher, taking into consideration not only the statistics but also the overall quality of the research methods.
- The graph used on page 312 was misleading. It plotted two set of proportions using different scales. The percentage scale for exposure and accidents run from 0% to 40%, and this was matched to a different scale showing relative accident proportion running from 0 to 6.
Jose D PEREZGONZALEZ (2013). Massey University, Turitea Campus, Private Bag 11-222, Palmerston North 4442, New Zealand. (JDPerezgonzalez).
Want to know more?
- PubMed - Original article's abstract
- Access to the original article can be obtained via ScienceDirect.
- WikiofScience - Relationship between pilot duty hours and accidents
- This WikiofScience page offers a review of Goode's work, once the pseudoscience evidence is remove.