Effect of safety information on ADM
Lee, Fanjoy and Dillman explored the effect of safety information on the aeronautical decision making of pilot in 2005. They experimented group of student pilots from bachelor degree program. It was found that the students who were exposed to safety information (through ADAM) recognized trigger events better on two of the four events ( see Table 1).Their findings highlights that student pilots who are exposed to periodic review of safety information had shorter recognition times for abnormal aircraft conditions than non-exposed students. The difference between the experimental and control groups on the recognition of trigger were significantly on two of the four events ( see Table 1). The analysis indicated that the corrective action scores of the experimental and control groups were significantly different (p<.0001).
| Table 1. Recognition time analysis between groups | |||
|---|---|---|---|
| trigger event | interpretation* | ||
| Alternate failure | not significant | ||
| High oil pressure | significant | ||
| High load meter | not significant | ||
| Vacuum system failure | significant | ||
| * p= .05 | |||
Response time between groups
Table 2 analyse the results for response time of student pilots who were exposed to safety information.The experimental group was significantly faster on the three of trigger events. This finding also supports the usefulness of exposing students to in responding to periodic review of safety information. For response time three of the experimental group results was significant faster on the trigger events (See table 2).
| Table 2. Response time analysis between groups | |||
|---|---|---|---|
| trigger event | interpretation* | ||
| Alternate failure | significant | ||
| High oil pressure | significant | ||
| High load meter | significant | ||
| Vacuum system failure | not significant | ||
| * p= .05 | |||
For SJT, the control group and experiment group results for the post-test scores showed a difference, p<0.0001 which suggest positive effect of ADAM treatment. Also the difference between post-test and pre-test SJT scores within the experimental group was analysed using regression model.The probability, p=0.0003 also concludes there is a positive effect from using ADAM.
Methods
Research approach
This research was an exploratory experiment on the effects of safety information from ADAM on the aeronautical decision making of pilots from a collegiate flight program.
Sample
- A sample of 34 student pilots was selected randomly from a pool of volunteers and was randomly divided into a control group (n=17) & an experimental group (n=17). These had already obtained private pilot licenses (PPL) and were undertaking commercial pilot license (CPL) training. This means they all had gone through the normal and abnormal procedures that are required for the license they hold. This was important as the research that was carried out tested the student pilot’s ability to recognize four trigger events of abnormal procedures: high oil pressure, vacuum pump system failure, alternator failure and high electrical load.
- Although, the volunteers were randomly selected the size of the volunteer population was not specified. As volunteers are self selected they may not be the representative of all student pilots and have different characteristics from non-volunteers. Additionally, no method was described on achieving equivalent groups to eliminate any potential selection difference between the control and experimental group.
Variable
The safety information was provided through Aircraft Discrepancy Analysis Metrics (ADAM) which is a database with the description of every aircraft discrepancy taken from logbooks that has been analysed.
1. Dependent Variables (DV): Flight student judgement
DV was measured using three variables 1) recognition time, 2) response time and 3) appropriate responses to the trigger events. Firstly recognition time was the number of seconds between the participants recognising the situation and making a verbal expression pointing out the performance indicator, or initiating a corrective action. Response time was the time (seconds) measured from recognition of abnormal aircraft condition up till the participants completed the corrective actions. The third variable was measured by analyzing accuracy the participant’s sequence of corrective actions.
2. Independent Variable (IV): (ADAM)
Participants are assigned to experimental conditions using independent group design. Experimental group were given access to ADAM throughout the study before they were tested on Flight Training Device (FTD). The control group did not have access to ADAM.
Instrumentation
- To measure participant’s decision processes two types of instruments were used. The Frasca 141, FTD and Situational Judgement Test (SJT).
- FTD was used to measure study participants’ decision process during abnormal flight conditions using two variables. These are recognition time and response time.
- Furthermore, SJT is a paper and pencil test that is used to examine the level of procedural accuracy. The SJT instrument is structured in a multi-choice format and the four alternatives to each question were ranked from most right to least right. Adherence of participant’s decision to the recommended solution was measured by comparing how close the participant’s choice was to the correct answers. The solutions are recommended by FAA-designated panel of subject matter experts and judgement/ decisions made by general aviation pilots.
Procedure
- The study experimental group were given access to current ADAM information through the University's online teaching structure WebCT. They could access ADAM data base throughout the duration of the study. The control group did not get access to ADAM.
- In FTD, the flight parameter was set uniformly for each participant and flight profile was divided into four legs. Each leg contained one trigger event and participants were told to make periodic heading or altitude changes to reduce anticipation of trigger events. Video recordings were made of each participant and their reactions to each trigger event. Timings were taken from the videos.
Data analysis
- A two-way Analysis of Variance (ANOVA) was used to analyse different corrective action scores. Analysis of Covaraince (ANCOVA), non-parametric statistical (Kruskal-Wallis) test.
Generalization potential
- Given the small convenient sample size that was taken out from baccalaureate degree program, it could be generalized to Purdue University student pilots doing similar training. However, it cannot be generalised to other populations. This is because their experience level may not be representative of diverse pilot training community.
- Also training standards which include memorising emergency checklists for every flight school may be different thus, this result can not be generalized as whole.
- The sample selected were students who were in their second to fourth semester. This could have threat to external validity because of the number of hours those participants will have before they volunteered themselves. For example, PPL holder will have around 50 hours compared to the pilots who are close to CPL who will have anywhere between 150 to 200 hours. This means the participants with more hours will possess more advanced theory and practical knowledge which could affect their results on DV. Thus, there could be significance impact of experience level on participant’s performance across all four areas that were monitored.
References
1. Lee, J. R., Fanjoy, R. O., Dillman, B. (2005).The effects of safety information on aeronautical decision making. Journal of Air Transportation, 2005, Volume 10, number 3.
Want to know more?
For details about Aeronautical Decision Making model, its use, process in pilot Aeronautical Decision Making
Wiki of Science Team (contributors to this page)
Authors / Editors
dotwikifan Abilash Thapa
