Effectiveness of basic skills training

Evaluation of training using different modules.

Bolstad et al1 carried out the research on evaluation of different computer based situation awareness training for the general aviation (GA) pilots. Results (see illustration 1) shows how effective basic skills training can be for the pilots with low flying hours. Checklist questionnaires, ATC comprehension and psychomotor skills were tested for each group of pilots. There was a drop of mean in post test performance (-.20) in psychomotor skills task ( tracking error) in the control group. However, the control group post test mean results were always better than their mean pre-test results in the checklist completion questionnaire, SAGAT queries on winds and altitude. This suggest that even the control group who had limited basic training, seemed to improve from their pre-results. Mean for the control group were always lower. This may be due to the fact that the control group was assigned the tedious task of reading and assembling 3-D figures for seven hours. It could be also due to their currency where the control group had an average of 3 months since their last flight compared to the experimental group with just 10.4 hours since their last flight. As a result the experimental group were more current and were up to date with the ATC comprehension, checklist completion and psychomotor skills. Training modules selected for the control group were also not directly associated with flying. In contrast, the experimental group was given more relevanct tasks that were directly associated with flying. For example, module was designed to enhance SA in GA pilots. A joy stick which is similar to control columns used by pilots in flying to gain their flying experience was used to train for the improvement of psychomotor skills, and a Microsoft visual basic module was provided as part of comprehension training.

Compared to the control group, the experimental group always outperformed in checklist completion questionnaire, psychomotor skills task at highest difficulty level, SAGAT query on current winds as well as altitude. This shows that training modules may have the potential for improving basic skills in pilots with low hours. Results showed that out of 15 SAGAT queries, the significant difference in change in score in experimental and control group were found in just two queries of current altitude and wind. This suggest out that majority of queries asked had no significant between two groups.

Illustration 1. Mean difference of pre and post-training performance by group
Results analysed by Variable measured Me Mc Result Better performance by
Basic skill performance Checklist completion questionnaire 3.33 .42 significant experimental
Reaction time not significant
No. of correct responses not significant
psychomotor skills task Highest difficulty level .66 -.20 significant experimental
Tracking error not significant
SA accuracy SAGAT queries
current winds .33 .19 significant experimental
current altitude .29 .01 significant experimental
on time not significant
current pitch not significant
indicated fuel not significant
altimeter setting not significant
current heading not significant
indicated airspeed not significant
ATC control in contact not significant
current altitude vs. planned not significant
conformance with clearance not significant
distance of special airspace not significant
planned vs current ground speed not significant
change required avoiding obstacles not significant
change required avoiding certain airspace not significant
Flight skills performance not significant
Me= mean of experiment group; Mc= mean of the control group; SAGAT= Situational Awareness Global Assessment Technique

Illustration two points out that the most progress in pre-test and post-test results of different variables by ranking order from one (best) to eight (worst). Illustration also shows that exp. group checklist completion questionnaire performed excellent whereas, exp. group psychomotor skills (highest level difficulty) and control group checklist completion questionnaire performed better. It is also noted that exp. group checklist completion questionnaire, exp. group SAGAT query (current altitude) and control group SAGAT query (current wind) performed good where as control group of SAGAT query (current altitude and Psychomotor skills ( highest level difficulty) performed bad.

Illustration 2. Performance of variable by ranking highest to lowest mean
Variable Group Mean difference pre and post Rank Performance interpretation *
Checklist completion questionnaire experimental 3.33 1 excellent
Psychomotor skills ( highest level difficulty) experimental 0.66 2 better
Checklist completion questionnaire control 0.42 3 better
SAGAT query: current wind experimental 0.33 4 good
SAGAT query: current altitude experimental 0.29 5 good
SAGAT query: current wind control 0.19 6 good
SAGAT query: current altitude control 0.01 7 bad
Psychomotor skills ( highest level difficulty) control -0.2 8 bad
*mean difference of < 1 is excellent, mean difference of 0.36 to .66 is better; mean difference of .02 to .35 is good; mean of 0.01 to -.34 is bad.


Research approach

  • This was an exploratory research to examine how effective basic skills training is for pilots with low flying hours using three modules.


  • A convenience sample of 24 pilots from the Embry-Riddle Aeronautical University(ERAU) in Daytona Beach, Florida. The experimental group has 12 pilots with a average flight hours of 98.4 and the control group has 12 pilots with average of 104.3 flight hours.
  • 2 participants in each group had instrument rating .
  • Pilots were chosen from ERAU which represented typical GA pilots. Although important to generalise to populaton about hte method was important to mention, how participants were self selected or randomly selected from ERAU is not given.


Between group design. Participants were not assigned to each groups randomly and this does not ensures extraneous variable are equally split. Thus, this will help in becoming confounding variable.


  • Dependent Variable: Checklist completion knowledge, ATC comprehension, psychomotor task performance, SA accuracy and flight skills performance.
  • Independent Variable: Three SA training modules.
  • Confounding Variable: The experimental group had average of 10.4 hours since last flight and control group had average of three months since last flight. This lack of currency of 3 months of control group may have had an negative effect on their performance.


  • Performance of 5-item questionnaires to asses pilot's knowledge of checklist procedure, psychomotor skills training ( highest level of difficulty, tracking error) and ATC comprehension training module (response and reaction time) were collected pre and post training.
  • Macromedia director was used to describe the problem that can occur when checklist is not completed, reasons why they are not completed and methods to guide pilots in carrying out checklist.
  • Microsoft visual basic module to check ATC comprehension training.
  • Vigilance was assessed using the Mackworth clock test taken from Teikari (1977) which was run in a computerised environment.
  • Aeronautical Information Manual/ Federal Aviation Regulation 2000, The pilot's radio communication handbook and Interlocking Kinex brand constructions sticks
  • Microsoft Flight Sim 2000, 15 SAGAT queries, personal computer with three screens for SA training module, headphones, joystick and yoke.


  • Participants were asked to complete the checklist questionnaires and they performed pre-training ATC comprehension, psychomotor skills task and flight scenario. Before flight, practice flights were given to the pilots to practice on SAGAT and the simulator.
  • In their own time, participants of both groups were told to spend 7 hours completing training modules. Each pilot kept their log recording of the time spent on training and software that is used by experimental group recorded the participants time on task for comprehension training and checklist completion training.
  • After completion of the training, participants completed the post-training measures for basic skills performance and post training flight scenario.
  • Finally, participants were debriefed and paid nominal fee for their participants.

Generalisation potential

  • As sample was taken from large university with about 2000 student pilots, these results could be applied to typical GA pilots of similar age with similar flying hours of 98 to 104 .
  • As convenience sample was taken without any randomisation of selection process, the results should be generalised to similar university population only.
1. BOLSTAD Cheryl A, Mica R ENDSLEY, Anthony COSTELLO A and Cass D Howell (2010). Evaluation of computer based situation awareness training for general aviation pilots. The International Journal of Aviation Psychology ( ISSN 1532-7108), 2010, volume 20, number 3, pages 269-294.

Want to know more?

Situation Awareness : This pages defines SA, level of SA, factor influencing and challenges of SA

Original article : Full original article of this research can be obtained here.

Contributors to this page

Authors / Editors

Abilash Thapa (2013), Massey University, New Zealand (Thapa AbilashThapa Abilash)

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License