PC-Based system enhancing cockpit resource mangement

Usefulness of PC-based system enhance flight-deck team work.

Brannick, Prince and Salas (2005) explored whether a PC- based system can enhance teamwork in the flight deck. They experimented using a PC-based instructional tool to teach Crew Resource Management (CRM) to a group of US Navy helicopter pilots who had completed their flight training. Results were compared with a similar group who practiced in a different way. It was found that there is a positive transfer of CRM skills from PC-based training to the high-fidelity simulator. The experimental group performed better on CRM skills compared to the control group (Table 1). These observed behaviors related to the transfer of tasks and the coordination that occurred between pilots and Air Traffic Controller (ATC) in the training scenarios. The analysis of variance (ANOVA) showed that the differences were significant for two of the five behaviors (p <0.05). The experimental and control group did not differ in their technical problem solving proficiency (Table 1).

Table 1. Evaluation by instructors between groups
Item evaluated	Mt	Mc	F *	Result	Interpretation of performance
Asked ATC for missing destination limit on clearance for take-off	0.88	.09	88.6	significant	Trained group better
Checked the legality of descending from 7000 feet to 3000 feet	0.55	0.2	7.12	significant	Trained group better
Icing problem	3.3	3.03	0.49	not significant	Both groups similar
Boost pump problem	3.83	3.42	3.15	not significant	Both groups similar
Electric fire	2.91	2.9	0.0	not significant	Both groups similar
Mt= mean of trained group; Mc= mean of the control group; * P<.05

---

Methods

Research approach

• This research was an exploratory experimental study of helicopter pilots to find out whether there was positive transfer of CRM skills from PC- based system to a more complex system, that is full axis simulator.

Sample

• A convenience sampling sample of 96 U.S. Navy pilots were selected from two squadrons and were randomly assigned to teams, two participants in each team. This made a total of 48 teams, 24 in the experimental group and 24 in the control group.

• These participants have already completed flight training and were waiting to begin co-pilot training as on helicopters. This was crucial as technical skills, legality to descend, clearance for take-off has all been part of their flight training.

Design:

A posttest-only design. A independent group design; i.e participants were randomly assigned to the experimental and control group so that each participates in only one group.

Variable

• Independent Variable is the feedback. Experimental group completed training scenarios and received feedback. Where as, the control group were given set of exercises and did not receive feedback.

• Dependent Variable is transfer of CRM skills i.e being able to transfer skill "assertiveness" from PC-based to full-motion simulator.

Materials

• Micro computer and flight Simulator software were used to measure the transfer of CRM skills from a PC-based system to flight simulator.

• The PC-based scenario required a headset for communication between pilot and ATC; an actor to played the part of ATC. In addition, maps and flight documents that are used in real flight were used.

• Two outcomes were measured to evaluate the effects of CRM training. The first was to identify two specific behaviors identified in checklist format. First was two specific behaviors presented in checklist format. These items were first two questions in Table 1. The navy instructors rated students' behavior either using "yes" or "no". This, meant that the behavior was observed (scored 1) or no, which meant that the behavior was not observed (0).

• The second outcome measured was the students' response to a series of three issues which were built into the scenarios; icing problem, boost pump problem and an electrical fire. The instructors rated student teams using Likert-type scale ranging from 1 (unsatisfactory) to 5 (outstanding). Instructors made the rankings from video-taped performance which was recorded in full-motion simulator.

Procedure

• Before being assigned to teams, a one hour briefing was given to all 96 participants. An instructor also provided a lecture that reviewed all elements of CRM; including communication, assertiveness and situation awareness. Participants were also shown where aircrew, acting in scenarios, demonstrated these three skills.

• Two scenarios were developed for team coordination assessment. The first was training scenario on a micro-computer where the ATC made two mistakes. The ATC omitted information that should have been part of routine instructions and also directed pilots on a route which leads into thunderstorm. In the second scenario, in the flight simulator, the ATC repeated the same omission of clearance information. These kind of errors, that had been part of the PC-based training, were reproduced in the simulator scenario. ( ATC omission & need to check legality/ prudence to descend). Also, pilots in the training group checked the legality/ prudence to descend from 7000 to 3000 feet. Only the experimental group had experience with ATC mistakes.

• The study's experimental group completed the training scenario through role-play and the PC-based practice. They received detailed feedback from an instructor pilot on their CRM skills exhibited in role play. Feedback also involved instructors discussing the problems integrated in the scenario.

• The control group were given a set of exercises. These included a problem solving exercise and computer video game. The control group spent equivalent amounts of time in exercises given and used the computer for a similar length of time. This group did not receive any feedback on their performance. A major difference between the experimental and control group was the amount of feedback details. After the end of first session each team was briefed about the scenario to be completed in full axis simulator and given materials required they would receive in actual flight preparation. The students' planning sessions were recorded on audiotape.

• Instructors were blind to the conditions, so they were unsure of which group belonged to the trained and control group. The reliability of measurements was achieved by designing an easy to use checklist and training in its use. Further, the reliability estimate of inter-judge agreement of five items were all high ( alpha values = .80 to .92).

Data analysis

• Univariate Analysis, ANOVA, Cronbach's Alpha was used to check interrater reliability.

Generalization potential

• Given the sample was taken from U.S. NAVY rotary wing pilots, results could be applied to pilots doing similar training. However, since participants were all navy pilots, it may not be necessarily generalised to civil and fixed wing pilot communities. 

• As this research was carried out when workload are relatively low, transfer of training from PC to full axis simulator may not be applicable during take-off and landing phase when workload are at maximum.

• There is one issue that is unclear and that is the amount of feedback received by the control group. The author discussion appears to contain a contradiction on p.183. They state that both experimental and control group "had the experiences during the training that involved the monitoring and feedback of team performances". But on page 181, they state " the control teams did not receive feedback on their performance". This is a unfortunate as the usefulness of feedback was noted as an important factor for understanding results.

References

1. Brannick Michael, Prince Carolyn, & Salas Eduardo. (2005). Can pc-based system enhance teamwork in the cockpit. The International Journal of Aviation Psychology, 2005, volume 15, number 2, pages 173-187.

Want to know more?

For full details about the above article click Can PC-based system can enhance teamwork in the cockpit?

For more details about CRM see Crew Resource Managment

For more details about improving CRM see Strategy to Improve Crew Resource Management