In 2007, Stedmon, Sharples, Littlewood, Cox, Patel, and Wilson developed a pilot study to investigate issues associated with different levels of trust between real and synthetic communications, and also the perception of male and female sythesised speech in relation to trust.1 They asked a group of 48 students to participate in an experiment, which assessed their situational awareness, while they were watching aircraft move on a computer screen, and determined their trust levels, with each aircraft.
The results were analysed into four categories, Trust, Perfomance (RTs to rate trust), Performance (response rate to number of commands), and situational awareness. As expected, the results showed real speech is trusted more than synthetic speech, however, the gender of the voice has, unexpectedly, no significance on the levels of trust. The following table displays and interprets the results:
| Trust | P Value | Significance | Interpretation |
|---|---|---|---|
| Mode of Communication | P<0.001 | Significant | Real speech is trusted more than synthetic speech |
| Accuracy of Communication | P<0.001 | Significant | True statements are trusted more than false statements |
| Interaction (Mode and Accuracy) | P<0.01 | Significant | True communication in real speech was trusted more than, false communication in real speech, true communication in synthetic speech or false communication in synthetic speech. In addition, true communication in synthetic speech was trusted more than false communication in real or synthetic speech |
| Voice Gender | P>0.05 | Insignificant | The gender of the voice has no effect on trust |
| Performance (RTs to rate Trust) | |||
| Mode of Communication | P<0.01 | Significant | Real speech was responded to faster than synthetic speech |
| Performance (Response Rate of Number/Type of Commands) | |||
| Response | P<0.001 | Significant | Number of correctly identified communications more than the number of incorrectly identified |
| Situational Awareness | |||
| Between Tests | P<0.001 | Significant | Significant increase in situational awareness between trial one and three, and similarly between trials two and three |
| Position Situational Awareness | P<0.001 | Significant | Significant increase inpositional situational awareness between trial one and three, and similarly between trials two and three |
| Direction Situational Awareness | P<0.01 | Significant | Significant increase in Directional situational awareness between trials two and three |
| Identity Situational Awareness | Insignificant | No significant changes between trials |
Methods
Research approach
- This was a pilot study to investigate the issues associated with different levels of trust between real and synthetic communications and also the perception of male and female synthesized speech in relation to trust.
Sample
- 48 participants were recruited from an opportunistic sample. The sample included 24 females and 24 males, with an average age of 20.9, and a range of 19 to 23 years.
- Each participant spoke English as their first language, had normal, or corrected to normal vision, and no previous ATC experience.
- Each participant was paid for their participation.
Design
- Each participant was familiar with the equipment, from a previous experiment earler that day, conducted on the same materials.
- The scenarios were counterbalanced between participants, to minimize the effects of the improved results between the trials.
- False communications would not be carried out by the aircraft
Variables
- The dependent variables were trust, performance and situational awareness.
- The independant variables were mode of communication (real speech and synthetic speech), and accuracy of communication (true or false)
Materials
- A Microsoft Powerpoint presentation on a 700 MHz PC and monitor, was used for an exercise which was presented to each of the participants.
- Each session was recorded by a JVC digital video camera in order to ensure that RTs for trust ratings could be calculated.
- Synthetic speech was produced using AT&T Natural Voices, Text-to-Speech engine
Procedure
- In this exercise, each of the participants were required to monitor the aircraft as they moved through the sector on a computer screen. * They were asked to provide a verbal trust rating for each command they heard.
- At three predetermined times throughout the exercise, unknown to the participant, the participant was required to complete a situational awareness test.
Data analysis
- T tests (one and two sided) were used to detemine the significance of the events.
- To meet the assumptions for parametric analysis the data was tested for normality and equality of variance.
- Post-hoc Tukey tests were conducted where necessary, to find which means are significantly different from one another2
Generalization potential
- Given the exploratory approach to research, the sample size and its convenience, the results of the study may not have sufficient scope for generalization. However, aviatiion as an industry is moving more towards synthetic speech, therefore these results could provide reason for further in depth research to ensure our aviation industry is the safest it can be.
- The results could be indicative of other populations, including:
Pilots, Air traffic controllers, Police and emergency services, and the general public, when radio transmissions are used.
References
1. http://www.sciencedirect.com.ezproxy.massey.ac.nz/science/article/pii/S0003687007000282
Applied Ergonomics Volume 38, Issue 4, July 2007, Pages 473–480 ISSN: 0003-6870
2. http://en.wikipedia.org/wiki/Tukey%27s_range_test
Flightdeck of the Future
Want to know more?
- item
- explanation
Contributors to this page
Authors / Editors
| Other interesting sites |
 Knowledge |
 WikiofScience |
 AviationKnowledge |
 A4art |
 The Balanced Nutrition Index |
