|[Data]||[<Normal page] [PEREZGONZALEZ Jose D (1997). Safety management systems in aviation maintenance (descriptive statistics). Knowledge (ISSN 2324-1624), 2013, pages 18-21.] [DOI]|
Illustration 1 tabulates the main results so that each AMO (columns) gets ranked within each safety variable (rows). Rankings are in ascending order, with poor performance attracting lower rankings (eg, 1 or -4) and good performance attracting higher rankings (eg, 4 or -1). This procedure allows for adding up each organization's ranking at the bottom of the table to gain an appreciation of how good the organization is overall. The procedure also allows for setting an ordinal effect size of approximately 15 units on the resulting scale as a big effect size5.
Illustration 1 shows that there were sensible differences in safety management styles among the four organizations. Organization C, the best organization in the group, achieved position 43 on the scale (out of 50). At a sensible distance (15 units on the scale) was organization A in position 28, followed by organization B, also at a relatively sensible distance (12 units on the scale), in position 16. Organization D, the worst organization in the group, achieved position 10, only slightly lower (6 units on the scale) than organization D .
|Illustration 1: Elements of safety management across four aircraft maintenance organizations|
|Safety policy||3, Good||2, Basic||4, Strong||1, Ineffective|
|Safety standards||2, Good||1, Basic||3, Strong||1, Basic|
|Planning & documentation||3, Good||2, Improving||4, Strong||1, Ineffective|
|Management training||2, Partly effective||1, Inconsistent||3, Effective||1, Ineffective|
|Auditing||1, Statutory||2, Basic||3, Strong||1, Statutory|
|Reporting||3, Backlogged||1, Starting||1, Starting||2, Poor|
|Incident investigation||3, Good||2, Ineffective||4, Strong||1, Absent|
|Feedback||3, Good||2, Ineffective||4, Strong||1, Poor|
|Change||3, Good||2, Ineffective||4, Strong||1, Absent|
|( SMS quality )||( 23 )||( 15 )||( 30 )||( 10 )|
|Climate & attitudes||A||B||C||D|
|Safety climate||3, Slightly positive||1, Negative||4, Positive||2, Slightly negative|
|Safety attitudes||2, Better||2, Better||2, Better||1, Good|
|( safety climate )||( 5 )||( 3 )||( 6 )||( 3 )|
|Impartial investigation||3, ≃53%||2, ≃35%||4, ≃62%||1, ≃28%|
|Incident discussed||3, ≃40%||1, ≃23%||4, ≃51%||2, =33%|
|Verbal reprimand||-2, ≃10%||-2, ≃10%||-3, ≃15%||-1, ≃7%|
|None||-3, ≃3%||-4, ≃4%||-1, ≃0%||-2, ≃3%|
|Disciplinary hearing||-4, ≃45%||-1, ≃23%||-2, ≃35%||-3, =37%|
|Suspension of license||-2, =15%||-4, =51%||-1, ≃8%||-3, ≃18%|
|Suspension of job||-4, =19%||-2, ≃9%||-1, ≃3%||-3, ≃13%|
|( positive discipline )||( 6 )||( 3 )||( 8 )||( 3 )|
|( negative discipline )||( -15 )||( -13 )||( -8 )||( -12 )|
|Reported violations||-1, =30%||-2, =36%||-3, =37%||-4, =47%|
|( behavior )||( -1 )||( -2 )||( -3 )||( -4 )|
|SMS & culture (0-50)||28||16||43||10|
|Ordinal||small||medium||large||( Prototype diff. )|
|effect size5||5||10||15||( 16 )|
Illustration 2 summarizes the correlations found between above groups of safety elements. In order to facilitate interpretation of the results, all ranks were transformed into positive values.
|Illustration 2: Correlations (Spearman)|
As illustration 2 shows, there were high and positive ranked correlations6 between SMS, organizational climate, and positive discipline, which suggest that better safety management systems went together with better climate and more constructive disciplinary cultures (and viceversa) among these four organizations. Smaller and negative correlations appeared between above groups of variables and negative disciplinary expectations and violations, which also hint to relationships between better SMS, climates and constructive cultures and lower rates of negative disciplinary expectations and lower reported violations. Notwithstanding this, there was also a relatively high and negative correlation between preference for punitive disciplinary actions and the amount of reported violations.
Although little can be generalized from this particular research, above results describe a situation were safety management and safety culture did not appear standardized among similar organizations doing similar jobs and having similar responsibilities for aviation safety. Safety management and safety culture appeared as quite variable, with little cross-over across organizations, at least at the time of the research.
On the other hand, safety management and safety culture also appeared as quite consistent within each organization, further suggesting that the quality of the management system and the safety culture was idiosyncratic to each organization, was coherent within it, and permeated all levels in that organization.
Exploratory study of the safety management systems of aircraft maintenance organizations3.
Case study, where most information obtained during the research was compiled to provide a snapshot of the management style of four AMOs.
The ultimate sample was the four organizations, once all results were compiled according to provenance. Nonetheless, information for that sample was obtained by researching several populations and subsamples within each organization, each informing about different aspects of safety management at different organizational levels. The sample and subsamples were, thus, the following:
- An overall sample of four AMOs, selected in a convenient manner among European organizations.
- The population of four 'Maintenance Exposition Documents' (MOEs), the required formal documentation depicting maintenance roles and responsibilities, maintenance procedures and quality systems.
- A sample of 33 maintenance and safety managers (including CEOs, production and quality managers, auditors and training personnel), selected in a convenient manner across all four organizations, who were interviewed about safety management.
- 622 participants, spanning from managers to technicians, selected in a convenient manner across all four organizations, who filled in a survey about safety attitudes and safety climate.
- 286 technicians selected in a convenient manner across all four organizations, who filled in a questionnaire about the use of official maintenance procedures in the job.
- Content analysis of documentation.
- Semi-structured interviews with management personnel.
- Questionnaires, using a combination of dichotomous and Likert-like scales, as well as open-ended questions.
Overall, there is no much information about the validity and reliability of the different tools and procedures used. There are indications of face-validation for the results derived from content analysis as well as for sections of the surveys. The authors also went into some detail explaining how they adapted the safety and climate survey developed by Isla Diaz and Diaz Cabrera (19971). Overall, however, there is not much guarantee of proper validation or reliability assessment of the tools and procedures used.
- Qualitative analysis (content analysis of documentation and interviews).
- Quantitative analyses, namely frequency percentages and inferential analyses (ANOVAs and chi-square)7.
To above analyses carried out by the authors, the following analyses were introduced anew here:
- Ranking of the AMO's according to perceived quality in all important variables.
- Setting up of appropriate ordinal (unstandardized) effect sizes for interpretation purposes.
This research has little generalization potential. The sample of AMOs is too small for carrying out inferential analyses, and little is known about the quality of the tools and procedures used. The convenient approach to sampling and tool selection, coupled with results which show rather idiosyncratic safety management styles within each organization, also limit generalization.
Want to know more?
- FAA - Safety Management System
- This FAA website provides more information about safety management systems in aviation.
- Wiki of Science - Safety management systems in aviation maintenance (introduction)
- This Wiki of Science page is an accompanying introduction to above article.
Jose D PEREZGONZALEZ (2012). Massey University, Turitea Campus, Private Bag 11-222, Palmerston North 4442, New Zealand. (JDPerezgonzalez).