1 FAA Academy Int’l Training Federal Aviation Administration 1 2000 The MITRE Corporation. All rights reserved. 1 FAA Academy International Training Federal Aviation Administration Safety Management Systems SMS Implementation & Tools

2 FAA Academy Int’l Training Federal Aviation Administration 2 2000 The MITRE Corporation. All rights reserved. 2 U.S. FAA SMS Framework The first version of the FAA’s standards was published in AC 120-92, Appendix 1The first version of the FAA’s standards was published in AC 120-92, Appendix 1 Each element in the U.S. standard has an associated functional objective – a performance expectation of each processEach element in the U.S. standard has an associated functional objective – a performance expectation of each process AVS is publishing a similar set of standards for both internal and external SMSsAVS is publishing a similar set of standards for both internal and external SMSs

3 FAA Academy Int’l Training Federal Aviation Administration 3 2000 The MITRE Corporation. All rights reserved. 3 Organization of the SMS Standard Safety Risk Management (SRM) Cl. 5 Safety Promotion (SP) Cl. 7 Safety Assurance (SA) Cl. 6 Policy Cl. 4 Data Analysis Prev/Corr Action Audits Invest. Reports Assmt Comm Training Risk Analysis Risk Assmt Risk Control Hazard Ident System Descrip Mgmt Structure Process Controls Procedures

4 FAA Academy Int’l Training Federal Aviation Administration 4 2000 The MITRE Corporation. All rights reserved. 4 SMS System Elements (“System 8”) 8.1 Organizational Management (Policy) 8.1.1 Organizational Management8.1.1 Organizational Management 8.1.2 Document & Records Management8.1.2 Document & Records Management 8.1.3 Emergency Response8.1.3 Emergency Response 8.2 Safety Risk Management 8.2.1 System/Task Analysis8.2.1 System/Task Analysis 8.2.2 Hazard Identification8.2.2 Hazard Identification 8.2.3 Risk Analysis/ Assessment8.2.3 Risk Analysis/ Assessment 8.2.4 Risk Control8.2.4 Risk Control 8.3 Safety Assurance 8.3.1 Information Management 8.3.2 System Analysis/ Assessment 8.3.3 Preventive/Corrective Action 8.4 Safety promotion 8.4.1 Preventive/Corrective Action 8.4.2 Process Qualification & Training

5 FAA Academy Int’l Training Federal Aviation Administration 5 2000 The MITRE Corporation. All rights reserved. 5 Methodology of SMS Implementation Phased implementation: a logical process for building the SMSPhased implementation: a logical process for building the SMS Management capability: Each step in the model will increase and strengthen management capabilityManagement capability: Each step in the model will increase and strengthen management capability Process based uponProcess based upon ICAO recommendations andICAO recommendations and Capability Maturity Model (CMM) conceptsCapability Maturity Model (CMM) concepts

6 FAA Academy Int’l Training Federal Aviation Administration 6 2000 The MITRE Corporation. All rights reserved. 6 Who’s Involved in Implementation? The OperatorThe Operator The Certificate Management Organization (CMO, FSDO, CHDO)The Certificate Management Organization (CMO, FSDO, CHDO) The Standardization and Assistance Team (SAT)The Standardization and Assistance Team (SAT)

7 FAA Academy Int’l Training Federal Aviation Administration 7 2000 The MITRE Corporation. All rights reserved. 7 SMS Implementation Process 0 Orientation & Commitment 2 Reactive Processes 3 Proactive Processes 4 Continuous Improvement 1 Planning & Organization

8 FAA Academy Int’l Training Federal Aviation Administration 8 2000 The MITRE Corporation. All rights reserved. 8 Draft AC 120-XX Contents Description of LevelsDescription of Levels Implementation ProcessImplementation Process Gap Analysis ToolGap Analysis Tool Functional Objectives Functional Objectives Maturity Rating Maturity Rating Cross Reference to Detail Design Assessment ToolsCross Reference to Detail Design Assessment Tools Cross Reference to ICAO FrameworkCross Reference to ICAO Framework

9 FAA Academy Int’l Training Federal Aviation Administration 9 2000 The MITRE Corporation. All rights reserved. 9 Voluntary Implementation of SMS Advisory Circular has been DraftedAdvisory Circular has been Drafted Process Based on AC 120-92Process Based on AC 120-92 Interim Solution while rulemaking is ongoingInterim Solution while rulemaking is ongoing Award SystemAward System Bronze: Maturity Level 2: Risk Management ProgramBronze: Maturity Level 2: Risk Management Program Silver: Maturity level 3: SMS (provisional)Silver: Maturity level 3: SMS (provisional) Gold: Maturity Level 4: SMSGold: Maturity Level 4: SMS

10 FAA Academy Int’l Training Federal Aviation Administration 10 2000 The MITRE Corporation. All rights reserved. 10 Gap Analysis Detailed briefing on the SMS standardDetailed briefing on the SMS standard Comparison between existing company systems and SMS requirementsComparison between existing company systems and SMS requirements Summary gap analysis with SAT team supportSummary gap analysis with SAT team support Company performs detailed gap analysisCompany performs detailed gap analysis Uses gap analysis tool from draft AC 120-XX Uses gap analysis tool from draft AC 120-XX Detail for analysis of SMS processes in 12 Design Assessment Tools Detail for analysis of SMS processes in 12 Design Assessment Tools

11 FAA Academy Int’l Training Federal Aviation Administration 11 2000 The MITRE Corporation. All rights reserved. 11 Implementation Plan Based on sequence in AC 120-XXBased on sequence in AC 120-XX Sequence can be varied to meet company and CMT schedules and resourcesSequence can be varied to meet company and CMT schedules and resources Completed plan ends Level oneCompleted plan ends Level one

12 FAA Academy Int’l Training Federal Aviation Administration 12 2000 The MITRE Corporation. All rights reserved. 12 SMS Pilot Projects 0 Orientation & Commitment 2 Reactive Processes 3 Proactive Processes 4 Continuous Improvement 1 Planning & Organization Phase 1: Readiness Phase 2: Experience

13 FAA Academy Int’l Training Federal Aviation Administration 13 2000 The MITRE Corporation. All rights reserved. 13 SMS Guidance and Tools SMS Standard: AC 120-92 Appendix 1SMS Standard: AC 120-92 Appendix 1 ISO 9001/14001 FormatISO 9001/14001 Format Gap Analysis/Audit Summary ToolGap Analysis/Audit Summary Tool Based on specific objectives related to each subclause of the standardBased on specific objectives related to each subclause of the standard Detailed assessment toolsDetailed assessment tools Format of ATOS SAIsFormat of ATOS SAIs SMS GuidebooksSMS Guidebooks Format of ISO 9004Format of ISO 9004 AC 120-XX (Draft) Voluntary SMS ImplementationAC 120-XX (Draft) Voluntary SMS Implementation

14 FAA Academy Int’l Training Federal Aviation Administration 14 2000 The MITRE Corporation. All rights reserved. 14 “Carelessness and overconfidence are more dangerous than deliberately accepted risk” Wilbur Wright, 1901 Contact: AFS SMS Program Office Don Arendt, Ph.D., Manager (703) 661-0516 (703)[email protected] Wilbur Wright gliding, 1901 Photographs: Library of Congress

15 FAA Academy Int’l Training Federal Aviation Administration 15 2000 The MITRE Corporation. All rights reserved. 15 Backup Slides

16 FAA Academy Int’l Training Federal Aviation Administration 16 2000 The MITRE Corporation. All rights reserved. 16 Level Zero: Orientation and Commitment Outreach presentationsOutreach presentations Understanding what’s aheadUnderstanding what’s ahead Establishing the relationshipEstablishing the relationship Securing the commitment from top managementSecuring the commitment from top management

17 FAA Academy Int’l Training Federal Aviation Administration 17 2000 The MITRE Corporation. All rights reserved. 17 Level 1: Planning & Organization Charting the Course The Gap Analysis: Evaluating existing processes, programs, and practices for safety managementThe Gap Analysis: Evaluating existing processes, programs, and practices for safety management The Implementation PlanThe Implementation Plan Organizing for ImplementationOrganizing for Implementation

18 FAA Academy Int’l Training Federal Aviation Administration 18 2000 The MITRE Corporation. All rights reserved. 18 Level 2: Reactive Processes Basic Risk Management Setting up the information infrastructure and processesSetting up the information infrastructure and processes Going after known problems: Reactive Hazard IdentificationGoing after known problems: Reactive Hazard Identification Designing and implementing risk controlsDesigning and implementing risk controls DocumentationDocumentation TrainingTraining General initial trainingGeneral initial training Specialist trainingSpecialist training

19 FAA Academy Int’l Training Federal Aviation Administration 19 2000 The MITRE Corporation. All rights reserved. 19 Level 3: Proactive Processes Looking Ahead System and task analysis of the operational (production) systemsSystem and task analysis of the operational (production) systems Proactive Hazard identificationProactive Hazard identification Updating:Updating: the risk controlsthe risk controls Updating the documentationUpdating the documentation Additional specialist trainingAdditional specialist training

20 FAA Academy Int’l Training Federal Aviation Administration 20 2000 The MITRE Corporation. All rights reserved. 20 Level 4: Continuous Improvement Continued Assurance Mature safety assurance processMature safety assurance process Analysis of dataAnalysis of data System performance assessmentSystem performance assessment Corrective/Preventive action: Maintaining the controlsCorrective/Preventive action: Maintaining the controls Identifying new and emerging hazardsIdentifying new and emerging hazards Management Reviews: Formal involvementManagement Reviews: Formal involvement Documentation & Training as requiredDocumentation & Training as required

21 FAA Academy Int’l Training Federal Aviation Administration 21 2000 The MITRE Corporation. All rights reserved. 21 Level 4: Continuous Improvement This is the final stage of maturity of the SMSThis is the final stage of maturity of the SMS Continuous improvement is based on:Continuous improvement is based on: Continuous risk management, andContinuous risk management, and Continuous safety assuranceContinuous safety assurance

22 FAA Academy Int’l Training Federal Aviation Administration 22 2000 The MITRE Corporation. All rights reserved. 22 Summary Audit/Gap Analysis Tool No action = 0 Action initiated=1 Implemented=2 Integrated=3 Evaluated and Sustained=4 State of Art=5

23 FAA Academy Int’l Training Federal Aviation Administration 23 2000 The MITRE Corporation. All rights reserved. 23 Assessment Tools Utilizes ATOS SAI format Each has functional objective - expectations and outcomes Questions in the procedures section of each element are based on specific requirements in the standard Requirements evaluated using assessment scales that measure level of conformance with each requirement 0 – No action 1 – Action initiated 2 – Implemented 3 – Integrated 4 – Evaluated and Sustained 5 – State of Art

24 FAA Academy Int’l Training Federal Aviation Administration 24 2000 The MITRE Corporation. All rights reserved. 24 Example: Design Assessment Tool

25 FAA Academy Int’l Training Federal Aviation Administration 25 2000 The MITRE Corporation. All rights reserved. 25 Example: Design Assessment Tool: Detail