KNIT

KSC NIAT Implementation Team (KNIT) Working GroupStatus UpdateNovember 13, 2001: 

KSC NIAT Implementation Team (KNIT) Working Group Status Update November 13, 2001 Joni Richards/QA-D/7-4817

Agenda: 

Agenda Purpose for Meeting 3 Working Group Members 4 Executive Board Members 5 Background 6 Survey for establishing baseline (e.g. NIAT-1) 9 Survey Results with Top 5 NIAT Actions 10 What to Implement? 11 Future Plans 12 Other formal HQ NIAT Actions 13 Potential Impact Statement 14 Summary 15 2

Purpose for Meeting: 

Purpose for Meeting Per the Executive Board Charter (signed 2/12/01)— The Working Group is to obtain direction from the Executive Board for assuring that the recommendations of the NIAT report are implemented at KSC. Working Group Deliverables include: KSC “Baseline”Assessment NIAT Implementation Plan To obtain approval of the NIAT Response and Implementation Plan to the NASA Integrated Action Team (NIAT) Recommendations 3

Working Group Members: 

Working Group Members Walt Feitshans, YA/Spaceport Engineering & Technology Roger Hall, UB/International Space Station/Payload Processing Ruth Harrison, PH/Space Shuttle Processing Joan Lyons, GG/Chief Financial Office Henry Molnar, OP/Procurement Gordon Perry, TA/Spaceport Services Joni Richards (Chair), QA/Safety Health & Independent Assessment Mark Ruether, VB/Expendable Launch Vehicles & Payload Carriers Pam Steel, XA/External Relations & Business Development Jim Thompson, BA/Workforce & Diversity Management 4

Executive Board Members: 

Executive Board Members Ken Payne (Chair), YA/Spaceport Engineering & Technology David King (Co-Chair), PH/Space Shuttle Processing Directors of: BA/ Workforce & Diversity Management GG/Chief Financial Office OP/Procurement QA/Safety Health & Independent Assessment Chief Engineer/Systems Management Office TA/Spaceport Services UB/International Space Station/Payload Processing VA/Expendable Launch Vehicles & Payload Carriers XA/External Relations & Business Development 5

Background: 

Background In addition to the four program-specific assessments such as the Mars Program and the Shuttle wiring problems, the NASA Administrator recognized the need to assess and respond to findings and recommendations that could be more broadly applied to the wide range of NASA projects. This resulted in the HQ/Chief Engineer chartering a NASA-wide team to evaluate the recommendations from the four assessments and define specific set of actions that focus on critical implementation issues. The goal is to strive for the reduction of residual risk to that inherent or acceptable which is understood and agreed to by the program/project, GPMC, and the customers. The NIAT actions represent a “required state” for implementing NASA programs and projects. Themes: Developing and supporting exceptional people and teams Delivering advanced technologies Understanding and controlling risk Ensuring formulation rigor and implementation discipline Improving communication 6

Background: 

Background January 11, 2001, the Administrator and the Chief Engineer communicated their views on the NIAT recommendations via webcast February 8: KSC Daily News article Encouraging all to be familiar with the NIAT Report Website to obtain report, https://webcasts.hq.nasa.gov/niat/niatfinal1221.pdf February 9: NIAT Working Group Kick-off Meeting Familiarized team with Executive Board and Working Group charters Discussed ambitious schedule and survey format for establishing a center baseline February 15: HQ Senior Management met to discuss approach to assess ongoing programs/projects against the NIAT Actions. February 16: the Administrator and Chief Engineer met and agreed that immediate focus would be on NIAT actions 7, 8, 10, 11, 13, 14, 15 for those projects in “Implementation” or “Operational” phase (i.e. ELV and CLCS projects). 7

Background: 

Background February 16: NIAT Working Group Meeting Team had begun to canvass their individual directorates to score the survey and determine the Top 5 Actions with the highest payoff March 2: NIAT Working Group Meeting Informed team that I forwarded our survey to HQ/Orlando Figueroa as suggestion for gathering consistent data from the field centers Team agreed on survey completion date of March 5 March 8: NIAT Working Group Meeting Team agreed to complete Supporting Data on their Top 5 Actions by mid-June March 20: Working Group Status presentation to the Executive Board of Baseline Assessment (survey results) and obtain approval to proceed. June 19: Presentation to CD in preparation for the Wye River Plantation Meeting with Headquarters’ Associate Administrators. August 13: Final Working Group meeting to cover any outstanding issues. 8

Survey (sample of NIAT 1 of 17) : 

Survey (sample of NIAT 1 of 17) Remaining NIAT Action Titles: 2: Development of the Workforce 10: Integrated Review Process 3: Revitalizing Engineering 11: Ensuring Adequate Resources 4: Balanced Technology Investment 12: Faster, Better, Cheaper 5: Integrated Technology Planning Process 13: Surveillance 6: Technology Development and Infusion 14: Verification and Validation 7: Risk Identification, Assessment & Management 15: Managing Responsibility & Accountability 8: Safety and Mission Assurance 16: Organizational Communication 9: Software Development and Assurance 17: Knowledge Management 9

Prioritization of NIAT Actions by Organization (Top 5): 

Prioritization of NIAT Actions by Organization (Top 5) *Note: YA has identified 7 priorities including 9 & 15 Essentially, all directorates performed a rough gap analysis that formed the basis for the center Implementation Plan. After comparing directorate results, the overall ranking that resulted for KSC was -- 1: Health and Safety 2: Development of the Workforce 7: Risk Identification, Assessment and Management 8: Safety and Mission Assurance 16: Organizational Communications 10

What To Implement?: 

What To Implement? Based on the supporting data in Appendix D along with the detailed explanations in the section titled--KSC’s Top 5 Actions with Supporting Evidence (p. 7), the Working Group agreed that there were no specific actions to implement at this time. As a result, the Working Group voted to rename the plan to— “Response and Implementation Plan to the NASA Integrated Action Team (NIAT) Recommendations” The Working Group believed that the initiatives that are currently underway was sufficient evidence that closed the gaps identified from the survey. 11

Future Plans: 

Future Plans Since the NIAT Report indicates that each NIAT Action will have a HQ Action Lead, all future NIAT related actions will be viewed as an Opportunity For Improvement (OFI). Appendix F contains the list of HQ Action Leads, prepared by Kitty Havens, Code UP. To date, we have received plans from 2 Action Leads— Jo Gunderson: Action 2, Development of the Workforce Wilson Harkins: Action 8, Safety and Mission Assurance 12

Other Formal HQ NIAT Actions: 

Other Formal HQ NIAT Actions March 13: HATS Action (1AE/2001-00019) - Code AE/Brian Keegan requested that the Project Managers for ELV and CLCS to address a specific set of questions regarding NIAT Actions 7, 8, 10, 11, 13, 14, & 15. Status: To be presented at the KPPMC, 11/13/01; ELV Response prepared by QA-D/Eric Ernst; CLCS Response prepared by Mike Bolger, Larry Wilhelm and Ric Hurt April 4: KAITS Action (AA/2001-00110.1) - Code F/Vicki Novak requested BA’s review and concurrence with their proposed NIAT Action 2 (Development of the Workforce) Plan. Status: BA has concurred is awaiting direction per the plan prepared by Jo Gunderson. April 5: QA Action (QA/2001-00025) - Code QS/Jim Lloyd requested QA’s review and concurrence with their proposed NIAT Action 8 (Safety and Mission Assurance) Plan. Status: QA/Ann Montgomery replied to Wilson Harkins with comments on April 18, 01 and is awaiting further action. April 16: via e-mail - Dr. Barry indicated that Dr. Mulville plans to roll-out the new “NASA Effective Stress Management” self-education/test which is pertinent to NIAT Action 1: Health and Safety. Status: Available at http://ohp.ksc.nasa.gov/disciplines/eap/survey 13

Potential Impact Statement: 

Potential Impact Statement   Several of the actions proposed in the formal Action Plans (i.e. Actions 2 & 8) represent a potential burden on existing resources. However, neither Action Lead has levied any new requirements as a result of their proposed plan. Until such time, the SMO will remain as the central interface for coordinating future NIAT Actions while allowing the affected organizations to determine what impact, if any, would occur by implementing each proposed plan. 14

Summary: 

Summary Headquarters direction was negligible and despite admiration for our methodical approach, no agreement was reached or further guidance provided on what they expected outside the Code AE action for CLCS and ELV Project. The Systems Management Office will remain as the central interface for coordinating future NIAT Actions. The Working Group will disband, allowing the responsibility for each future action to reside with the affected organization(s). 15

Backup slides: 

Backup slides Supporting data on Actions 1, 2, 7, 8, 16 Slides 17-27 Four Reports referenced in NIAT Report Slide 28 Code AE Action details Slides 29--34 16

NIAT Action 1: Health and Safety: 

NIAT Action 1: Health and Safety Chief Health and Medical Officer (CHMO), at NASA Headquarters, is the lead individual responsible for ensuring implementation of the employee health portion of the NIAT 1 Health and Safety Action Plan. The Manager, Agency Occupational Health Program, located at the Principal Center KSC, in the Office of Safety, Health, and Independent Assessments, is responsible for developing and coordinating of EAP policy implementation, procedures, and guidelines for approval along with providing EAP operations management and staff assistance. The CHMO urges that Center Directors strongly encourage supervisors to take the EAP training module with a goal of 75% completion rate by June 1, 2002. KSC intends to meet this goal by the projected date and has successfully met the OHP recommended guideline to have 1 EAP professional per 3,000 civil service employees. 17

NIAT Action 1: Health and Safety: 

NIAT Action 1: Health and Safety In addition to individual counseling, the EAP provides training on a number of subject matters such as stress management, sexual harassment, communication skills and team building to name a few. These services are performed as needs are identified in the workplace. The RehabWorks Program has been developed to provide all KSC and CCAS employees with a free rehabilitation for work, non-work or sports-related injury in order to maximize long-term recovery and reduce lost work time (by Certified/Licensed Athletic Trainers (ATC/L)). As supporting evidence, KSC continues to track payroll indicators by posting the Lost Time Injury metrics on the web to allow both NASA and contractors to analyze their individual trending data and determine ways to improve their efficacy while meeting workload demands. Supervisors must encourage and support employee participation in safety and health activities and include this theme in their individual performance plans, including KSC’s Goal Performance Evaluating System (GPES). Lastly, in a recent Center Director Communication letter (dated July 25, 2001) which addressed the Agency’s commitment to safety 18

NIAT Action 2: Development of the Workforce: 

NIAT Action 2: Development of the Workforce Headquarters Action Lead, Code FT/Jo Gunderson, submitted a formal Action Plan (KAITS Action: AA/2001-00110.1) to all the centers on April 5, requesting concurrence to their draft Implementation Plan on April 10, 2001. The implementation plan was reviewed by the Systems Management Office along with the BA/Workforce & Diversity Management Office. Our formal response to Code FT was that KSC supports the requirements outlined in the NIAT Action 2 Response Plan contingent upon the milestones being met by the originating organizations at NASA Headquarters. Awaiting further direction from Headquarters 19

NIAT Action 2: Development of the Workforce: 

NIAT Action 2: Development of the Workforce Until further notice from HQ is received, KSC has some on-going activities to highlight in Workforce Development. The Human Resources Development Board Systems Management was chartered to provide guidance and direction on training, rotational assignments, and and to oversee the Spaceport Technology And Research (STAR) career ladder program which was established as part of the overall KSC 2000 reorganization. The Goal Performance Evaluation System (GPES) allows the supervisor to become reacquainted with the employees’ accomplishments and promotes communication about their Individual Development Plan (IDP). SMO is working closely with BA to develop a KSC Project Management Development Process (PMDP) certification board. This information will be defined so that Project Managers can easily assess what they need to achieve to fulfill each level of the PMDP. 20

NIAT Action 7: Risk Identification, Assessment and Management : 

NIAT Action 7: Risk Identification, Assessment and Management KSC has implemented numerous Risk Management (RM) activities in recent years to improve the Centers’ Program/project lifecycles with SHIA serving as the focal point for: Establishing RM/S&MA policy and standards Consulting with respect to RM/S&MA planning, tools, and analyses Implementing integrated RM/S&MA tools and capabilities Accomplishing KSC Program/Project Management Council (KPPMC) assessments in support of the KSC Systems Management Office (SMO) The KSC SE&T S&MA Project Assessment Office, in conjunction with KSC SHIA, have developed RM and Project Plan templates that can be tailored for each projects’ specific needs. An assessment of the analysis tools used for KSC projects was conducted in July 2001. Their findings revealed strong usage of hazards analysis, FMEA/CIL, FTA, as well as a PRA (where project complexity warranted its use). KSC also facilitated separate Probabilistic Risk Assessment (PRA) and Saphire fundamentals workshops in the January/February 2001 timeframe, where a total of 45 personnel (32 PRA, 13 Saphire) were trained. The SE&T directorate has also been involved in developing ground processing-related PRAs for the Second Generation Reusable Launch Vehicle Program. 21

NIAT Action 7: Risk Identification, Assessment and Management : 

NIAT Action 7: Risk Identification, Assessment and Management In September 2000, SHIA provided Risk-Based Acquisition Management (R-BAM) training to 79 personnel (45 Procurement/Legal, 34 Engineering), which resulted in: Integrated, innovative teaming approach for employing R-BAM tool for acquisition risk assessment Production of a Top 10 Risk List, Acquisition Stoplight Charts, and Risk Wheels Formulation of generic R-BAM risk assessment templates for use at Acquisition Strategy Meeting (ASM) meetings In October 2000, the ELVIS PDT R-BAM results and products were successfully accepted at the NASA Headquarters ASM. As a result, several of the KSC R-BAM products have been adopted by GRC and used in subsequent Agency R-BAM activities. An assessment of the analysis tools used for KSC projects was conducted in July 2001. Their findings revealed strong usage of hazards analysis, FMEA/CIL, FTA, as well as a PRA (where project complexity warranted its use). As of July 2001, 127 employees have attended the R-BAM Overview training course and was utilized during the acquisition processes for major procurements in FY01. 22

NIAT Action 7: Risk Identification, Assessment and Management : 

NIAT Action 7: Risk Identification, Assessment and Management KSC Continuous Risk Management (CRM) program implementation activities were initiated in July 2000. KSC personnel completed a 2-day CRM Course at Goddard Space Flight Center (GSFC) to evaluate CRM training and lay the groundwork for further CRM training at KSC. As of August 2001, 10 KSC employees have become certified CRM Instructors which will serve to broaden CRM implementation across multiple KSC Programs. Those projects that participated in a CRM Risk Baselining Workshop were: Advanced Technology Development Center (ATDC) Optical Plume Anomaly Detector Launch Testbed Systems Second Generation RLV PRA KSC Project Resources On-Line KSC Complex Control System 23

NIAT Action 7: Risk Identification, Assessment and Management : 

NIAT Action 7: Risk Identification, Assessment and Management KSC has successfully implemented many Agency RM activities that enhance our ability to identify, assess, mitigate, and communicate risks as part of normal project and procurement team activities. The activities documented will continue to be performed as part of KSC’s ongoing RM activities. KSC will also has become involved in the Agency deployment of the Process-Based Mission Assurance (PBMA) methodology across all centers in support of program and project teams. SHIA is incorporating cutting edge training programs to foster KSC Program/Project Management capabilities in this critical area, including Earned Value Management. EVM is a risk management tool that integrates work scope, schedule, and cost objectives and employs techniques for program planning and control. EVM is a systematic approach that integrates various management subsystems. 24

NIAT Action 8: Safety and Mission Assurance : 

NIAT Action 8: Safety and Mission Assurance Similar to Action 2, Headquarters Lead, Code QS/Wilson Harkins, submitted a formal Action Plan (QA Action: QA/2001-00025) to all the centers on April 10, requesting concurrence to their draft Implementation Plan on April 16, 2001. KSC will also has become involved in the Agency deployment of the Process-Based Mission Assurance (PBMA) methodology across all centers in support of program and project teams. The Systems Management Office and the KSC Chief Engineer’s Office (CE) are co-located within the Safety, Health & Independent Assessment (SHIA) Directorate. The KSC SMO provides support, mentoring and independent assessments of KSC programs and projects for compliance with NPG 7120.5 NASA Program & Project Management Processes & Requirements, KSC internal processes (KDPs). 25

NIAT Action 16: Organizational Communication : 

NIAT Action 16: Organizational Communication The Center Director frequently sends memos (i.e. CD Comms) to keep all personnel informed of new initiatives as well as recent changes to center objectives or priorities. An archive of these letters reside at KSC’s Documentation Home Page (http://www.ksc.nasa.gov/docs/management) The Center has a Labor-Management Partnership Forum comprised of top managers and union officers to discuss issues, mutual concerns and potentially divisive issues which meets on a monthly basis. The development of a new database (CPI Tracker) is underway to replace the Opportunity For Improvement system by FY02, 3rd quarter. Analysis and reporting of CPI actions will be performed by the Safety, Health & Independent Assessment Office in coordination with appropriate Database owners and presented to management at the appropriate KSC councils (i.e., Executive Management Council & Safety and Health Council). The ultimate goal is to combine and/or electronically link all or most of these CPI related stand-alone applications to a centralized database or “hub.” 27

NIAT Action 16: Organizational Communication : 

NIAT Action 16: Organizational Communication KSC Daily (e-mail) News and the Goal Performance Evaluation System (GPES) are positive results of an employee suggestion. The Goal Performance Evaluation System (GPES) links employee performance plans to each Business Objectives and Agreement (BOA), a KSC developed performance/communication system. The purpose of BOAs was to enhance communications by reducing role confusion and clarify mission goals. In July 2001, a new position was formed in the XA/External Relations directorate, Public Affairs Specialist/Internal Communications. The incumbent is responsible for advising KSC personnel on effective communication tools along with determining how internal events are publicized. 28

Four Reports referred to in the NIAT: 

Four Reports referred to in the NIAT Four reports in response to failures in the Mars Program, Shuttle wiring problems and generic assessment of NASA’s approach to executing “Faster, Better, Cheaper.” Mars Climate Orbiter (MCO) Mishap Investigation, chaired by Arthur Stephenson, Director, Marshall Space Flight Center Mars Program Independent Assessment (MPIA), chaired by A. Thomas Young, Lockheed Martin (retired) NASA Faster, Better, Cheaper (FBC) Task, chaired by Anthony Spear, Jet Propulsion Laboratory Mars Pathfinder Project Manager (retired) Shuttle Independent Assessment (SIA), chaired by Dr. Henry McDonald, Director, Ames Research Center (ARC) 29

HQ/Code AE list of KSC Proposed Projects to Assess : 

HQ/Code AE list of KSC Proposed Projects to Assess As a result of the February 16 meeting between the Administrator and the Chief Engineer, the Systems Management Offices were tasked to address a specific list of projects to be assessed as defined by Code AE. KSC projects: Expendable Launch Vehicle (ELV) As discussed with the ELV Director, SMO will lead the assessment by obtaining information from existing Red Team Reviews to essentially answer the 7 actions. Checkout & Launch Control System (CLCS) Assessment report was prepared by Mike Bolger, Larry Wilhelm and Ric Hurt. Reporting/Approval hierarchy: GPMC and Field Center Director Enterprise (Code M) Code AE 30

Code AE Action: 

Code AE Action   March 13: HATS Action (1AE/2001-00019) - Code AE/Brian Keegan requested that the Project Managers for ELV and CLCS to address a specific set of questions regarding NIAT Actions 7, 8, 10, 11, 13, 14, & 15. 7: Risk Identification, Assessment and Management 8: Safety and Mission Assurance 10: Integrated Review Process 11: Ensuring Adequate Resources 13: Surveillance 14: Verification and Validation 15: Managing Responsibility and Accountability Ø     Status: - ELV Response prepared by QA-D/Eric Ernst - CLCS Response prepared by Mike Bolger, Larry Wilhelm and Ric Hurt 31

NIAT Actions to Focus on for Assessment: 

NIAT Actions to Focus on for Assessment NIAT 7: Risk Identification, Assessment and Management Is there a current clearly defined mission level Success Criteria that can guide risk decisions and is it flowed down appropriately to lower levels of the system? Are risks and their mitigation strategies, and acceptance rationale appropriately defined and updated as needed? Does the project comprehensively define and assess risks using, when appropriate, analytic tools such as Failure Modes Effects Analysis, Fault Tree analysis, and Probabilistic Risk Assessment? Is the project risk posture periodically reviewed for acceptability by the Governing Program Management Council? NIAT 8: Safety and Mission Assurance Is the role and level of involvement of the safety and mission assurance community compatible with the level of risk acceptable for the project? Is there adequate SMA involvement in the reporting, assessment, and closeout of problems and is their adequate awareness by all project personnel, of their role in problem reporting and closeout? 32

NIAT Actions to Focus on for Assessment: 

NIAT Actions to Focus on for Assessment NIAT 10: Integrated Review Process Is the approach to conducting an integrated set of programmatic and technical reviews compatible with the acceptable level of risk for the project, in particular as related to: Degree of independence of review team members Frequency of reviews Scope of review (I.e. appropriate combination of engineering peer reviews, system reviews, major milestone reviews) Reporting of review results and close out system for project responses NIAT 11: Ensuring Adequate Resources Are the allocated resources (budget, schedule, and staff) compatible with the current scope of the project and are the budgetary reserves, schedule slack, and design margins compatible with current unmitigated risks? If not, is action underway to achieve such compatibility, recognizing that additional budget is not the only alternative? Is that compatibility assessment periodically updated and reviewed by the Governing Program Management Council? Are the allocated budgetary reserves readily available to the project manager? 33

NIAT Actions to Focus on for Assessment: 

NIAT Actions to Focus on for Assessment NIAT 13: Surveillance Is the overall project approach to tracking and directing the efforts of all providers compatible with the level of acceptable risk for the project and is it tailored for the various project elements according to the degree of risk associated with that element? (Note that this question extends beyond the formal surveillance provided by the assurance function to also include activities of the project management staff, the engineering support staff, and the business support staff. The question further extends beyond only those projects and services provided by formal contract to include all projects and services no matter how provided.) NIAT 14: Verification and Validation Is the approach to verifying and validating the system compatible with the level of acceptable risk for the project? This assessment shall include the adequacy of development, acceptance, and qualification testing of individual hardware and software components as well as of the composite flight and ground elements and the fully integrated mission system. Adequacy of operational simulations should be considered. Is the approach to conducting Independent Verification and Validation of mission critical software compatible with the level of acceptable risk for the project and is it compatible with the current agency policy? 34

NIAT Actions to Focus on for Assessment: 

NIAT Actions to Focus on for Assessment NIAT 15: Managing Responsibility and Accountability Is the process for assessing and providing guidance and support to the project team understood by both senior and line management of all organizations associated with execution of the project? Is that process executed in a way that is compatible with the acceptable level of risk for the project? 35