RR 07 Lazear Risk Analysis of Multi Spacecraft

COST RISK ANALYSIS of a MULTI-SPACECRAFT SATELLITE SYSTEMJUNE 13, 2002NAME OF PRESENTER:YVONNE LAZEAR: 

COST RISK ANALYSIS of a MULTI-SPACECRAFT SATELLITE SYSTEM JUNE 13, 2002 NAME OF PRESENTER: YVONNE LAZEAR

INTRODUCTION: 

INTRODUCTION Methodology Assumes Planning Phase for a Long-Term Spacecraft and Ground Engineering, Production, Replenishment, and Operations and Support Program Our Goal is to Explain a Cost Risk Analysis Methodology Used to Produce Realistic Estimates of Costs This Approach is to Fully Integrate the Risk Management Process with the Cost Risk Analysis in Order to Maintain Consistency in the Evaluations of Risk. This Consistency and Iteration of Cost Risk with Updated Evaluations can Also Direct Program Management to Focus on Specific Risk Areas This Approach May Also Improve Program Risk Management by Focusing on Cost Risk Drivers Identified in Cost Risk Analysis

HOW TO PERFORM COST RISK ANALYSIS: 

HOW TO PERFORM COST RISK ANALYSIS Cost Risk Analysis is Part of Both the Risk Management and Cost Analysis Process Identify, Assess, and Mitigate Risks Define, and Measure Uncertainties to Determine the Cost Variance Quantify the Potential Risk Cost Impacts Interpret and Analyze the Cost Data Resulting from Cost Risk Model

RISK MANAGEMENT PROCESS: 

RISK MANAGEMENT PROCESS The Scores Determined in the Risk Assessment Process are Transferred and Used in the Cost Risk Assessment Process

RISK ASSESSMENT SCORING RESULTS: 

RISK ASSESSMENT SCORING RESULTS The Risk Integrated Product Team (RIPT), Comprised of Contractor and Government Technical Experts, Identify and Analyze Program Area and Critical Technical Risks Each Risk is Ranked and Scored Based on Specific Criteria Developed by the RIPT, Then Each WBS Element is Scored Using a Cross-Reference Matrix

COST RISK ANALYSIS: 

COST RISK ANALYSIS Cost Assessment Uses Risk Scores From the Same Nine Risk Categories, Although Technology and Maturity Hardware and Software Scores are now Combined Using the Highest Score of the Two Risk Categories are Separated by Nonrecurring and Recurring as Follows: Hardware and Software Technology – Nonrecurring Hardware and Software Maturity – Nonrecurring Performance – Nonrecurring Producibility – Recurring (TFU and Operations and Support) Supportability – Recurring (TFU and Operations and Support) Cost – Nonrecurring and Recurring (TFU and Operations and Support) Schedule – Nonrecurring and Recurring (TFU and Operations and Support)

RISK REDUCTIONS FOR SEVEN RISK CATEGORIES OVER TIME, SPANNING SRR THROUGH CDR: 

RISK REDUCTIONS FOR SEVEN RISK CATEGORIES OVER TIME, SPANNING SRR THROUGH CDR The Effectiveness of Risk Mitigation Activities is to Reduce the Risk Level Over Time. As Risk Levels are Reduced, Risk Scores are Reduced and Updates to the Cost Risk Analysis are Required Maturity Performance Producibility Supportability Cost Schedule High Low Medium SRR SDR PDR CDR Risk Categories Evaluated Initial Scores Actual Projected Current Average Risk Low - Medium Initial Average Risk Medium PDR Average Risk Low

COST RISK ANALYSIS METHODOLOGY STEPS: 

COST RISK ANALYSIS METHODOLOGY STEPS Identify the Most Probable Life Cycle Cost (LCC) Point Estimate Identify the Most Probable Learning Curve Percentages for Spacecraft (or Bus), Payload, Level of Effort (LOE), and Orbital Insertion System (OIS) Elements Identify Probability Distribution (Triangular or Uniform) for Most Probable Costs and Learning Curve Uncertainties Majority of Distributions Will be Triangular. Uniform Distribution Will be Used for Very Low and Low Symmetric Cost Risk Factors Identify Levels of Uncertainty by Evaluating the Risk Scores for the Most Probable Costs. Pick Learning Curve Uncertainties Based on Historical Production Data, etc. Select the Cost Distribution Symmetry Based on Confidence in the Point Estimate: Skew Left, Symmetrical, or Skew Right

RISK ANALYSIS METHODOLOGY (CONTINUED): 

RISK ANALYSIS METHODOLOGY (CONTINUED) Establish the Range of the Distribution (Minimum/Optimistic and Maximum/Pessimistic) from the Look-up Table for most Probable Costs and Learning Curve Uncertainties Identify the Correlation Between WBS Elements for Costs Define Assumptions and Forecasts in Crystal Ball for Costs and Learning Curves Load the WBS Element Correlation Matrix Perform Monte Carlo Simulation in Crystal Ball (10,000 Iterations) Interpret and Publish Data

COST RISK ASSESSMENTS: 

COST RISK ASSESSMENTS Assessment Scores are Reviewed to Ensure Consistent Cost Evaluations From Both a Risk and Cost Management Perspective Averaged Risk Factors from Each Category Represent the Cost Risk

FINAL RISK LEVEL DETERMINED BY AVERAGE RISK FACTORS AND DISTRIBUTION SYMMETRY: 

FINAL RISK LEVEL DETERMINED BY AVERAGE RISK FACTORS AND DISTRIBUTION SYMMETRY A Distribution Symmetry Input, Based on the Confidence in the Point Estimate for Each Low-Level WBS Element is Also Required The Averaged Probability Risk Score Along With the Distribution Symmetry Factor are Mapped Into the “Cost Risk Factors” Table to Derive a Low and High Cost Risk Factor 2.50 0.90 1.30 0.70 1.10 0.50 VH 4.8 < Pr £ 5.0 1.90 0.90 1.20 0.80 1.10 .070 H 4.0 < Pr £ 4.8 1.60 0.90 1.15 0.85 1.10 0.80 MH 3.5 < Pr £ 4.0 1.45 0.95 1.10 0.90 1.05 0.85 M 2.5 < Pr £ 3.5 1.30 0.96 1.07 0.93 1.04 0.90 ML 2.0 < Pr £ 2.5 1.21 0.97 1.05 0.95 1.03 0.93 L 1.2 < Pr £ 2.0 1.12 0.98 1.03 0.97 1.02 0.96 VL 1.0 < Pr £ 1.2 High Low High Low High Low Skewed Right SR Symmetric Skewed Left SL Cost Risk Factors Risk Levels Average Probability Risk Factor Value (Pr) 2.50 0.90 1.30 0.70 1.10 0.50 VH 4.8 < Pr £ 5.0 1.90 0.90 1.20 0.80 1.10 .070 H 4.0 < Pr £ 4.8 1.60 0.90 1.15 0.85 1.10 0.80 MH 3.5 < Pr £ 4.0 1.45 0.95 1.10 0.90 1.05 0.85 M 2.5 < Pr £ 3.5 1.30 0.96 1.07 0.93 1.04 0.90 ML 2.0 < Pr £ 2.5 1.21 0.97 1.05 0.95 1.03 0.93 L 1.2 < Pr £ 2.0 1.12 0.98 1.03 0.97 1.02 0.96 VL 1.0 < Pr £ 1.2 High Low High Low High Low Skewed Right SR Symmetric Skewed Left SL Cost Risk Factors Risk Levels Average Probability Risk Factor Value (Pr)

DETAILED COST RISK DISTRIBUTIONS: 

DETAILED COST RISK DISTRIBUTIONS The Point Cost Estimate is Multiplied by the Low and High Factors Obtained From the Table to Calculate the End Points of a Distribution for Each WBS Element

RISK IDENTIFICATION RISK CROSS-REFERENCE MATRIX: 

RISK IDENTIFICATION RISK CROSS-REFERENCE MATRIX Risks are Mapped to the Work Breakdown Structure Elements They Affect A Particular Risk May Only Affect the NR or REC (or Both) Portion of the Cost WBS ID WBS Correlation to Top Risks SV IAT&C PL IAT&C PL Thermal PL LOS System SV and PL Processing PL Focal Plane Assembly Ground Software Ground IA&T Survivability System Certification System IA&T Number Name 70 113 24 75 60 9 222110 Focal Plane Electronics/Cable X NR 222120 Focal Plane Design Integ X NR 22221121 Pointing and Control Structure X NR 22221124 Pointing and Control Flex X NR 22221125 Pointing and Control Substrate X NR 22221126 Pointing and Control Design Integ X NR 22222111 Processor Structure X NR TFU 22222112 Processor Chips X NR TFU 22222115 Processor Substrate X NR TFU 22222116 Processor Flex X NR TFU 22222117 Processor Design Integ X NR 222310 Heat Pipe Assy X NR TFU 222320 Heat Pipe Electronics X NR TFU 222370 Transport Heat Pipe X NR TFU 222380 Transport Rod X NR TFU 2223A0 Transverse Assys X NR TFU 2223C0 Thermal Design Integ X NR 2223D0 Thermal I&T X NR TFU 222700 Payload Software X NR 23210 Structure Design and Analysis X NR TFU

WBS MAPPING AND CORRELATION COEFFICIENTS: 

WBS MAPPING AND CORRELATION COEFFICIENTS The Correlation Between WBS Elements is Based on a Dependency of the Statistical Parameters Associated With Each Element and Provides a More Realistic Calculation for the Variance WBS ID Correlation Correlation Risk # Risk Description Number WBS Description Top Risks Factor #9 PL Focal Plane Assy 222110 Focal Plane Electronics/Cable NR 0.2 222120 Focal Plane Design Integ NR 0.3 #75 PL LOS System 22221121 Pointing and Control Structure NR 0.1 22221125 Pointing and Control Substrate NR 0.2 22221126 Pointing and Control Design Integ NR 0.5 #60 SV and PL Processing 22222111 Processor Structure NR TFU 0.2 22222112 Processor Chips NR TFU 0.5 22222115 Processor Substrate NR TFU 0.3 22222116 Processor Flex NR TFU 0.3 22222117 Processor Design Integ NR 0.2 222700 Payload Software NR 0.5 23A00 Spacecraft Integ, Assy/Test/Ckout NR 0.2 #24 PL Thermal 222310 Heat Pipe Assy NR TFU 0.2 222320 Heat Pipe Electronics NR TFU 0.5 222330 Heat Pipe Distribution Box NR TFU 0.3 222370 Transport Heat Pipe NR TFU 0.2 222380 Transport Rod NR TFU 0.1 2223A0 Transverse Assys NR TFU 0.1 2223C0 Thermal Design Integ NR 0.4 2223D0 Thermal I&T NR TFU 0.5

DISTRIBUTION FUNCTIONS METHODOLOGY: 

DISTRIBUTION FUNCTIONS METHODOLOGY

DISTRIBUTION FUNCTIONS EXAMPLES: 

DISTRIBUTION FUNCTIONS EXAMPLES For the Very Low Risk Symmetric Distribution Function, the Minimum and Maximum Values Tend to be Very Close to the Point Estimate (PE) These Cost Estimates are Based on Catalog Prices or Vendor Quotes for Off-the-Shelf Items As Risks Increase, PEs Become Less Precise. Prices are Based on Estimates with Less Historical Background and More Reliance on Experience with Similar Systems Than on Actual Prices Skewed Left Triangular Distribution Functions Have Greater Confidence That the Ultimate Cost is at or Below the PE For Very Low Risk, the Minimum Very Close to the PE, and Represents an Off-the-Shelf Item With Significant Cost History As Risk Level Increases, the Potential for Greater Cost Reductions are Possible, Similar to Electronic Items Such as Computer Processors or Increasing Buys From Lower to Higher Quantities The Right Skewed Distributions Have a Greater Confidence That the Ultimate Cost Will be at or Higher Than the PE Developing New Technologies for Low Volume Components Might Cost Several Times More Than Expected With Little Pay Back in Low Volume Productions

RISK MODEL COST CALCULATION PROCESS: 

RISK MODEL COST CALCULATION PROCESS The Total Spacecraft’s Recurring Costs has Uncertainties not Only in the TFU Cost, but in the Learning Curves Associated With the Units Following the TFU The Risk Model Analyzes the TFU Cost Uncertainty, and Simultaneously Analyzes the Learning Curve Percentage The Risk Model Then Statistically adds the Nonrecurring, the Total Recurring Spacecraft, and Operation and Support Cost Uncertainties

TFU COST CALCULATION PROCESS: 

TFU COST CALCULATION PROCESS Costs Vary due to Monte Carlo simulations based on Point Estimate, Minimum and Maximum Cost Estimates. Learning Curves Vary due to Monte Carlo simulations based on Point , Minimum & Maximum Learning Curves

SAMPLE ASSUMPTIONS AND FORECASTS BOUND THE COST RISK: 

SAMPLE ASSUMPTIONS AND FORECASTS BOUND THE COST RISK Update the Probability Distributions and Ranges for the Assumptions ( Non-Summing WBS Cost Elements), and Forecasts (Summations of Assumptions) in Risk Model

SUMMARY LCC AT 50% CONFIDENCE LEVEL: 

SUMMARY LCC AT 50% CONFIDENCE LEVEL A Monte Carlo Simulation Run is Performed Generating Random Values for Each Assumption for the Costs and Learning Curves, Effectively Simulating a Single “What-if” Scenario for Each Assumption A Cumulative Distribution is Generated for Each Forecast Cell (the Total, Level One, and Level Two WBS Cost Elements) at the 20th, 50th, and 80th Percentiles

INTERPRETATION OF COST MODEL RESULTS: 

INTERPRETATION OF COST MODEL RESULTS Risk Model Results Show a $5,686.7M 50-Percentile Risk Cost for Contractor Total Costs The Deviation Between the Total Point Estimate and Total 50th Percentile Risk Cost is Approximate $219.4M or 4% Approximately 1.15% is Attributable to Recurring Learning Curve Variances, and 2.86% due to Probability Cost Risk Factors

STATISTICAL DESCRIPTION OF THE TOTAL COST DISTRIBUTION PROVIDED BY CRYSTAL BALL SOFTWARE: 

STATISTICAL DESCRIPTION OF THE TOTAL COST DISTRIBUTION PROVIDED BY CRYSTAL BALL SOFTWARE

CRYSTAL BALL SOFTWARE GRAPHIC DESCRIPTION OF THE TOTAL COST DISTRIBUTION: 

CRYSTAL BALL SOFTWARE GRAPHIC DESCRIPTION OF THE TOTAL COST DISTRIBUTION Percentile FY 02 Dollars Percentile FY 02 Dollars 0% $5,345,703,513 60% $5,724,405,313 10% $5,512,029,589 70% $5,763,867,456 20% $5,574,256,762 80% $5,809,176,425 30% $5,619,526,390 90% $5,864,661,580 40% $5,653,042,940 100% $6,047,505,132 50% $5,686,660,462

CONCLUSION: 

CONCLUSION A Cost Risk Analysis Methodology has Been Demonstrated in Detail, Giving Specific Sample Cost Risk Results for a Multi-Spacecraft Satellite System The Total Results at a Particular Percentile (50% in This Case) Subtracted from the Total Point Estimate Gives us the Total Cost Risk of the Program In a $5.7B Notional Program, Another $219.4M or 4% Should be Reserved for Risk at the Planning Phase Timeframe of the Program Approximately 1.15% of the 4% is Attributable to Recurring Learning Curve Variances, and 2.86% due to Probability Cost Risk Factors At all Stages in the Program, the Cost Team Provides Valuable Information to Program Managers and the Customer. Approach Fully Integrates the Risk Management Process with the Cost Risk Analysis and Maintains Consistency in the Evaluations of Risk Helping Direct Program Management to Focus on Specific Risk Areas. This Approach Also Focuses Program Risk Management on Cost Risk Drivers Identified in Cost Risk Analysis. The Complex Process of Risk Cost Analysis Itself Leads to a Realistic Cost Risk Product.