SCEA National Conference El Segundo, CA June 2004 : SCEA National Conference El Segundo, CA June 2004 Estimating Future Capability
Bruce Fad, Vice-President PRICE Systems, LLC
Agenda: Agenda Defense Transformation: What is it? What does it mean to Cost Estimators/Analysts?
The Affordability Management Case for Future Capability Estimating
Answering the Challenges to Starting the Estimating Process
Understanding the Importance of Time
Military Transformation: Military Transformation “The Department seeks to ensure that changes occur not only in the operating concepts we develop and the systems we acquire but also in our military culture and process that drive investment decisions.” Arthur Cebrowski, Director, Office of Force Transformation, Office of the Secretary of Defense
Scope of Military Transformation: Scope of Military Transformation Transforming How We Fight
Doctrine, organization, training, materiel, leadership, education, personnel, and facilities
Transforming How We Work With Others
Integration of military power with other elements of national power and with foreign partners; enhanced coordination with the interagency and across all levels of government (federal, state, and local)
Transforming How We Do Business
More adaptive business and planning practices that are more entrepreneurial, future-oriented, and capabilities based.
Transforming Business Processes: Transforming Business Processes “The Department is reducing acquisition cycle time and aligning acquisition with a new capabilities-based resource allocation program built around JOCs. Instead of building plans, operations, and doctrine around individual military systems as often happened in the past, the Department will explicitly link acquisition strategy to future joint concepts in order to provide the capabilities necessary to execute future operations.” Military Transformation: A Strategic Approach; OSD, Fall 2003
Understanding the Strategy: Understanding the Strategy Grounded in theory of limited resources to handle rapidly growing transaction rates of needed capabilities
Everyday Examples
Telephone call switching: In 1930s, Operators a limited resource, but call transaction rates were growing rapidly
Old thinking: control transactions to available resources
Transformation thinking: shift operator function to point of transaction origin
Health care today: limited MDs but growing medical transaction rates
Old thinking: wait for the doctor
Transformation thinking: shift more health care to patient
Military Transformation: Military Transformation Understand transaction (capability) rate requirements and evolve the doctrine and systems that satisfy them.
Implications to Estimators:
Must be able to estimate cost for performance/capability
Specific systems will be a result rather than an input
Estimating will be an integrated component of a collaborative system engineering process
Credibility is keyed to continuous knowledge creation and management
Agenda: Agenda Defense Transformation: What is it? What does it mean to Cost Estimators/Analysts?
The Affordability Management Case for Future Capability Estimating
Answering the Challenges to Starting the Estimating Process
Understanding the Importance of Time
Why is Future Capability Estimating Important?: Why is Future Capability Estimating Important? Aging equipment/support mechanisms
Increasingly costly to maintain and operate
Stagnant or declining performance
Eroding reliability increasing O&S costs
Permanent diminishing funding
Demand for good stewardship of government resources
The Death Spiral
What is the Objective?: What is the Objective? Develop Solutions (Systems) that provide needed capability
Ensure capability is met at a reasonable cost
Employ a development process that meets the 2 objectives above
Affordability Management
What Is Affordability?: What Is Affordability? An item is affordable if:
It is needed
It fits within a budget without displacing other items that are needed as much or more
There are no less expensive alternatives that meet the need
Affordability : Affordability
Affordability SOW : Affordability SOW “The Supplier shall be responsible for implementing the XXX Life Cycle Cost (LCC) Containment Strategy as defined and in support of program milestones specified elsewhere in this contract. The Supplier shall develop an overall plan consistent with the Containment Strategy to achieve program affordability. The Supplier shall incorporate the principles of Cost as an Independent Variable (CAIV) to balance cost, performance, and schedule. The Supplier shall develop Total Ownership Cost (TOC) estimates to track progress toward meeting containment requirements.”
Extract from 2003 Weapon System SOW
Cost and Affordability: Cost and Affordability “Fiscal constraint is a reality … Cost must be viewed as an independent variable … Components shall plan programs based on realistic projections of funding likely to be available … DoD Components shall identify the total costs of ownership, and … the major drivers of total ownership costs … the user shall treat cost as a military requirement … state the amount … willing to invest to obtain, operate, and support the needed capability over its expected life cycle. Acquisition managers shall establish aggressive but realistic objectives for all programs … working with the user to trade off performance and schedule, beginning early in the program (when the majority of costs are determined).”
Department of Defense Directive (DoDD) 5000.1, Paragraph 4.5.2
TOC is a Primary Affordability Metric: TOC is a Primary Affordability Metric Total Ownership Cost is:
A concept of the totality of costs associated with a weapon system
To be managed and, where possible, reduced by a reformed and improved acquisition process
TOC is an Overarching Umbrella of topics, like:
Reduction of TOC (R-TOC)
Cost As an Independent Variable (CAIV)
Target Costing (TC)
Design to Cost (DTC)
Activity-Based Costing/Management (ABC/M)
Earned Value Management (EVM)
and others
TOC = LCC (per DoD 5000.4M)
AM, TOC, and CAIV: AM, TOC, and CAIV
Affordable Ownership Execution Process – 6 Steps: Affordable Ownership Execution Process – 6 Steps Establish Affordability consciousness by defining TOC goals and developing an Affordability Management Plan and supporting methodology
Establish TOC Baselines, Thresholds and Objectives; Identifying TOC cost drivers; Identify and quantify Reduction initiatives
Establish meaningful Affordability metrics
Identify appropriate tools
Analyze and prioritize initiatives
Assess (track and measure results against plan) Affordability Plan implementation
Affordability Management Process: Affordability Management Process How do design, operation, and support decisions affect system total ownership cost? What are the program cost drivers and cost reduction opportunities? What are the procedures, processes, and tools for analyzing program cost factors? What are the cost baselines, reduction targets, and program success measures? What are the results, and lessons learned? What best practices emerged? Promote achievement
Agenda: Agenda Defense Transformation: What is it? What does it mean to Cost Estimators/Analysts?
The Affordability Management Case for Future Capability Estimating
Answering the Challenges to Starting the Estimating Process
Understanding the Importance of Time
Why Start in R&D? : Why Start in R&D? Cost Incurred
Estimating Future Capability: Estimating Future Capability What are some of the biggest problems associated with estimating the cost of future capability systems? i.e. systems to be built with technology that does not yet exist or that is extremely immature; systems that will be tested in 10+ years and operationally deployed in 15 to 20 years. 2 that dominate my experience are:
How do we start?
What do we use as reference?
Starting – The Important First Step: Starting – The Important First Step Overcoming Trepidation (Fear)
The person possessing the ability to provide needed answers is a technologist (known as the Source) who dislikes cost estimating
The Source’s prevailing attitude is that cost is a result, not a control variable (despite CAIV)
Building a trusting relationship with the Source is the best hope
How to build trust in the Source?
Gradually
By deflecting, “What do you need?” with,
“What do you have?”
Investing Psychology: Investing Psychology Behavioral Finance
Field pioneered by research of Daniel Kahneman (Princeton) and Amos Tversky (Stanford); late 70s
Problem framing (gain or loss) importance
Prospect Theory
Based upon subjective experience of feelings
Greater distress at prospect of a loss than joy of prospect of an equivalent gain
Leads to loss aversion behavior: taking risks (often foolish) to avoid losses.
Investing Psychology: Investing Psychology Anchoring is perilous
Giving more weight to recent events than historical perspective
Danger in extrapolating a short term phenomena that is at odds with long-term averages.
Herding: following everyone else even if an objective assessment says it is irrational; a.k.a. Greed
What’s the Response to this bad behavior?
Fundamentals: Balance sheet and income statement for investments
Balance: Select and adhere to allocation types (stocks, bonds, cash, etc.)
Estimating Psychology: Estimating Psychology Over-Under Theory
Excludes “Feel Good” estimating
Greater fear at prospect of an underestimate than for an equivalent overestimate
How many press articles or GAO reports deal with overestimates?
Leads to over-adjustment behavior: magnifying risks to temper expectations; anchoring and herding often set in.
What’s the Response to this bad behavior?
Fundamentals: Relevant, verified data as basis for estimating
Balance: Select and adhere to estimating methods (Performance cost models, BOM estimating, etc.)
Estimating Psychology: Estimating Psychology
Golden Rule
Do not modify your methods while estimating
…don’t “invent” new drivers
…don’t add new bias adjustments
…don’t ignore history
Supporting Evidence: Supporting Evidence Actual
Performance
Supporting Evidence: Supporting Evidence
Point of Reference: Point of Reference The most easily obtainable like item (system, product, function) is the best starting point of reference
Pride renders any selected reference unsuitable to the Source
Quickly engages the Source in costing
Follow the Fad Axiom
Most of us are better at being critics than we are at being authors
Starting to Forecast the Future: Starting to Forecast the Future Most methods rely on “hard” information
The future is soft, indiscernible, and full of alternatives
Quantitative methods, though preferred, often inhibit the initiation of future forecasting
Boxes in thinking when concepts are many and varied
Intimidates sources of information
A Qualitative approach often fits better in moving an organization to start dealing with the problem of costing the future
Once forecasting begins,
Alternatives are managed and systematically eliminated
Systems Engineering accelerates
Quantitative methods gradually usurp qualitative ones
The Desk Blotter as a tool for Future Forecasting: The Desk Blotter as a tool for Future Forecasting
Desk Blotter Model Ground Rules: Desk Blotter Model Ground Rules
Blotter – Rules – Tables - Forecast: Blotter – Rules – Tables - Forecast An advanced design requiring technology development
Transition to more Rigor: Transition to more Rigor
Agenda: Agenda Defense Transformation: What is it? What does it mean to Cost Estimators/Analysts?
The Affordability Management Case for Future Capability Estimating
Answering the Challenges to Starting the Estimating Process
Understanding the Importance of Time
Forecasting the Future: Forecasting the Future Use time as the primary determining factor
Relate time to:
Cost variables/complexity
Risk and technology
Program characteristics
Improvements in methods and processes-general
Performance
Output (computational power, recharge cycles, range, etc.)
Performance Efficiency
Energy conversion
Accuracy (drift rate, resolution)
Design Efficiency & Specialized design and manufacturing tools
Cost and performance is directly associated with time
Trends are remarkably stable (predictable) over time
20th Century Futurology that Flopped: 20th Century Futurology that Flopped It will be 1000 years before man is able to build a contraption that will fly. Wilbur Wright lamenting to brother Orville, 1902
Everything that can be invented has been invented.
Charles H. Druell, U.S. Commissioner of Patents, 1899
Computers in the future may…perhaps weigh only 1.5 tons.
Popular Mechanics, 1949
I think that there is a world market for maybe five computers.
Thomas Watson, Chairman of IBM,1943
Professor Goddard does not know the relation between action and reaction and the need to have something better than a vacuum against which to react. He seems to lack the basic knowledge ladled out daily in high schools.
New York Times,1921
New or Improved Technology? Know the Difference : New or Improved Technology? Know the Difference Source - Ships of the Line - Brian Lavery
Cost Life Cycle of a Technology: Cost Life Cycle of a Technology Initial part of the curve; high cost due to low producibility, small production runs and limited sources
Center portion of the curve; low cost due to mature manufacturing processes, high yields and multiple sources
Latter part of the curve; increase in cost due to outdated processes, low procurement quantities and limited sources
Depth of curve a function of market size and number of applications
Generations of Technology: Generations of Technology Three generations of battery technology depicted
Initial high peaks of each generation caused by subcontractor processes maturity costs, prime contractor design integration costs and low producibility
When obsolete for several generations, cost is higher than current technologies of much greater performance
Quantitative Application Example : Quantitative Application Example Program:
Future Digital Processing Capability
Task:
When will technology be available for multi-function hand-held battle management?
What will it cost and what will the size be?
Highlights:
Trend analysis of computing development over past 60 years
Demonstrates steady and predictable improvements in packaging
Demonstrates steady and predictable improvements in cost/performance/size metrics
Return on Investment:
Credible expectations established
Viable planning undertaken
Future Digital Processing Capability: Future Digital Processing Capability
Future Digital Processing Capability: Future Digital Processing Capability
Future Digital Processing Capability: Future Digital Processing Capability
Future Digital Processing Capability: Future Digital Processing Capability
Future Basis from Knowledge: Future Basis from Knowledge
Forecasting the Future from Knowledge: Forecasting the Future from Knowledge
The Value of Applying Knowledge: The Value of Applying Knowledge
Summary: Summary Transformation is driving to more functional or performance based estimating
Responsiveness dictates prior study and knowledge capture of performance cost drivers over time
Affordability Management rests on credible cost/performance trade-off analysis conducted in collaborative environments spoken in management vernacular
Slide50: FASTER DECISIONS. BETTER DECISIONS.
Additional Material: Additional Material
Context of Affordability: Context of Affordability Tangible Factors (e.g. investment, performance capability)
Intangible Factors (e.g. good will, threat deterrence)
Conflict of capability and affordability objectives
Limited flexibility in constraints to deal with conflict
Affordability Implementation Plan: Affordability Implementation Plan The document describing how the Affordability Program is defined, managed, and implemented.
Specifies the processes and procedures for performing ongoing affordability assessments to ensure the satisfaction of requirements within program target cost.
From Objective Setting to Target Establishment to Trade Studies to Monitoring.
TOC & DOD LCC: TOC & DOD LCC Dr, J. S. Gansler, USD(A&T) Memorandum of November 13, 1998 …. For consistency with past initiatives, Defense Systems TOC is defined as Life Cycle Cost (LCC). LCC (per DoD 5000.4M) includes not only acquisition program direct costs, but also the indirect costs attributable to the acquisition program (i.e., costs that would not occur if the program did not exist).…
TOC Elements: TOC Elements “The Supplier shall define, maintain, document and deliver TOC estimates on a quarterly basis. The estimates shall represent the sum of all direct and indirect costs associated with the research, development, procurement, construction, operation, logistical support and disposal, including the total supporting infrastructure that plans, manages, and executes that system program over its full life.”
Extract from 2003 Weapon System SOW
CAIV & TOC Elements: CAIV & TOC Elements “The Supplier shall develop and implement throughout the development process an acquisition strategy and philosophy, focused on cost/performance/schedule quality/prioritized-functionality trade-offs in relation to Total Ownership Cost (TOC) of the system. The Supplier CAIV approach shall include the customer in decision-making and requirements trades to meet available budget resources. Supplier shall not only make up-front tradeoffs to balance cost, schedule, and quality, but also make continuing tradeoffs to balance schedule, prioritize functionality, and quality throughout the development process.”
Extract from 2003 Weapon System SOW
CAIV & TOC Elements: CAIV & TOC Elements “The Supplier shall develop and implement throughout the development process an acquisition strategy and philosophy, focused on cost/performance/schedule quality/prioritized-functionality trade-offs in relation to Total Ownership Cost (TOC) of the system. The Supplier CAIV approach shall include the customer in decision-making and requirements trades to meet available budget resources. Supplier shall not only make up-front tradeoffs to balance cost, schedule, and quality, but also make continuing tradeoffs to balance schedule, prioritize functionality, and quality throughout the development process.”
A Continuous Development Process
CAIV & TOC Elements: CAIV & TOC Elements “The Supplier shall develop and implement throughout the development process an acquisition strategy and philosophy, focused on cost/performance/schedule quality/prioritized-functionality trade-offs in relation to Total Ownership Cost (TOC) of the system. The Supplier CAIV approach shall include the customer in decision-making and requirements trades to meet available budget resources. Supplier shall not only make up-front tradeoffs to balance cost, schedule, and quality, but also make continuing tradeoffs to balance schedule, prioritize functionality, and quality throughout the development process.”
A Balanced Acquisition Strategy
CAIV & TOC Elements: CAIV & TOC Elements “The Supplier shall develop and implement throughout the development process an acquisition strategy and philosophy, focused on cost/performance/schedule quality/prioritized-functionality trade-offs in relation to Total Ownership Cost (TOC) of the system. The Supplier CAIV approach shall include the customer in decision-making and requirements trades to meet available budget resources. Supplier shall not only make up-front tradeoffs to balance cost, schedule, and quality, but also make continuing tradeoffs to balance schedule, prioritize functionality, and quality throughout the development process.”
A Collaborative Action
Create TOC Consciousness: Create TOC Consciousness Establish and publicize top-level program goals for TOC savings/avoidance
Create guidelines and templates for TOC evaluation
Develop a mechanism for publicizing and promoting TOC achievements and total progress
Incentivize and recognize team
members for TOC
achievements
Capture and disseminate
TOC program lessons learned
R-TOC Process Establishing a Baseline & Goals: What are the cost baselines, reduction targets, and program success measures? R-TOC Process Establishing a Baseline & Goals
Establishing Goals: Establishing Goals Step 1 in establishing reduction targets is to set top-level program goals
Goals are the stimuli and incentives to drive to affordability
Goals should be challenging and realistic
Variable or uniform allocation over time and program elements
Goals: Percent reduction from a baseline value of the life cycle cost of all pertinent products
Targets: Stated in absolute terms (as dollars, wan, etc.) as either a unit cost or cost per function
Affordability Process Allocate TOC Baseline: What are the cost baselines, reduction targets, and program success measures? Affordability Process Allocate TOC Baseline To apportion top-level requirements to lower indenture levels and make baseline cost allocations to functional groups or IPTs
R-TOC Process Establish Reduction Targets: What are the cost baselines, reduction targets, and program success measures? R-TOC Process Establish Reduction Targets Decomposition of cost targets to product pieces often needed in order to relate them to meaningful design parameters – especially for acquisition costs
R-TOC Process Cost Driver Identification How is this done?: R-TOC Process Cost Driver Identification How is this done? What are the program cost drivers and cost reduction opportunities?
Cost Drivers: Cost Drivers Cost Drivers are:
Factors that have major impact on total ownership cost
In aggregate, influence most of the cost of an item
Identifiers of the greatest opportunities for cost reduction
Determined by investigating for root causes
Cost Drivers are not:
High-level, high-cost WBS items such as
maintenance, manpower, or training
Examples
A cost driver for manpower may be the
need to provide an operator
whenever an item is operating
A cost driver for maintenance may be
the need for unique support equipment
Identifying Cost Drivers: Identifying Cost Drivers Develop a cost summary
Identify the high-cost contributors
Determine "cause and effect" relationships
Identify the causes for high cost areas
Conduct sensitivity analysis to determine the effect of input factors on the analysis results
Identify high-risk areas
Cause and Effect Relationships : Cause and Effect Relationships What are the drivers of the cost contributors?
How do drivers influence cost?
Cause and Effect Relationships : Cause and Effect Relationships Cost Driver insight requires understanding of product type
Drivers will differ from a ship to a airplane to a software development, for example
Candidate drivers might be armament and level of manning for a ship, speed and range for an airplane, and number of components and processing function for software
Capturing Cost Driver effect on cost requires understanding of cost modeling
Regression Analysis captures response of cost to changing parameters, yielding CERs
Known as Parametric Estimating