Slide1: Managing Tough Decisions With Decision Analysis INFORMS Southern California Chapter Meeting California State University, Northridge Phil Beccue


What makes resource allocation decisions difficult?: What makes resource allocation decisions difficult?


Many companies use informal approaches when allocating resources which may not be effective: Many companies use informal approaches when allocating resources which may not be effective


Slide4: Decision Analysis (DA) addresses the challenges of real-world decisions Provides a method to decompose complex problems Broad set of alternatives Specific agreement (up front) on criteria Explicit accounting of uncertainty Offers a set of tools and processes to bring clarity to the best course of action Based on foundational axioms of utility theory Is a prescriptive approach to decision-making applied to important, real-world problems


The Decision Analysis process provides a guide to think systematically about R&D decisions: The Decision Analysis process provides a guide to think systematically about Randamp;D decisions Decision analysis is a rigorous, transparent, quantitative approach for balancing the difficult tradeoffs inherent in Randamp;D decisions. A formal process provides a common language for thinking and communicating about decisions within a multidisciplinary team. Structuring Modeling and Data Collection Evaluation Communication and Integration Action Decision Problem


A case example illustrates the application of DA to a tough decision for a drug company: A case example illustrates the application of DA to a tough decision for a drug company Phase 2 trials for Leapogen to address an unmet medical need (jumping ability) are nearly complete High efficacy shown in a few of the 17 major potential clinical settings Management had differing views on the best way to proceed to develop and commercialize Leapogen Senior management review meeting in 2 months We started by carefully crafting a decision statement to keep the team focused:


We took a comprehensive look at 8 strategies and included all significant issues: We took a comprehensive look at 8 strategies and included all significant issues 8 well-defined Strategies Approved Label Narrow Medium Broad Probability of Tech Success Probability of Reg Approval Launch Timing Price Randamp;D Costs Sandamp;M Costs COGS Patient Population by Indication Treated patients Patient growth rate Disease Severity Therapeutic Penetration Competition Market Share Marketing Focus


Slide8: Strategy 1 $128M Strategy 2 $140M Strategy 3 $ 82M Strategy 4 $ 40M Strategy 5 $120M Strategy 6 $ 103M Strategy 7 $ 25M Strategy 8 $180M A strategy table narrowed the feasible alternatives to 8 clearly defined development strategies Strategy Name Dev Cost


We carefully defined 3 label outcomes for each strategy by specifying the clinical setting included: We carefully defined 3 label outcomes for each strategy by specifying the clinical setting included KEY a – basketball b – soccer c – volleyball d – long jump e – high jump f – hurdles g – rugby h – ballet i – gymnastics j – figure skating k – …


The decision tree specified the scenarios to be analyzed for each strategy: The decision tree specified the scenarios to be analyzed for each strategy The model calculated NPV for over 4 million scenarios!


Slide11: An influence diagram defined the key input requirements to compute NPV Decision Uncertainty Value


We specified uncertain inputs using probability distributions: We specified uncertain inputs using probability distributions


Slide13: Technical success probabilities were assessed through structured conversations and compared to industry benchmark data .60 .91 .73 .46 New, Active Substances (NAS) CMR Int’l Data, 1997 Launch


The probabilities of technical and regulatory success varied by strategy: The probabilities of technical and regulatory success varied by strategy Strategy Probability Technical Success Probability of Regulatory Success Given Label


The top strategy (#4) provides over $150M additional value than the status quo (#8) strategy: The top strategy (#4) provides over $150M additional value than the status quo (#8) strategy 4 8 3 6 7 2 5 1 Size of bubble is proportional to expected peak sales.


The expected NPV for Strategy 4 is $10M, including all technical and commercial risks: The expected NPV for Strategy 4 is $10M, including all technical and commercial risks


Slide17: Assuming technical success, peak year revenue could vary from $100 to $250 M, with expected sales of $150M Peak Sales ($M) Probability Density 75 100 125 150 175 200 225 250 275 300 325 Expected Value


Slide18: Competition Peak Share Basketball Label Severity of Disease Long Jump Peak Share High Jump Peak Share Long Jump Severity of Disease Long Jump Peak Share Soccer Severity of Disease High Jump Peak Share Volleyball Peak Share Hurdles Severity of Disease Volleyball Peak Share Gymnastics Peak Share Ballet 0 50 100 150 200 250 300 350 400 Peak Sales ($M) The key drivers of risk for Strategy 4 are Competition, Peak Share for Basketball, and Label Assumes technical and regulatory success


Revenues for Strategy 4 are uncertain, and key contributors of value are clinical settings A, D, and E: Revenues for Strategy 4 are uncertain, and key contributors of value are clinical settings A, D, and E


Risk/return tradeoffs for each strategy were made explicit: Risk/return tradeoffs for each strategy were made explicit Strategy 8    Strategy 4    Strategy 7    0 50 100 150 200 250 300 350


Slide21: Strategy 3 will provide positive value if the chance of success exceeds 50% Nominal Probability = 25%


The optimal marketing focus depends on the outcome of future uncertainties: The optimal marketing focus depends on the outcome of future uncertainties Region 1 Region 2 Region 3 Region 4 Early Marketing Focus Region 1 Region 2 Region 3 Region 4 Late Narrow Competition Timing Region 1 Region 2 Region 3 Region 4 Early Region 1 Region 2 Region 3 Region 4 Late Medium Competition Timing Broad S1 Label S2 S3 S4 S5 S6 S8 Leapogen Strategy Bold line indicates highest NPV path


Key insights from the strategic analysis: Key insights from the strategic analysis Strategy 4 (a focused strategy) has best overall value Peak sales is $150 million Optimal marketing focus should be determined closer to launch High value indications (% of total value): Clinical setting A (62%) Clinical setting E (26%) Clinical setting D (12%) Chance of success = 30% Clinical setting G is not as critical as once thought


The initial DA had a significant impact at Amgen: The initial DA had a significant impact at Amgen Jumping ability is a viable indication for Leapogen Structuring the complex set of development options provided direction and a clear development plan Senior management agreed to follow the recommended strategy of a focused program The recommended strategy gave $150M additional value over the status quo strategy Key drivers were identified as needing further investigation; senior management asked to update the decision analysis in the future A few months later, the team asked our group to translate the decision model into a forecasting tool for ongoing use We have performed additional strategic projects for Leapogen based on the initial work


It is critical to keep management informed at each step of the process: It is critical to keep management informed at each step of the process Action !


The benefits of using decision analysis should be weighed against the costs: The benefits of using decision analysis should be weighed against the costs Documents the decision process Information collection is focused and efficient Helps resolve conflicts and debates Fewer 'surprises' because uncertainty is explicitly considered Avoids common pitfalls in analyzing complex situations solving the wrong problem analyzing what is known rather than what is important getting lost in the process More time required of decision-makers More time for analysis team Possible discomfort with new process Reveals logic of decision confidential information lack of knowledge embarrassing motivations Benefits Costs


Slide27: The significant problems we face cannot be solved at the same level of thinking we were at when we created them. - Albert Einstein