Importance of Precision in Geopolitical Forecasting
Explore the value of precision in geopolitical forecasting as discussed by Jeffrey Friedman, Joshua Baker, and other experts. Delve into the challenges of providing numerical assessments of probability, the impact of uncertainty, and the communication issues related to using words versus numbers.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Comments on: The Value of Precision in Geopolitical Forecasting by Jeffrey Friedman, Joshua Baker, Barbara Mellers, Philip Tetlock and Richard Zeckhauser Justin Wolfers University of Michigan also Brookings, CEPR, CESifo, IZA, PIIE and NBER July 19, 2015; NBER Summer Institute Economics of National Security
Why not give numerical assessments of probability? 1. Difficulty of quantification But regular people can make fine-grained probability assessments 2. Uncertainty about these estimates is important But uncertainty about probabilities is decision-irrelevant 3. Suggest overconfidence But coarsening predictions does not solve overconfidence problem Justin Wolfers, Comments on Precision in Forecasting 2
The problem with words When President John F. Kennedy asked the Joint Chiefs of Staff to assess the feasibility of the Bay of Pigs invasion: This plan has a fair chance of success Competing interpretations: President Kennedy: This is an optimistic assessment General David Gray: We thought other people would think that a fair chance would mean not too good. Would numbers have yielded clearer communication? Justin Wolfers, Comments on Precision in Forecasting 3
Are Fine-Grained Probability Estimates So Hard? Justin Wolfers, Comments on Precision in Forecasting 5
Why not give numerical assessments of probability? 1. Difficulty of quantification But regular people can make fine-grained probability assessments 2. Uncertainty about these estimates is important But uncertainty about probabilities is decision-irrelevant 3. Suggest overconfidence But coarsening predictions does not solve overconfidence problem Justin Wolfers, Comments on Precision in Forecasting 6
How important is uncertainty about probabilities? An example: Is Osama living in Abbottabad? Estimates range from 35% to 95% Obama asked whether these judgments disguised uncertainty as opposed to actually providing you with more useful information. 2004 Intelligence Reform and Terrorism Prevention Act: Analysts must properly caveat and express uncertainties or confidence in analytic judgments. Justin Wolfers, Comments on Precision in Forecasting 7
Uncertainty about probabilities is irrelevant Is there a difference between: There s a 50% chance Osama is in Abbottabad ? ?????? =1 2????????+1 2????? There s either a 25% chance or a 75% chance Osama is in Abbottabad (and both are equally likely) ? ?????? =1 2.25????????+ .75?????+1 2????????+1 Expected utility is linear in probability Uncertainty about uncertainty is just uncertainty Uncertainty about probabilities is irrelevant 2.75????????+ .25????? =1 2????? Justin Wolfers, Comments on Precision in Forecasting 8
Why not give numerical assessments of probability? 1. Difficulty of quantification But regular people can make fine-grained probability assessments 2. Uncertainty about these estimates is important But uncertainty about probabilities is decision-irrelevant 3. Suggest overconfidence But coarsening predictions does not solve overconfidence problem Justin Wolfers, Comments on Precision in Forecasting 9
Probability weighting function 100% Underconfidence Probability weight 80% 60% 40% 20% Overconfidence 0% 0% 20% 40% 60% 80% 100% True probability Justin Wolfers, Comments on Precision in Forecasting 10
Optimal reporting: Inverse probability weights 100% Optimal reporting function Actual Probability 80% Correct Underconfidence 60% 40% Correct overconfidence 20% 0% 0% 20% 40% Analyst s belief 60% 80% 100% Justin Wolfers, Comments on Precision in Forecasting 11
How best to report probabilities? 100% Truth / Analyst s report Optimal reporting function 80% Report beliefs truthfully 60% Intelligence Directive: Coarsen beliefs 40% 20% 0% 0% 20% 40% Analyst s belief 60% 80% 100% Justin Wolfers, Comments on Precision in Forecasting 12
Why not give numerical assessments of probability? 1. Difficulty of quantification But regular people can make fine-grained probability assessments 2. Uncertainty about these estimates is important But uncertainty about probabilities is decision-irrelevant 3. Suggest overconfidence But coarsening predictions does not solve overconfidence problem Justin Wolfers, Comments on Precision in Forecasting 13
There is no need for empirical work Analyst says that an outcome will occur with probability ? ? ????? ?2 Actual outcome occurs with probability ?: ? ????? ????? = ? 1 ?2+ 1 ? 0 ?2 If expectations are biased: ? = ? + ? ? is the bias ? ????? ????? = ? 1 ? + ?2 Coarsened expectations ? = ? + = ? + ? + ? coarsening ? ????? ????? = ? 1 ? + ? + ?2 Percentage difference in accuracy ? ????? ????? ? ????? ????? = ? 1 ? ????? ????? = ? ?+?2 ? ????? ????? Justin Wolfers, Comments on Precision in Forecasting 14
Why not give numerical assessments of probability? 1. Difficulty of quantification But regular people can make fine-grained probability assessments 2. Uncertainty about these estimates is important But uncertainty about probabilities is decision-irrelevant 3. Suggest overconfidence But coarsening predictions does not solve overconfidence problem Justin Wolfers, Comments on Precision in Forecasting 15
