Philip Tetlock of the University of Pennsylvania and author of Superforecasting talks with EconTalk host Russ Roberts about his work on assessing probabilities with teams of thoughtful a Show EconTalk, Ep Philip Tetlock on Superforecasting - Dec 20, 2015 Youre expected to doubt what you know, be curious about what you dont know, and update your views based on new data.. They give examples of successful and unsuccessful decision-making processes, none more diametrically opposed as two US Army missions. He has published over 200 articles in peer-reviewed journals and has edited or written ten books.[1]. As Prosecutor, we automatically attack any ideas that don't f the usefulness of hypothetical-society experiments in disentangling fact and value judgments of the impact of competing policy proposals. It's also the question that Philip Tetlock, a psychologist at the University of Pennsylvania and a co-author of "Superforecasting: The Art and Science of Prediction," has dedicated his career to answering. But when the iPhone was released, Lazaridis failed to change his thinking to respond to a rapidly changing mobile device market. The test group outperformed the control group significantly and tended to pivot twice as often. The truth remains that for all our social science, the world manages to surprise us far more often than not. Opening story: Smokejumpers and the Mann Gulch fire (Montana) of 1949. Chapter 11: Escaping Tunnel Vision. Professor Tetlock, who's based at the University of Pennsylvania, famously did a 20-year study of political predictions involving more than 280 experts, and found that on balance their rate of . Counterfactual thinking: considering alternative realities, imagining different circumstances and outcomes. The attack on Osama bin Ladens compound employed red teams and statistical risk assessments before the operation; whereas, the battle of the Bay of Pigs was undone by a failure to employ targeted questioning.5, When the scientist tells you he does not know the answer, he is an ignorant man. Alternative view: intelligence is the ability to rethink and unlearn, i.e. Competence and confidence dont progress at the same rate: Humility is often misunderstood. Something about the book felt superficialeach of the individual parts could have been a book unto itself. New York: Cambridge University Press, 2001. The first is the "Preacher". Wilbur Wright: Honest argument is merely a process of mutually picking the beams and motes out of each others eyes so both can see clearly.. When he is pretty sure of how it is going to work, and he tells you, This is the way its going to work, Ill bet, he still is in some doubt. From 1984 to 2004 Tetlock tracked political pundits' ability to predict world events, culminating in his 2006 book Expert Political Judgment. Great listeners are more interested in making their audiences feel smart., Part III: Collective Rethinking After publishing this study in 2005, he spent years attempting to uncover what sets these superforecasters apart.1Research into superforecasters was conducted by The Good Judgment Project, an initiative Tetlock founded with Barbara Mellers, a colleague from the University of Pennsylvania.2The research Tetlock and his team conducted demonstrated that the key attributes of a superforecaster are teamwork, thinking in terms of probabilities, drawing knowledge from a variety of sources, and willingness to own up to their mistakes and take a different approach.3, Forecasters who see illusory correlations and assume that moral and cognitive weakness run together will fail when we need them most., Philip Tetlock inSuperforecasting: The Art and Science of Prediction, Superforecasters have been shown to be so impressive in their ability to forecast future outcomes that they have outperformed highly trained intelligence analysts who have access to classified information that the superforecasters do not.4In their 2015 book,Superforecasting: The Art and Science of Prediction,Tetlock and his co-author Dan Gardner trace patterns in forecasting through history. Binary thinking results in fewer opportunities for finding common ground. Task conflict: Arguments over specific ideas and opinions (e.g. Forecast, measure, revise: it is the surest path to seeing better., Superforecasting: The Art and Science of Prediction. Presumes the world is divided into two sides: believers and non-believers. Grant argues these cognitive skills are essential in a turbulent and changing world. **Chapter 1: A Preacher, a Prosecutor, a Politician, and a Scientist ** , traces the evolution of this project. How Can we Know? [Adam Grant]: Two decades ago, I read a brilliant paper by Phil Tetlock, who introduced me to this idea of thinking like a preacher, a prosecutor or a politician. Skeptics are those who dont believe everything they hear. The Expert Political Judgements study was run over 20 years in which Tetlock asked a group of pundits to rate three possible outcomes for a political or . Rational models of human behavior aim to predict, possibly control, humans. You wouldn't use a hammer to try to cut down a tree, and try to use an axe to drive nails and you're likely to lose a finger. Second thoughts on expert political judgment. on government, politics, or international affairs and the Robert E. Lane Award for best book in political psychology. Good outcomes arent always the result of good decisions. The concept of superforecasters was developed by The Good Judgment Project and is arguably their best-known discovery. The stronger a persons belief, the more important the quality of the reasons or justifications. Thoughtful self-critical analysis? Tetlock, P. E. (1994). Conformity with group orthodoxy maintains cohesion. So too do different mental jobs. They argue that tournaments are ways of signaling that an organization is committed to playing a pure accuracy game and generating probability estimates that are as accurate as possible (and not tilting estimates to avoid the most recent "mistake"). document.getElementById( "ak_js_1" ).setAttribute( "value", ( new Date() ).getTime() ); document.getElementById( "ak_js_2" ).setAttribute( "value", ( new Date() ).getTime() ); Your email address will not be published. Tetlock, P.E., (2000). He evaluates predictions from experts in different fields, comparing them to predictions by well-informed laity or those based on simple extrapolation from current trends. Everyone carries cognitive tools that are regularly used and seldom questioned or subject to reflection or scrutiny. During a spacewalk, Luca felt water in his helmet. Tetlock P. and Mellers B. In theory, confidence and competence go hand in hand. Home; Uncategorized; philip tetlock preacher, prosecutor, politician Actively seek out reasons why you might be wrong. [29][30] In this view, political actorsbe they voters or national leadersare human beings whose behavior should be subject to fundamental psychological laws that cut across cultures and historical periods. [10][11], In a 1985 essay, Tetlock proposed that accountability is a key concept for linking the individual levels of analysis to the social-system levels of analysis. Tetlock also realized that certain people are able to make predictions far more accurately than the general population. The child is premature. Moore, D., Tetlock, P.E., Tanlu, L., &Bazerman, M. (2006). Pros: Important topic well worth pondering. philip tetlock preacher, prosecutor, politician2nd battalion, 4th field artillery regiment. [24][25][26][27] Rather, humans prefer to believe that they have sacred values that provide firm foundations for their moral-political opinions. Being aware of these can dramatically change the approach we take for ourselves and our audience. He found that overall, his study subjects weren't. Tetlock has received awards from scientific societies and foundations, including the American Psychological Association, American Political Science Association, American Association for the Advancement of Science, International Society of Political Psychology, American Academy of Arts and Sciences, the National Academy of Sciences and the MacArthur, Sage, Grawemeyer, and Carnegie Foundations. Opening story: Marie-Helene Etienne-Rousseau of Quebec gives birth to a child. The Adversarial Collaboration Project, run by Cory Clark and Philip Tetlock, helps scientists with competing perspectives design joint research that tests both arguments. Still, Tetlock has gone beyond journal articles, turning to a Tetlock has advanced variants of this argument in articles on the links between cognitive styles and ideology (the fine line between rigid and principled)[31][32] as well as on the challenges of assessing value-charged concepts like symbolic racism[33] and unconscious bias (is it possible to be a "Bayesian bigot"?). (2006). View being wrong as a good thing; an opportunity to learn something new. Essentially, there are three modes, according to Tetlock: Preacher: In Preacher mode, we hold a fundamentally inarguable idea that we will passionately express, protecting it with great devotion. Structuring accountability systems in organizations: Key trade-offs and critical unknowns. Politician mode seeks the approval of others and has little conviction for the truth. Grit is essential for motivation (passion and perseverance), but it can also blind us to rethinking. Tetlock was born in 1954 in Toronto, Canada and completed his undergraduate work at the University of British Columbia and doctoral work at Yale University, obtaining his PhD in 1979. Superforecasting is both a fascinating leap into the art of decision making as well as a manual for thinking clearly in an increasingly uncertain world. Skepticism is foundational to the scientific method, whereas denial is the a priori rejection of ideas without objective consideration.. Here, Philip E. Tetlock explores what constitutes good judgment in predicting future events, and looks at why experts are often wrong in their forecasts. Tetlock, P. E., Visser, P., Singh, R., Polifroni, M., Elson, B., Mazzocco, P., &Rescober, P. (2007). New York: Elsevier. Comparative politics is the study. Arkes, H., &Tetlock, P.E. How Can We Know? Alternatively, those wanting to get a good sense for the book without reading it cover to cover will profit from reading the introduction, Part 1, and the helpful appendix of practical takeaways titled Actions for Impact.. Philip E. Tetlock (born 1954) is a Canadian-American political science writer, and is currently the Annenberg University Professor at the University of Pennsylvania, where he is cross-appointed at the Wharton School and the School of Arts and Sciences. The team was inserted into challenging conditions and the fire quickly overtook them. The three modes (and a quick explanation of each) are: Preacher we hold a fundamentally inarguable idea that we will passionately express, protecting our ideals as sacred, Prosecutor we will pick apart the logic of the oppositions idea to prove our own point, marshaling the flaws in others, Politician we will sway a crowd or sway with a crowd to stay in a relative position of power, politicking for support. Full Text HTML Download PDF Article Metrics. A rivalry exists whenever we reserve special animosity for a group we see as competing with us for resources or threatening our identities.. Use a steel man (instead of straw man) and consider your opponents strongest argument. Philip Tetlock, Superforecasting: The Art and Science of Prediction 3 likes Like "Here's a very simple example," says Annie Duke, an elite professional poker player, winner of the World Series of Poker, and a former PhD-level student of psychology. What he found is that a person who is knowledgeable in a variety of areas is a better forecaster than a person who has an in-depth, but extremely narrow area of expertise. COLUMBUS, Ohio -- How do political experts react when their predictions -- about election results or the fate of countries or other important issues -- turn out to be completely wrong? Instead, we tend to double down and sink more resources into the plan.". Philip E. Tetlock on Forecasting and Foraging as Fox. ; Unmaking the West: What-if Scenarios that Rewrite World History; and Counterfactual Thought Experiments in World Politics. We can strategize from there and know which mode, preacher, prosecutor, or politician, to operate from and to. Enter your email below and join us. (2004). It is the product of particular ways of thinking, of gathering information, of updating beliefs. Rethinking is not only an individual skill, its also an organizational one. and comparison of domestic politics across countries. He also identified "overpredicting change, creating incoherent scenarios" and "overconfidence, the confirmation bias and base-rate neglect." 1996-2001 Harold Burtt Professor of Psychology and Political Science The Ohio State University. Released in 2015, it was aNew York TimesBestseller and brought this concept into the mainstream by making it accessible to behavioral economists and the general population alike. One of Philip Tetlocks big ideas* is that we are typically operating in one of three modes when expressing or receiving an idea. Whats the best way to find those out? But we rarely question or consider this knowledge which includes beliefs, assumptions, opinions, and prejudices. When promoting your idea, you were being a Preacher - arguing your point of view based on a set of prior beliefs.