You have permission to edit this article.
Edit
Hansen: Superforecaster's success raises the question: Why are most of us so bad at predicting things?

Hansen: Superforecaster's success raises the question: Why are most of us so bad at predicting things?

  • Updated
  • 0
Only $5 for 5 months

KEARNEY, Neb. — Bill predicts we will see many sparrows.

It is just after dawn, and we are hiking a trail near the Platte River.

The path is canopied by giant ancient trees, thousands of them, an unending square footage of prime avian real estate. But the dense morning fog shrouds everything in mystery, which is one way to say that I can’t see the rising sun. I can’t see into the trees. And I sure as heck can’t see many sparrows.

“Listen,” Bill says. He stops, points to a low branch and then lifts his binoculars to his eyes. “Hear that? That’s a field sparrow.”

And then the blanket of fog starts to lift, and a vast world of sparrows is all around us, chirping in surround sound. American field sparrows. Harris’s sparrows. White-throated sparrows, which, Bill says, “have a beautiful singing voice.” And they do.

Bill Flack was right about the sparrows, and that isn’t surprising. It’s not surprising because Bill is a serious birder who often travels from his Kearney apartment to the far corners of Nebraska in search of sparrows and eastern phoebes and buff-breasted sandpipers.

And it’s not surprising because Bill is good at predicting things. Really good. So good that he’s been interviewed by the co-authors of “Freakonomics” and featured in a book written by a University of Pennsylvania professor who has studied the art of predictions — what he calls “forecasting” — for decades.

Bill Flack is in fact so good at predicting things that when he, on a whim, entered into a massive tournament sponsored by a U.S. intelligence agency, the following things happened:

1. He predicted the future results of super-complex foreign policy issues better than pretty much every foreign policy expert and intelligence analyst who participated.

2. He got a new title from the University of Pennsylvania professor. Now he’s not just Bill Flack, retired U.S. Department of Agriculture employee, or Bill Flack, 57-year-old amateur bird-watcher from Kearney.

Now he is Bill Flack, superforecaster.

“The whole (superforecasting) thing is a little strange,” Bill says after we finish bird-watching, drive back to Kearney and take seats at a neighborhood doughnut shop, where he orders two doughnuts. “I mean, I could maybe find Burkina Faso on a map, but I for sure didn’t know who was president.”

But Bill Flack isn’t a superforecaster because he knows the president of Burkina Faso, or the rules that govern Chilean elections, or the history of Japanese monetary policy.

He’s a superforecaster because some strange brew of genetics and learned behavior makes his brain work differently from the way the rest of ours do.

All of us make forecasts, both trivial and crucial, each and every day: Should I bring an umbrella? Should I buy this house? Is buying this engagement ring a good idea? Can I cross the street before that 18-wheeler hits me? Will we find weapons of mass destruction in Iraq?

Yet experts say the truth is that, for all sorts of reasons, we are pretty crummy at forecasts. In viewing how Bill and people like him get forecasting right, experts say we can learn something about why the rest of us predict the future so very badly.

Bill’s incredibly successful foray into forecasting started on a whim. He noticed the mention of a superforecasting tournament run by IARPA — the Intelligence Advanced Research Projects Activity, if you are scoring at home — an office within a U.S. intelligence agency. The yearlong competition would score individuals and teams based on how they predicted a series of complex geopolitical questions: What would happen with the Japanese stock market? Who would win the upcoming Chilean election? How long before the next war broke out in Africa?

Sure, Bill thought, when he looked at the tournament’s rules and the mystifying questions. I’m retired. Why not?

Because Bill didn’t know the president of Burkina Faso — because he didn’t have a particular opinion about any of these questions — he began to study. Hard.

This of course shows that Bill was willing to spend long hours researching, which is clearly one character trait of anyone good at forecasting. But it also shows another, more-hidden trait of good forecasting, which is this: Sometimes being an expert actually makes you a crummy predictor.

Because Bill didn’t know much, and was willing to admit it, he went into that question about the Chilean election with zero preformed assumptions about Chilean politics. Instead, he came into that question and every other question with a truly open mind.

As it turns out, far too often both foreign policy experts and us regular folk let our dogmatism — our unwillingness to change our minds based on new evidence — guide our thinking. And that adherence to dogmatism, to what we think we already know, is maybe the biggest reason we stink at predicting the future, according to Philip Tetlock, Penn professor and the author of several books on predictions.

“We do need people who can give you the detailed biographies of every member of the Chilean Congress,” Bill says. “But it could be that in the course of acquiring all this expertise you develop a model of how a country works, and then you tend not to consider things that go against your model.”

Bill also had some other things going for him in this tournament. He’s really smart. He understands numbers and probabilities, can feel the difference between a 50-50 chance and an 80-20 chance in a way that most of us can’t. And he’s a tad obsessive, too; the sort of man whose record collection is meticulously alphabetized.

Those are all important. But so is the style in which he attacked each problem.

Most people, when they are trying to predict the future, start from what they already know. They then search for information in a way that Tetlock describes as an “inside view.” For example: In trying to predict if and when an African dictator will lose his grip on power, most people will research the African country in question, and the dictator.

Bill doesn’t do it that way, because his brain works differently. Instead, he tends to start from what Tetlock calls “an outside view.” He researches other African countries and asks these questions: What has happened to the past 10 dictators in similar situations? Can he derive an educated guess based on the past history of all African dictators?

The combination of Bill’s skills — his tenacity, his intelligence and his style of problem-solving — worked incredibly well. For a year, Bill competed against hundreds of others. The retired USDA field assistant was pitted against dozens of political scientists and dozens of intelligence agents who have spent their entire careers being paid to answer these types of questions.

He finished in the Top 10.

During the second year of the contest, he worked in a team of other superforecasters. He and his team dominated again.

So at the end of year two, when he and various teams organized by Tetlock had proven all sorts of startling things about forecasting — about how the experts and the government do it wrong, and how outsiders sometimes do it right — was Bill Flack swimming in job offers from government agencies and massive corporations?

No, he wasn’t. He didn’t get a single offer, he says with a shrug. Neither did any of the other superforecasters he knows.

He went back to his life as a retiree and bird-watching enthusiast.

As he finished his second doughnut at the coffee shop, I asked Bill: Shouldn’t it scare us a little bit that a Kearney bird-watching enthusiast is better at this than some analysts who work at the CIA, FBI and NSA?

Bill grins. “I would say it should be an object of concern, yes,” he says.

And then I have another question. OK, smart guy, I say. You were right a bunch of the time. Is there anything that you have been just shockingly, terribly wrong about?

The superforecaster grins wider.

“Donald Trump,” he says. “I agreed he wouldn’t last. I was as wrong as everyone else.”

Contact the writer: matthew.hansen@owh.com 402-444-1064, twitter.com/redcloud_scribe

“The whole (superforecasting) thing is a little strange. I mean, I could maybe find Burkina Faso on a map, but I for sure didn’t know who was president.” — Bill Flack

Flack

Be the first to know

* I understand and agree that registration on or use of this site constitutes agreement to its user agreement and privacy policy.

Related to this story

Get up-to-the-minute news sent straight to your device.

Topics

all

Breaking News

Huskers Breaking News

News Alert