Hail The Experts!

Nick Kristof is so down on experts he stops making sense:

Ever wonder how financial experts could lead the world over the economic cliff?

One explanation is that so-called experts turn out to be, in many situations, a stunningly poor source of expertise. There’s evidence that what matters in making a sound forecast or decision isn’t so much knowledge or experience as good judgment — or, to be more precise, the way a person’s mind works.

...

The expert on experts is Philip Tetlock, a professor at the University of California, Berkeley. His 2005 book, “Expert Political Judgment,” is based on two decades of tracking some 82,000 predictions by 284 experts. The experts’ forecasts were tracked both on the subjects of their specialties and on subjects that they knew little about.

The result? The predictions of experts were, on average, only a tiny bit better than random guesses — the equivalent of a chimpanzee throwing darts at a board.

“It made virtually no difference whether participants had doctorates, whether they were economists, political scientists, journalists or historians, whether they had policy experience or access to classified information, or whether they had logged many or few years of experience,” Mr. Tetlock wrote.

Efficient market students are not exactly agog at this.  Nor is anyone who studies the sports pages - many newspapers have their panel of expert reporters make weekly football picks against the spread.  The general result supports the hypothesis that these guys, who follow football professionally and spend time in the locker rooms of the local teams, need to keep their day jobs.

That said, I question this conclusion.  Back in 2006 political "experts" could have delivered a short list of viable Presidential contenders in the two parties.  I daresay almost all such lists would have included John McCain and many (most?) would have given a nod to Barack Obama.  But would a panel of chimps throwing darts at, e.g., a list of 100 Senators, 435 Congressman, and 50 governors really have consistently hit those two?

I am sure Tetlock has an interesting point.  I just wonder what it is.

Kristof closes on a personal note:

The marketplace of ideas for now doesn’t clear out bad pundits and bad ideas partly because there’s no accountability. We trumpet our successes and ignore failures — or else attempt to explain that the failure doesn’t count because the situation changed or that we were basically right but the timing was off.

For example, I boast about having warned in 2002 and 2003 that Iraq would be a violent mess after we invaded. But I tend to make excuses for my own incorrect forecast in early 2007 that the troop “surge” would fail.

Well, well.  I have seen Kristof's fellow savant Paul Krugman put himself in that lonely camp of heroes who foresaw the housing collapse.  With that as an excuse, let me repeat Krugman's table-pounding jeremiad from May 2006:

Ben Bernanke, the chairman of the Federal Reserve, contends that what's happening in the housing market is "a very orderly and moderate kind of cooling." Maybe he's right. But if he isn't, the stock market drop of the last two days will be remembered as the start of a serious economic slowdown.

Maybe he's right!  Thanks for that.  And on the famous economist's famous other hand if he's wrong, it could be "serious".  Geez, can we get a belated bid for "apocalyptic"? 

Yeah, Krugman sure called the heck out of it.

PILING ON:  In January 2008 Krugman has a helpful blog post explaining why the Fed may have limited ability to prop up the housing market and fretting about a possible bust.  His table-pounding finish:

I’m actually not sure how bad things will get — remember, we still have help from booming exports. But it’s not too hard to tell stories in which monetary policy doesn’t have enough mojo to deal with our current problems.

He wasn't sure!  But he's sure now that he was sure then.

CHEAP SHOTS:  Here is another lesser-known (and deservedy so) ex post genius.

Comments

[bio mom]

So then does Kristof not buy into the collective wisdom of the globing warning "experts"?

[Charlie (Colorado)]

Or the political experts who were saying the Obama is underqualified?

[matt]

isn't Kristoff one of those same "experts" he derides?

[Ignatz Ratzkywatzky]

Am I the only one with the experience of placing my most supremely brilliant points on old threads five minutes after TM starts a new one?

[hit and run]

Kristoff:
The result? The predictions of experts were, on average, only a tiny bit better than random guesses — the equivalent of a chimpanzee throwing darts at a board.

Holy carp that is racist.

[f1guyus]

Maybe being an honest person has something to do with outcomes as well.

[hit and run]

Kristoff:
There’s evidence that what matters in making a sound forecast or decision isn’t so much knowledge or experience as good judgment

Holy carp he's still fighting the Democratic primary.

[Pat Curley]

The problem with analyzing supposed "political experts" is that of course each of them comes to the table with their personal bias. If you ask 100 Republican-leaning political experts who's going to win the election in 2012, you'll probably get 90 of them picking the GOP, and you'd get probably the same number of Democrat-leaning experts picking the Dems. Half will be right and half will be wrong.

Kristof even acknowledges this towards the end when he notes that he correctly predicted postwar violence in Iraq, but was wrong about the surge. He approached it from the standpoint of his liberal bias.

[rse]

Isn't this article's point (though poorly supported) that presumed expertise is overrated consistent with the current movement to limit private compensation above $250,000?

Is this an indirect way of asserting that Desantis and others aren't in fact worth what the market wants to pay them so the government is justified in its attempts to restrict, tax, or seek forfeiture?

[Charlie (Colorado)]

Am I the only one with the experience of placing my most supremely brilliant points on old threads five minutes after TM starts a new one?

Yes. In another six months or so we'l let you in on the mailing list.

[Charlie (Colorado)]

Is this an indirect way of asserting that Desantis and others aren't in fact worth what the market wants to pay them so the government is justified in its attempts to restrict, tax, or seek forfeiture?

I suspect that's giving Kristof more credit for coherent thought and long-term view than he deserves.

[Carl Pham]

I think the conclusions by Kristof and Tetlock are a bit logically or perhaps semantically sloppy.

It's not that experts are no good at prediction. An "expert" is good at prediction by definition. He's the one who turned out right, who was left standing by events. An expert miner knows how to get the coal out safely, and when he says: we need another prop here or the roof will cave in, he's right, every time. An expert banker knows how to lend money at a profit, and when he says I don't think you're good for a $50,000 loan, Mrs. Pooh-bah, but I'll OK $2,000 he's right, every time.

The problem is that in our post-modern age we have redefined "expert" from "he whose decisions and predictions are proved correct, over and over again" to "he who has a degree in this field, has studied it theoretically for umpty years, who writes long erudite opinions on the subject for many publications."

It doesn't surprise me in the least that "experts" defined in the latter academical theoretical way turn out not to generally be experts defined in the former, practical way.

What we need here is to return to the better definition of "expert." That way -- just to take a random example -- a man with zero experience running any profitable enterprise whatsoever, but who gives great persuasive speeches on how to theoretically do it ("Our economy only works if we recognize that we're all in this together..."), would be titled, despite his wordy brilliance, a complete newbie. And maybe we wouldn't be so quick to hand him complex problems to solve. Just a thought.

[Captain Hate]

When did Kristof ever make sense?

[Daddy]

Plato and the Delphic Oracle explained all this to us 2,500 years ago. (And I might ad, much better than Nick Kristof).

"Socrates is the wisest man in the world", said the Oracle.

"How the heck can that be?" wondered Socrates, knowing he wasn't an expert at anything. So off he went around the Greek world to carefully question all the experts about their expertise, and in the process wonderfully using his Socratic method of questioning to reveal that all the supposed experts were anything but in their chosen fields of expertise.

Thus concluded Socrates, "The Oracle was correct. I am the wisest man in the world because I am the only man in the world who is truly aware of how little he knows."

Couldn't our non-Constitutional Solon's use a little bit of that mental humility nowadays.

[sbw]

"Ce n'est pas une pipe."

[MayBee]

I only wish he would have had this "come-to-Jesus" moment at some point before he decided Joe Wilson was an expert on Iraq.

[matt]

ce n'est pas une TARP

[bad]

Mark Steyn is on Glen Beck.

[clarice]

I'm so depressed fer chrissakes--I am actually stenciling cut things on canvas bags which I plan to use when I bring homemade bread and wine to hosts and hostesses.I suppose it's better than randomly shooting people, but not much.

[Charlie (Colorado)]

"Ceçi".

Helpfully,

Pedants 'Я' Us.

[clarice]

**cutE thing**

(Like no left turn symbols and cats and girls at computer keyboards..Oh just shoot me and put me out of my misery!!)

[bad]

Turn on FOX Clarice. Mark Steyn just said "a guy with a banana in Victoria Secret panties."

He was talking about Abu Graib.

[matt]

merci, charlie!

[Barry Dauphin]

... and then the magic Kristoff read his own column and decided to stop writing columns. The End. Except that since Kristoff the writer was an expert, Kristoff the reader decided that maybe he shouldn't listen to him and so he decided to keep writing columns after all. And he lived happily ever after with Krugman.

[actor212]

Wow, I must have really gotten under your skin to rate a mention in your pathetic blog!

First you come over to my blog, and leave a post of such magnificent ignorance...."magnificence" in this instance being more along the lines of "Oh look at the turd the constipated mongrel finally squeezed out of his hemorrhoidal anus!"...that, frankly, my breath was taken, then you came back for more whipping with a cat-o-barbed-wire, and now you carry the fight over here to your own place, reinforcing the obvious point that your monumental stupidity could only possibly be exceeded by your overinflated and undeserved sense of self-worth!

I hope you aren't managing anyone's money!

Still...It's nice to know I still got it....thanks. I was worried I was becoming a nice guy.

By the way..."Just One Minute"?

Is that just the foreplay or is it all-inclusive? :lol:

Piece of advice: never get into a fight with someone who has no skin in the game when you clearly have a whole carcass riding on the outcome.

[Patrick R. Sullivan]

Kristoff was soooooooo expert on Dick Cheney sending Joe Wilson to Africa.

[MarkJ]

"When did Kristof ever make sense?"

Well, I'm told Nicky has a 100% rate of accuracy in his prognostications....

....on Neptune.

[Ignatz Ratzkywatzky]

First you come over to my blog, and leave a post of such magnificent ignorance...."magnificence" in this instance being more along the lines of "Oh look at the turd the constipated mongrel finally squeezed out of his hemorrhoidal anus!"...that, frankly, my breath was taken, then you came back for more whipping with a cat-o-barbed-wire, and now you carry the fight over here to your own place, reinforcing the obvious point that your monumental stupidity could only possibly be exceeded by your overinflated and undeserved sense of self-worth!

Deservedly so, indeed.
Jughead hath a blog.

[Captain Hate]

Still...It's nice to know I still got it.

An STD? Single-digit IQ? The conspicuous lack of self-awareness to be the only person to lol at your "joke"?

[patch]

The reason the experts are wrong so often is that they don't have skin in the game.

I'll go with sports bettors or the folks at Intrade over most writers, reporters, and experts.

[Patrick R. Sullivan]

actor212, shouldn't you get back to preparing your company's monthly sales tax report?

Btw, how much of a killing did you make shorting WAMU and Countrywide Financial?

[Charlie (Colorado)]

Piece of advice: never get into a fight with someone who has no skin in the game when you clearly have a whole carcass riding on the outcome.

So what you're saying is it makes no sense to make fun of you, because you don't care whether what you say is right or not?

[Charlie (Colorado)]

merci, charlie!

Actually, I was mainly showing off that I could get a backwards 'R'.

[clarice]

Well, never let it be said that I was utterly unproductive today when all this idiocy is swirling about me--ditched a bunch of stuff from my closets and --ta da--decorated 12 stupid canvas carryalls..
URGH

[hit and run]

Charlie:
I was mainly showing off that I could get a backwards 'R'

Actually, that could come in handy. http://hotlineoncall.nationaljournal.com/archives/2009/03/about_that_omni.php>For example...

In the voting so far this week, Collins (Я, ME), Snowe (Я, ME) and Specter (Я, PA) have again proved among the least reliable votes for their caucus.

...left-leaning Republicans.

[Jim Ryan]

Is that a Russian 'ya'? Inquiring minds want to know.

[ras]

Knowledge in a single area comes with a diminishing return; it's not linear. But the confidence/hubris that it inspires is, or much more nearly so.

Know a few things about the ponies and when you go to place a bet, you'll be humble and constrain your wager. Know a hundred things and your odds of being right are still about the same but your overconfidence will cause you to bet too much, too often, because you have so many more plausible rationales and data points to support your conclusion.

Beginner's luck might actually be better labeled Beginner's Humility

[Jane]

Who's the fool?

[Jim Miller]

There's actually some good advice in that column (though I don't think Kristof would apply it the same way I do).

Untrustworthy experts (or, if you prefer, "experts") tend to be famous. And they are famous because they are loud, and journalists prefer the loud to those with good records of prediction.

Now, does that famous-loud combination bring any economist to mind? OK, probably more than one, but I think one leads all the rest, and Kristof has every reason to be familiar with that economist's work.

[Charlie (Colorado)]

Collins (Я, ME), Snowe (Я, ME) and Specter (Я, PA)

I like it.

[Charlie (Colorado)]

The hell of it with Krugman is he really did some good work once upon a time, as well as some fun speculation about, eg, the economics of interstellar travel.

He seems to have forgotten how, though. In fact, he seems to have some issues with mere arithmetic.

It's enough to make one think dark thoughts about amyloid plaques.

[Jim Miller]

Charlie (Colorado)- You may prefer my theory about Krugman's columns: The columns are being written by a grad student who has been assigned to the task, and hates it. So the grad student is sabotaging Krugman by putting out badly written columns, columns that even contain obvious methodological errors, such as the ecological fallacy.

When I first made that suggestion, I said, very plainly, that I was joking. And I have repeated that since. But my joke theory has begun to make enough sense so that I have thought, off and on, of checking out, just for the heck of it.

(Since it is a joke, I haven't tried to work out all the details. I suppose that you would have to assume that the grad student runs the blog and that someone impersonates Krugman in some public appearances.)

[Jim Miller]

For "checking out", substitute "checking it out"

[bad]

Thanks for that correction, Jim. We don't need you contemplating checking out.

[mel]

I still prefer this (LUN)

[mel]

Oh, And another expert:

Teresa Ghilarducci, chairwoman of economic policy analysis at the New School for Social Research, is the author of “When I’m 64: The Plot Against Pensions and the Plan to Save Them.”

The 401(k) is a failed experiment of how well individuals can save for their retirement in commercial individual accounts. Instead, we need a supplement to Social Security that competes with the 401(k) — what I like to call a “guaranteed retirement account” plan to which all workers and employers would contribute 2.5 percent.

What we need is a guaranteed retirement account to which all workers and employers contribute 2.5 percent.
The contributions would be offset by a $600 refundable tax credit provided to all regardless of income. These contributions would create credits toward a lifetime pension based on a guaranteed real 3 percent annual rate of return. Congress would distribute the surplus — above a balancing fund maintained to ride out periods of low returns — to all accounts if actual returns are higher than 3 percent inflation.

Under these accounts, retirees do not outlive their savings and inflation does not erode buying power. The plan could provide for a partial lump sum of 10 percent of the account balance or $10,000, whichever is higher. Through Social Security and this guaranteed retirement account, a full-time worker making $40,000 per year who contributes into the plan for 40 years would receive roughly 71 percent of their pre-retirement income."

It's a bill in the House already.

[mel]

From the NYT, natch.

[MayBee]

There is nothing I trust more than Congress protecting a giant vat of funds.

[mel]

Like banks, and retirement accounts.

[MayBee]

And the Social Security Trust Fund. Which I understand is zipping right along.

[bad]

Why would anyone want to send their own money to DC?

Only the insane...

[bad]

Al Gore may be a nutcase but I always liked the idea of a lockbox.

[mel]

Yeah, but he had the only key, because he was the only one who could "see" it.

G'Night, and you get some rest too. OK?

[hit and run]

Bad:
Why would anyone want to send their own money to DC?

The address for making an investment in Washington Sharpened Pikes is a P.O. Box in DC.

[clarice]

WSP--Quality is Our Middle Name. Well maybe our last name, depends if your Spanish or something,

Ms. DeFarge, Proprietress.

[Chris]

I read that blog from the 212 genius and left a comment. He was haranguing Desantis as a "commodities trader" at AIG. Left a comment with the question being, how could a commodities guy be to blame for the CDS mess? I don't know what Desantis did or didn't do there. But if 212 was correct that he is/was in commodities, how could he have been part of the problem?

[sbw]

Hey, Charlie, I bow to your greater pedantry [ ;-) ] and your knowledge of what Magritte actually painted, but, since my last French class was 45 years ago, what is the difference between saying "Ce" and "Ceci" in his "... n'est pas une pipe" sentence?