In his most recent EconTalk interview, Nassim Taleb gave a humorous anecdote about an economic panel he sat on in which a representative of the International Monetary Fund stood up and authoritatively made precise economic forecasts based on their data for 2011, 2012, and 2013. Nobody questioned him. Taleb became enraged, stood up, and advised the audience that they would be fools to listen to the man, pointing out that the IMF had been grossly mistaken in its forecasts for 2008, 2009, and 2010. Similarly most of the government experts in the Fed, Fannie and Freddie, predicted a healthy housing market through 2007 and 2008 right up until the collapse. Another anecdote in Taleb's economics book The Black Swan tells about the Department of Defense predicting the price in oil 50 years into the future. When the price unexpectedly soared in the 1970's, it became clear that their predictions were terribly off target. Rather than admit the inherent foolishness of the endeavor at that point, they simply revised their estimates to create new 50-year forecasts based upon the new numbers.

Experts know more than the average person, but they are also far more prone to confirmation bias and are very poor at delineating what they don't or cannot know.

A good illustration of the quant jock hubris is seen in a recent contribution over at RIFuture, in which "Frymaster" again and again touts the reliability of his own selected data and self-proclaimed expertise to commenter jgardner03 -

"The facts make opinions irrelevant."

"You seem to dabble in these things; I'm a full on wonk. You need more facts and fewer feelings in these discussions."

"That's what the FM Global data say. The "why's" of that question are as irrelevant as the data are indisputable."

"I have DATA THAT PROVE higher co-pays are counterproductive in every way."

Content of the argument aside, the inherent problems present here should be obvious to any detached observer. There is no place for reasonable discussion when one side claims to have a complete monopoly on facts, expertise, and understanding. And the potential for confirmation bias and overreach is enormous here. There is no acknowledgement of the limitations of his own data and error margin, and he then astoundingly purports to be able to solve every problem from healthcare to education if only he is given the requisite amount of top-down control.

This is not to say that we should ignore science, data, or experts - quite the contrary. But we should be wary of experts (especially the self-proclaiming kind) who fail to acknowledge the basic limitations of their own data and models, especially when insourmountably complex systems such as the health care (in totality), the stock market, housing, etc. are involved. The error rates in such systems becomes monstrous almost immediately out of the gate. Any one man who claims to be able to model and forecast such systems in totality should be met with the most severe skepticism, especially when they are incapable of acknowledging their own basic biases and sets of assumptions.

Every lawyer knows about examining "experts". Before you can begin to challenge their opinion, you have to challenge their assumptions. Are they including "facts" which are not included in the facts on which they opine?

"An expert is anyone from more than 40 miles away" A. Lincoln