Performance Charlatans – Reflections on Mark Thoma’s “The Strong Demand for Charlatans”

Prof. Mark Thoma posts an extract about The Strong Demand for Charlatans from The strong demand for charlatans, by Chris Dillow, in which Dillow says that:

In the improbable event of ever being invited to give a commencement address, my advice to graduates wanting a lucrative career would be: become a charlatan. There has always been a strong demand for witchdoctors, seers, quacks, pundits, mediums, tipsters and forecasters.

Dillow outlines an experiment in which coins are tossed and there are envelopes containing predictions for the outcomes. Participants could pay to see the prediction before the toss, or see the prediction afterwards for free.

And here’s the thing. Subjects who saw just two correct predictions were 15 percentage points more likely to buy a prediction for the third toss than subjects who got a right and wrong prediction in the earlier rounds. Subjects who saw four successive correct tips were 28 percentage points more likely to buy the prediction for the fifth round.

This tells us that even intelligent and numerate people are quick to misperceive randomness and to pay for an expertise that doesn’t exist; the subjects included students of sciences, engineering and accounting.

Thoma points to Daniel Kahneman for an explanation. The interview appears to based on Kahneman’s book, Thinking, Fast and Slow which I am currently reading. Kahneman explains his theory of dividing human gonition into two (2) systems:

Kahneman: …Psychologists distinguish between a “System 1” and a “System 2,” which control our actions. System 1 represents what we may call intuition. It tirelessly provides us with quick impressions, intentions and feelings. System 2, on the other hand, represents reason, self-control and intelligence. …

System 2 is the one who believes that it’s making the decisions. But in reality, most of the time, System 1 is acting on its own, without your being aware of it. It’s System 1 that decides whether you like a person, which thoughts or associations come to mind, and what you feel about something. All of this happens automatically. You can’t help it, and yet you often base your decisions on it. …

System 1 can never be switched off. You can’t stop it from doing its thing. System 2, on the other hand, is lazy and only becomes active when necessary. Slow, deliberate thinking is hard work. It consumes chemical resources in the brain, and people usually don’t like that. …

The excerpt from the interview concludes with Kahneman’s conclusions about expert intuition:

Spiegel: Do we generally put too much faith in experts?

Kahneman: I’m not claiming that the predictions of experts are fundamentally worthless. … Take doctors. They’re often excellent when it comes to short-term predictions. But they’re often quite poor in predicting how a patient will be doing in five or 10 years. And they don’t know the difference. That’s the key.

Spiegel: How can you tell whether a prediction is any good?

Kahneman: In the first place, be suspicious if a prediction is presented with great confidence. That says nothing about its accuracy. You should ask whether the environment is sufficiently regular and predictable, and whether the individual has had enough experience to learn this environment.

Emphasis Mine

Thoma’s concluding comment is:

One implication of this is that we use rules of thumb rather than rational, deliberative thought (i.e. rational expectations) for many of our decisions.

I cannot recall how many times I have been called upon to make snap performance judgements about critical systems. SQL Profiles in Oracle 10G have been a great help in this regard. They are simple to apply and fix a great deal of performance issues relatively quickly. And yet there are performance problems are that are really design issues which were probably the result of snap judgements.

So, the Oracle performance charlatans have a promising future as bad decisions are piled upon bad decisions. As long as you make enough of them, some of them come out right by pure chance thereby enhancing your reputation.


2 thoughts on “Performance Charlatans – Reflections on Mark Thoma’s “The Strong Demand for Charlatans”

  1. Reflections on “Consider Yourself an Expert? Think Again” « Yet Another OCM

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s