Sunday, April 12, 2009

Experts… well, not quite (Part 1)

Note: If you don’t wish to spend your time reading a long post by a rookie, directly go to the example in the closing part of the post.

Who do we call an expert? I think somebody who has been trained in a particular field by a reputed institute and has sufficient experience in the same field. The definition is not a comprehensive one but that’s not what I want to discuss here, rather this is regarding the credibility of all those whom we can refer to as experts. This includes professors, scientists, pundits, senior engineers, and all those who occupy some ‘top’ positions in organizations. (I am very curious about why do we need to create so many levels in an organization? Assistant Vice President, Vice President, Sr. Vice President, President… common, give me a break. More about that later...)

It is very natural for any of us to turn to experts when we are faced with situations we are not confident to handle or when we want some predictions about a very complicated system. Who is more likely to form the government after upcoming elections?, how should India handle terrorism?, or how can we solve traffic problem in Pune? Even though each individual can have his own opinions about such issues; any sensible person, if put in decision maker’s shoes, would most probably discuss the matter with experts in the particular field before deciding anything. Same is true for day to day decisions, either professional or personal. An engineer facing a technical problem which he has never faced before or falls outside his expertise will most probably go to his senior. (Provided the poor fellow is not working under a prof from COEP.)

In some situations, like technical problems, the solution is objective which can be endorsed by sufficient scientific references. Most of the times, however, we are forced to take decisions which cannot be weighed against objective standards. From a minister addressing the issue of national security to a site engineer dealing with customer, each one of us faces different dilemmas.

….I just realized that I have written three paragraphs without actually coming to the point I wanted to write about. I had never faced such a situation before, whenever I was writing an essay (and it goes without saying that it was during some damn exam) I always had to keep on adding something so as to cross the minimum limit….

Coming to the point… To what extent can we really depend upon opinions/judgments of experts for making decisions, especially the ones without any objective standards?

Psychologists and neuroscientists have proved over the last four decades that we humans do not make decisions by deliberately evaluating all the possibilities, especially in complicated situations. Rather, as the great duo of Kahneman and Tversky say in their legendary 1974 Science paper (Ref 1),

“...people rely on a limited number of heuristic principles which reduce the complex tasks of assessing probabilities and predicting values to simpler judgmental operations. In general, these heuristics are quite useful, but sometimes they lead to severe and systematic errors.”

And this process of decision making is not limited to laymen, even the experts are subjected to the same for a simple reason as put by Nassim Nicholas Taleb

“We are, whether we like it or not, prisoners of our biology.”

Here is what Kahneman and Tversky say

“The reliance on heuristics and the prevalence of biases are not restricted to laymen. Experienced researchers are also prone to the same biases-when they think intuitively. For example, the tendency to predict the outcome that best represents the data, with insufficient regard for prior probability, has been observed in the intuitive judgments of individuals who have had extensive training in statistics. Although the statistically sophisticated avoid elementary errors, such as the gambler's fallacy, their intuitive judgments are liable to similar fallacies in more intricate and less transparent problems.”

I think the post is already too long to be read by any practical person. So I would stop here for today and continue with the same topic later. I will close it with following example from Fooled by Randomness.

"The following quiz was given to medical doctors

A test of a disease presents a rate of 5% false positives. The disease strikes 1/1,000 of the population. People are tested at random, regardless of whether they are suspected of having the disease. A patient’s test is positive. What is the probability of the patient being stricken with the disease?

Most doctors answered 95% simply taking into account the fact that the test has a 95% accuracy rate… Less than one in five professionals got it right."

What do you think is the right answer? Even if you don’t like probability, you should give it a best try as someday you can be the one being diagnosed.

References

  1. Amos Tversky and Daniel Kahneman. "Judgment under Uncertainty: Heuristics and Biases" Science, Sept 1974, Vol. 185 (4157), pp. 1124-1131.
  2. Nassim Nicholas Taleb. “Fooled by Randomness”.
  3. Jonah Lehrer. “How We Decide".

4 comments:

  1. Well, there's something i would like to chip in about making decisions...
    Would you agree:'If decisions weren't taken at the last moment they would never be taken'...

    ReplyDelete
  2. Is the answer to your problem 95/100000 ???

    ReplyDelete
  3. yes thats right... 1/51... even less than 2%

    ReplyDelete