Many swear by the Wisdom of the Crowds. But when does a crowd make accurate predictions? Michael Mauboussin studied the question and came up with some interesting insights. Mauboussin wears many hats, at once being an investment advisor on Wall Street with Credit Suisse First Boston (CSFB), and being an academic with credentials in behavioural science. He is the author of many books, including More Than You Know, Think Twice, and Success Equation, and his observations are always insightful.
Mauboussin says crowds tend to make accurate predictions when three conditions are present. There should be Diversity, Aggregation, and Incentives.
According to him, Diversity reduces collective error. Aggregation gives out the assurance that the market is considering everyone’s info. And finally, Incentives reduce individual errors by encouraging people to participate only when they have an insight.
HOW TO DISTINGUISH BETWEEN SKILL AND LUCK?
Mauboussin has studied this question at great length, and has some very insightful observations to make. According to him, to detect whether an activity involves Skill or Luck, ask if you can lose on purpose. If you can, then Skill is involved.
Feedback should focus on the part of the outcome a person can control. Call it the Skill part, or the process. Feedback based only on outcome is nearly useless if it fails to distinguish between Skill and Luck. Here’s a checklist to follow:
1. Evaluate the mix of Skill and Luck. If you can lose on purpose, Skill is involved.
(Now you know why cricket bookies contact the most skilled among the players to throw a match! They sure know how to lose on purpose!!)
2. Carefully consider the sample size. People extrapolate unfounded conclusions from small sample sizes. The more Luck contributes to the outcome, the larger the sample size you will need to distinguish between Skill and Luck. When a large number of people participate in an activity influenced by sheer chance, some of them will succeed by sheer Luck.
3. A streak (hot hand) is one of the best indicators of Skill in a field. Luck alone can’t carry a streak. Streak holders are among the most skilled in their fields.
(With all due respect, I don’t buy this Mike. I once bowled out three players in quick succession while playing cricket at school — a hat trick. If that was a hot hand, and indicated great Skill, you will be surprised to know that let alone the Indian cricket team, I never made it to the school cricket team even!)
4. Watch out for change within the system, or of the system, eg. individual changes in the skill level.
5. Watch out for the Halo Effect (human tendency to make specific inferences based on general impressions).
e.o.m.
Mauboussin says crowds tend to make accurate predictions when three conditions are present. There should be Diversity, Aggregation, and Incentives.
- Diversity refers to the presence of people having different ideas and views.
- Aggregation brings the group’s information base together.
- Incentives ensure there are rewards for being right, and penalties for being wrong.
According to him, Diversity reduces collective error. Aggregation gives out the assurance that the market is considering everyone’s info. And finally, Incentives reduce individual errors by encouraging people to participate only when they have an insight.
HOW TO DISTINGUISH BETWEEN SKILL AND LUCK?
Mauboussin has studied this question at great length, and has some very insightful observations to make. According to him, to detect whether an activity involves Skill or Luck, ask if you can lose on purpose. If you can, then Skill is involved.
Feedback should focus on the part of the outcome a person can control. Call it the Skill part, or the process. Feedback based only on outcome is nearly useless if it fails to distinguish between Skill and Luck. Here’s a checklist to follow:
1. Evaluate the mix of Skill and Luck. If you can lose on purpose, Skill is involved.
(Now you know why cricket bookies contact the most skilled among the players to throw a match! They sure know how to lose on purpose!!)
2. Carefully consider the sample size. People extrapolate unfounded conclusions from small sample sizes. The more Luck contributes to the outcome, the larger the sample size you will need to distinguish between Skill and Luck. When a large number of people participate in an activity influenced by sheer chance, some of them will succeed by sheer Luck.
3. A streak (hot hand) is one of the best indicators of Skill in a field. Luck alone can’t carry a streak. Streak holders are among the most skilled in their fields.
(With all due respect, I don’t buy this Mike. I once bowled out three players in quick succession while playing cricket at school — a hat trick. If that was a hot hand, and indicated great Skill, you will be surprised to know that let alone the Indian cricket team, I never made it to the school cricket team even!)
4. Watch out for change within the system, or of the system, eg. individual changes in the skill level.
5. Watch out for the Halo Effect (human tendency to make specific inferences based on general impressions).
e.o.m.
RSS Feed
