Harnessing The Wisdom Of Crowds
This article is the third installment in Orlando's series on virtual operations support teams (VOSTs). The first installment, Lessons Learned From The Social Media Tabletop Exercise, is available here. The second installment, Structured Networks & Self-Coordinated Disaster Response, is available here.
“We all have to understand that there will never again be a major event in this country that won’t involve public participation. And the public participation will happen whether it’s managed or not.”
— Coast Guard Admiral Thad Allen, Incident Commander for the Deepwater Horizon Oil Spill
“A cynical streak in society looks at all forms of amateur participation as either naïve or stupid.”
— Clay Shirky
The Wisdom Of Crowds
A central tenet of disaster response is that disasters must be handled by a small group of professionals, rather than the large mass of amateurs. This seems plausible enough, but it turns out that some types of problems are better addressed by a large group of amateurs than a small group of professionals. As counter-intuitive as this seems, a large group of amateurs will consistently out-perform a small group of professionals in certain types of situations.
- In 1906, Francis Galton was visiting a livestock fair when he stumbled upon an interesting contest. Local villagers were asked to guess the weight of an ox, with the closest guess winning a prize. Of the over 800 guesses, nobody got the exact weight of 1198 pounds. Afterwards, Galton asked for the ballots and upon compiling them found that the average guess was 1197 pounds. To his surprise, the average guess was more accurate than any particular guess.
- Sociologist Gate H. Gordon had 200 students guess and rank items by weight, and found that the group average was 94 percent accurate, which was better than all but 5 of the students. In another, Jack Treynor had 56 students guess the number of jelly beans in a jar. The jar had 850, the average guess was 871, and only one student made a better guess (Surowiecki, page 5).
- When the submarine Scorpion was lost at sea in 1968 the U.S. Navy wanted to find the wreckage, but its intelligence was too scant to produce a reasonable search zone. Then naval officer John Craven had an idea: He gathered together a wide group of individuals with different talents, from math to salvage to submarines. He then asked them to bet on the submarine’s location. He compiled all of the bets and came up with a group average. The submarine was found an amazing 220 yards from that location.
- The Defense Advanced Projects Research Agency (DARPA) is also actively identifying those types of problems that are better handled through “the wisdom of crowds.” One fascinating study was its “Network Challenge.” DARPA announced that on a specific day it would hoist 10 red balloons around the country, and the first person or group to find all 10 would win $50,000. The choice of competition was not an accident. They deliberately chose a challenge that could not be handled by the current US intelligence community. As they reported:
“The geo-location of ten balloons in the United States by conventional intelligence methods is considered by many to be intractable; one senior analyst at the National Geospatial Intelligence Agency characterized the problem as ‘impossible’.”
The location of 10 red balloons in DARPA's Network Challenge.
A team from MIT found all ten in less than 9 hours. How? They created a network that began with four members and rapidly grew to over 5,000. The original crew created a formula that rewarded people who found a balloon. But it also rewarded those who passed along information on a balloon and those that dispelled false claims. You see, it turns out that some teams were deliberately feeding misinformation to other teams to throw them off of scent.
DARPA did this to test what would happen if terrorists tried to feed misinformation into intelligence gathering systems. What they found was that when misinformation was received into the social media system, the system tends to self-correct.
“Crowdsourcing” is also being used in disaster response. For instance, within days of the Haitian earthquake a group of volunteer programs created Open Street Map, a system that allowed people to compare post-earthquake satellite imagery of the disaster with a Google streetmap and tag a map of the area with information such as “destroyed building, partially destroyed building, hospital,” etc. Then hundreds of volunteers around the world did the tagging, which was turned into a cell-phone and GPS app that rescuers used to guide their efforts.
More recently, the United Nations used “The Standby Task Force,” a group of 700 volunteers from around the world, to gather information about the Libyan crisis. The volunteers monitored information from both mass media and reports on the ground, feeding it into the Ushahidi crowdsourced mapping service, which allowed the UN to keep abreast of the situation on the ground. Ushahidi is now being used to crowdsource information in every disaster that arises in the world, including the current situation in Syria:
Harnessing The Wisdom Of Crowds
These examples do not mean that there is no place for professionals in disaster response, but rather that crowds can provide information and accomplish tasks that disaster managers cannot. It would have taken weeks, if not months, for a governmental agency to set up and populate the Open Street Map for Haiti, by which time the benefits of it would greatly reduced. Disaster responders must stop thinking of the public as a problem to be managed, and start thinking of it as a resource to be harnessed. Similarly, business continuity professionals can start harnessing the power of their company’s employees in a disaster. This requires understanding the factors that lead to “Wisdom of Crowds.”
Surowiecki has identified four basic elements for the Wisdom of Crowds:
- Diversity of opinion: Each person should have private information, even if it’s just an eccentric interpretation of the known facts.
- Independence: People’s opinions aren’t determined by the opinions of those around them.
- Decentralization: People are able to specialize and draw on local knowledge.
- Aggregation: Some mechanism exists for turning private judgments into a collective decision.
The traditional “command and control” model of disaster response closes decision-making around a small group of experts who are assumed to be more reliable than all others. But this model can undermine good decision-making by making it blind to outside input:
“Groups of smart and not-smart agents make better decision than just smart agents because of a diversity of opinion.” (Surowiecki, p. 30)
A group of experts in a particular field will have a specific perspective based on selectively filtering information. However, “the crowd is holding a nearly complete picture of the world in its collective brain.” (Surowiecki, p. 11)
Small groups of experts are also susceptible to the interpersonal forces of committees. Before both the space shuttle Challenger and Columbia disasters, people sounded warnings about the conditions that led to the events. But in both cases the relevant committees that could have acted on the warnings chose to ignore them because once one or more people lent their opinion to one side of the debate, others fell in line. This is often called “groupthink,” and occurs when opinions are formed sequentially among a small group, rather than in aggregate.
Once one or two people chime in on a particular position, others assume that it must carry the weight of logic and agree for no other reason than: others agree. This leads to a decision cascade that overwhelms all of the better judgments (Surowiecki, p, 63).
Crowdsourcing can serve as a counter-balance to the interpersonal forces that might steer a closed group of experts off course. The public is a resource that can assist disaster responders by providing valuable information and perspectives.
The secret is knowing how to gather the information so as to make the public a resource. This is done with a “Virtual Operations Support Team” (VOST)—a group of individuals who monitor social media to feed information back to responders. The Red Cross, public agencies and private companies are increasingly turning to the VOST as a resource during disasters.
Orlando will conduct a one-day pre-conference workshop on Creating & Running A Virtual Operations Support Team (VOST) at the 11th Annual Continuity Insights Management Conference, April 22-24, 2013 at the Sheraton San Diego Hotel & Marina. See http://www.cimanagementconference.com/pre-post-conference-workshops for more information.
The next article in this series focuses on the structure of a VOST.