Crowdsourcing can indeed be powerful. 75 pages translated last weekend, lightning speed for a complex technical document. I believe that v0.9 is highly accurate (more on why later), and this will improve further with v1.0. 1/
3
13
30
Replies
Accuracy was no surprise, as I’m an old hand at crowdsourcing. The shocker was the superficial: the document turned out polished. Tables, diagrams, fonts—they all look amazingly nice for such a rush job. 2/
1
1
3
“What’s the secret?” I’m sometimes asked. My lab has been crowdsourcing for eight years through the citizen science project
@eye_wire
. 3/
1
3
5
I’ve actually been too busy to ever write much about how it’s done. So here’s my attempt to teach Crowdsourcing 101 in a Tweet thread. 4/
1
1
5
#1
Explain significance. Easy here because excellent reporters like
@max_fisher
@choesanghun
@chadterhune
@soheefication
@heesu_lee
@DaslYoon
had already explained why other countries should be interested in S Korea’s COVID response. 5/
6/ S Korea has succeeded in controlling a major outbreak “without China’s draconian restrictions on speech and movement, or economically damaging lockdowns like those in Europe and the United States.”
2
54
116
1
0
4
In the call to action tweet thread, all I had to do was link to their writings and summarize the purpose at the top as “SAVE LIVES by translating Korean→English!” 6/
1
0
2
#2
Set a deadline. I challenged the crowd to finish “by Monday morning.” Deadlines are important for building a sense of urgency. Speed is a universal thrill, and you want people to get that feeling. Video game designers know this. 7/
1
0
3
#3
Quantify progress. “75 pages” made the goal numerical. At the beginning we could count how many pages we had finished, and near the end how many pages were left. 8/
1
0
3
Founding member Alex made a chart of the 75 pages so that we could visualize progress by watching the squares gradually turn orange. 9/
1/ Every square is orange! Thanks to round-the-clock effort by amazing volunteer team, complete first draft translation of all 75 pages. Crowd power is unbelievably swift. Moving to Phase 2, quality control in three areas: technical accuracy, verbal style, visual style.
1
12
33
1
1
3
#4
Open call. This was a complex technical document, so we could have asked for people with backgrounds in public health or medicine. We didn’t because projects usually end up requiring a diversity of skills and backgrounds. 10/
1
0
1
#5
Immediate participation. The tweet linked to a Google Doc seeded with an automated English translation of the original Korean document. The Doc was open to editing by anyone. Within seconds, anyone could start contributing. 11/
1
0
1
We had originally considered putting up barriers, like requiring people to email for permission to edit. This would have created delays, and is less motivating. 12/
1
0
1
#6
Trust strangers. Opening the Doc to editing by anyone meant that it could be attacked by vandals. Malicious people are rare, and anyway a Google Doc can be restored from its version history. 13/
1
0
1
More importantly, why would some evil person attack your crummy crowdsourcing project in the beginning, when it looks like it’s going nowhere? Save your paranoia for later on, when the project becomes successful. 14/
1
0
1
#7
Make it social. Online crowdsourcing can be tough, because participants may feel lonely, especially when the project hasn’t taken off yet. You have to provide them with ways to interact, or at a minimum feel each others’ presence. 15/
1
0
1
People come for the cause, but stay for the camaraderie. These are people who want to make a difference. And they like meeting other people who feel the same way. 16/
1
0
1
Here it was easy because Google, to their credit, engineered Docs to be social. Icons for visitors are displayed at the top. Editors’ cursors are visible. People can leave comments and chat. 17/
1
0
2
#8
Attract an audience. You may find that only 1% of visitors end up working a lot on the project. Don’t worry; this is typical. “Many are called but few are chosen.” 18/
1
0
1
But the other 99% are actually part of the project too. They are the audience. Their cheering will keep you motivated. And when you achieve something great, they will tell the world about it. 19/
1
0
2
#9
Quality control. Once the project is on a roll, define the goals of quality control, and project leads will step up to achieve them. Here the goals were technically accurate content, good verbal style, and good visual style. 20/
1
0
2
Quality control requires more organization and communication, facilitated by productivity tools like Slack and Zoom. We tracked progress with a spreadsheet that recorded review of the three QC goals for the 75 pages. 21/
1
0
1
Getting over the finish line is tough. The first draft translation was done by early Sunday afternoon, but careful quality control made us miss our Monday morning deadline. 22/
1
0
1
#10
Release early and often. Technical accuracy of the translation was an important concern, so it was tempting to delay. Instead we decided to release version 0.9 so that people could start benefiting from the information. 23/
1
0
1
#11
Dissemination. The translation was ready by midday on Monday, but we realized it would have no impact unless we could get people to read it. Luckily, people attracted to online crowdsourcing tend to be savvy at social media. 24/
1
0
3
The team sprang into action. In short order, we had a Twitter account. 25/
1
0
5
A Facebook page 26/
1
0
2
And a web site. Wow that was fast. What a team! 27/
1
0
1
#12
Site analytics. Get some idea of your reach by activating Google Analytics for your site. 28/
1
0
2
#13
Don’t be discouraged by the naysayers. This person is absolutely correct that accuracy is important. 29/
2
0
2
But it’s been shown time and time again that crowdsourcing can actually produce superior accuracy, if you do it right. I could cite the studies but the reason is intuitive: recruiting more eyes tends to reduce the error rate. 30/
1
0
2
And anyway the project attracted many volunteers with expertise in public health and medicine, and others with experience in translation. 31/
1
0
2