[an error occurred while processing this directive]

learning.now: at the crossroads of Internet culture & education with host Andy Carvin

[an error occurred while processing this directive]

Steve Fossett, The Mechanical Turk and Educational Crowdsourcing

Earlier this week I spent part of my lunch break searching for aviator Steve Fossett. And I didn’t even have to leave my desk to do it. It’s an example of what’s known as crowdsourcing, and its educational potential can’t be understated.

You’ve probably heard the news that adventurer Steve Fossett has been missing for the last week, ever since failing to return from a solo flight in rural Nevada. Law enforcement and volunteers have been scouring the region from the air, though in recent days the authorities have started discouraging those volunteers from joining the search, over fears that they might experience crashes or other mishaps.

Nonetheless, I’ve managed to become a volunteer myself, searching the Nevada landscape in the hopes of finding any signs of Fossett and his plane. No, I don’t have a pilot license, but I do have Internet access. And that’s allowed me to participate in a volunteer search effort organized by Google and Amazon.com. Google, of course, is publisher of Google Earth, their virtual map of the entire world. Not long after Fossett went missing, Google managed to update their databases of satellite photos covering the area of his last known coordinates. Assuming he’s somewhere in the region, and his plane isn’t obscured or camouflaged, Google Earth’s photos have the resolution to spot the plane if you look very carefully.

But how do you coordinate Google Earth users around the world when it comes to scouring all of that satellite imagery? That’s where Amazon.com comes in. They’re best known as a pioneering online retailer, but they’ve also developed a tool that makes it possible for anyone to coordinate online projects that involve large numbers of participants. The tool is called Mechanical Turk, named in honor of an 18th century device called The Turk. The Turk appeared to be an mechanical automaton in an Ottoman costume that had the uncanny ability to defeat anyone at chess, more than three centuries prior to IBM’s invention of its Deep Blue supercomputer. In reality, The Turk was an elaborate hoax: an actual chess master was hidden inside the machine, directing its moves against its opponents.

Amazon’s Mechanical Turk, in some ways, is a tribute to that hoax, because it acknowledges that there are some tasks best left to people rather than machines. Let’s take translation, for example. There are plenty of free translation tools out there, but the results are often laughable. And if you had a large translation project, you probably wouldn’t want to rely on translation software as your sole source of translation. That’s where Mechanical Turk comes in. The website allows users to put out requests for assistance on projects that need human assistance, but in manageable amounts. Rather than asking for people to translate an entire book, a user could solicit translation requests for a single page or paragraph at a time. That way, another user who is in a position to help out for brief period of time can contribute to the effort without committing themselves to it for an extended period. And because Mechanical Turk is a website with users from around the world, you can potentially get hundreds or even thousands of participants helping you out, contributing a massive amount of people power to your endeavor.

This concept is known as crowdsourcing. It’s a new word, having just been coined in the last couple of years, but the concept predates the Mechanical Turk’s late 18th century namesake. In 1714, the British government offered a prize to any person, expert or otherwise, who could come up with a methodology for measuring longitude. Flash forward to the 21st century, and you’ve got Wikipedia - an all-volunteer effort in which anyone can spend a few minutes to make additions or edits to an encyclopedia. Individually, those efforts don’t amount to much, but collectively, you’re looking at industrial-scale knowledge production. Another great example is Jay Rosen’s NewAssignment.net project, an experiment in “networked journalism” that helps amateur contributors work with professional journalists on serious news assignments.

So let’s return to Steve Fossett. By using Google’s satellite imagery and Amazon’s Mechanical Turk crowdsourcing community, the effort to find Fossett has gone global. As I ate lunch at my desk on Monday, I perused half a dozen satellite photos, comparing them with samples provided by the organizers so I could get a sense of what I should be looking for. Meanwhile, someone else was looking at the same images, so every one of them gets double- or triple-checked, while other users were looking at images representing adjacent map coordinates.

Only time will tell whether this all-volunteer effort will make a difference in this search-and-rescue effort. But it’s gotten me thinking in recent days about the need for a Mechanical Turk-like site dedicated for educational projects. Mechanical Turk is a commercial application, in which users requesting assistance typically pay others small amounts for their work, typically a few dollars per contribution or much less. Though the Fossett search is a notable exception, the site wasn’t designed for volunteer efforts, particularly educational ones.

I’ve wanted to see a tool like this for some time now. Back in April 1999, I tried to launch a crowdsourcing project of my own in which students from Europe and Israel would work with students in the U.S. to translate yizkor books. These books were tributes written by survivors of the Holocaust in memory of relatives and neighbors who perished in World War II. There are several hundreds of these books, almost everyone one of them long out of print and in the public domain, serving as some of the last remaining records for communities that no longer exist. The yizkor book for my father’s family’s hometown of Busk, Ukraine contained texts written in English, Hebrew, Polish, Yiddish and even Latin. It seemed like an excellent opportunity to create a collaborative project between students around the world who speak different languages, but the project never got off the ground because of criticism from some genealogists who felt that students couldn’t be trusted to do scholarly translations. Using the tools of 1999 - email lists, static webpages and bulletin boards - they might have been right. But would crowdsourcing and social networking tools today make it easier today for a project like this to actually succeed?

I would love to see the principles of Amazon’s Mechanical Turk applied to a website available to the education community, in which students and teacher alike could post requests for assistance regarding educational projects that would benefit from distributed participation. Collecting local water samples to compare water quality on a global scale. Retooling a lesson plan to make it appropriate for their grade level or local testing requirements. Partnering with classes studying a particular language to translate classic works that are in the public domain and making them available to anyone. And yes, translating important historical and cultural works that are in danger of being forgotten.

What about you? I’d be interested in hearing about your own experiences with crowdsourcing, including brainstorms of the types of educational projects that might be possible if tools like Mechanical Turk were used. Maybe we could do a little crowdsourcing ourselves and see what kinds of projects we can come up with. -andy

Filed under : Cool Tools


One cool project that might be considered crowdsourcing is at www.galaxyzoo.org. Three universities, Johns Hopkins, Oxford and University of Portsmouth, have a developed database of millions of deep-space images of galaxies that they need help classifying. After a short tutorial and a quick test to make sure you’re at least more than 50% accurate, participants get to help classify these galaxies. The cool parts are the amazing images you get to see and the fact that you’re seeing images that few, if any, other human eyes have seen. It would be a great project to get a high school physics or astronomy or earth-space science class involved in. Here’s a link: Galaxy Zoo

I’ve thought of this idea many times and now thanks to your post I finally know what to call it: “crowd-sourcing!” Awesome. For foreign languages there would be many many applications for this concept, but what comes to mind first for me is the possibility through crowd-sourcing to allow students to experience “virtual immersion” and practice their language skills with native speakers in a designated location—it could be a place in the real world of course…but also on a computer: a website, chat room or in a virtual world like Second Life.
Finally real-life language practice could be open to all students…not just the ones who can afford an international flight. Of course, when dealing with young people we would have to make sure that such environments are safe - so that parents and the public would be willing to embrace their potential to learning.

It strikes me that crowdsourcing could be an interesting way for the OLPC project to provide educational support to the millions of students they hope to deliver their machines to.

Crowdsourcing. I am reading The World is Flat and i am also about to open a Youth Center for young people between 12 - 17 who are most endangered by gangwarfare, death and the loss of their life dreams. Our computer center would love to see collaboration with other innercity centers internet-linked so as to weighin collaboratively on the question and problem-solving of the challenge: HOW CAN WE BEST FIGHT THIS GREAT THREAT TO OUR INNERCITY YOUTH AND THEIR ENVIRONMENT? A national project poised to crowdsource a solution (or solutions) to this issue would be a realtime approach rather than an annual conference to which only a few well-funded directors can attend. This way, the victims and practicioners of this violence can contribute without the administrative translators (middlepersons) like myself. The results would be richer and so much closer to the victims and social networking solutions that are needed. Can anyone feel what i am saying in this 21st century?

Any updates? Any news on this? It would be nice if Mr. Carvin would write an article with any news on this front.

I’m looking at Mechanical Turk now as a teacher and I’m trying to figure out how to pay for it or how to get the community behind it to help with the grading/evaluation. What I’m interested in is the community Volunteer Mechanical Turk (VMT).

Student: 23423342 submitted sentence containing incorrect grammar . A VMT from the community or from the network evaluates it and tags the sentence as being grammatically incorrect.

If done properly, the evaluation would be immediate and the feedback to students in real time (if there are enough VMTs!).

Cold real world truth:
Teachers don’t assign a lot of homework that they have to grade later. They try to choose the best tool to measure the student’s understanding but for every tool as assignment deliverable they use in evaluaton teachers have to measure how long it will take to evaluate the students’ deliverable. It’s just too much work to grade 100s of papers a day, even small items, as you have to handle/read/evaluate/provide feedback/enter the results into a grade book/handle … it’s all very secretarial and time intensive for most teachers. So, a Volunteer Mechanical Turk could easily fill that void, if the teacher and students are properly equipped with access to the internet and training.

Security? Students are protected as their names would never be included so there’s little worry of child predators… and hey, if colleges do this, some people might have trouble telling the difference between a grade school sentence and a college level sentence… so adult colleges or training programs could use the VMT as well.

You could submit whole essays, chopped into individual one sentence bits for review by the VMT. I wonder if Amazon’s Mechanical Turk has such a tool?

Short answer questions could be processed in a flash… and each answer could then be added to the testing database so that future usages of the test are graded according to the inputs from the prior Volunteer Mechanical Turk input/evaluation.

Think of the opportunities for asking VMTs to produce educational materials. There are plenty of physics grads out there looking to add some more coin to their pockets. What’s the going rate for solving the amount of energy expelled when forces collide? What kind of pushback would the community express when they found out the ‘old way of grading has been replaced with a VMT’. That’s a physics problem for you, how much energy is generated when the forces of old and new collide? Good bye teacher, hello VMT? I can hear the roar of those forces already.


[an error occurred while processing this directive]