Targeted, Democratic Content Moderation
Dan Schultz
2:10 AM | bio | Comments: 3
That description probably sounds very similar to Digg's Upcoming section, but this post should help differentiate the two. I'll describe a quick twist that turns an open and fairly loose peer review scheme into a targeted one that (I think) stands a decent chance at providing accurate regional and topic specific news without losing article integrity. Please keep in mind that this is all a continuation of that "perfect news system" discussion from way back.
What does someone living in Wyoming know about the local elections of a Pennsylvania suburb? How can someone who doesn't follow the latest in physics rate the potential validity of an article about the 100 billion light year void in space? I'm betting similar questions can be asked about most stories that don't have to do with iPods or national politics. This is a problem because the whole point of this giant news system is that it would be comprehensive in its coverage and would contain exactly the type of niche news that is only familiar to niche audiences. This means I'm going to have to take some extra precautions during the review process or people will be voting on articles without any understanding of the related community standards or the surrounding issues.
To address this predicament I'll steal a page from academia and try to target domain experts during the peer review process. Of course it wouldn't be much of a democratic system if I restricted the voting process to proven experts, and figuring out who is an expert simply isn't feasible in such a dynamic setting, so I'll take the next best thing - residence and interest. Since the system is centralized and users specify regions of residence and topics of interest anyway, it should be easy to have members of a given physical or topical community be the ones to review their community's news.
Now, using the targeting method, when an article is uploaded, categorized, and tagged to relevant locations it can be pre-screened by users who are somewhat connected with the affected community. Clearly a connection doesn't make one an expert, but it does indicate a clue. My underlying assumption here, which I would love for Ben Melançon (and others) to comment on at some point, is that informed individuals (i.e. those with a clue) can do a fine job of democratically moderating the content that falls within their collective domain.
The next thing to do is give the journalists a unique say of their own on top of it all. That will be a bit trickier, but I have hope that it can be done in an elegant way that makes as many people happy as possible. I'll open up that can of worms the next time I continue this thread of discussion.
Topics: Best Practices, Participation
Tagged: moderation, targeted moderation, user content
Creative Commons Attribution NC SA 3.0 License
Very good thoughts, Dan, and well-presented.
Knowledgeable input is essential. You're absolutely right, and it's a question I've avoided. Not sure what the answer is, but it seems you're on a good track.
Very quickly: In PWGD's conception of democratically moderated information, all submitted articles would be available somewhere, and the extent of distribution (push to e-mail, RSS, SMS, prominence on a web site, etc.) would be determined per-community of interest or geographical area. Unlike Digg, which has whoever happens along (or lives on the internet watching Digg) do the ratings, a random sample of people in relevant communities would be asked "should this go out to people" (in Pennsylvania, or who care about elections of some type). By fortune, this system seems a bit like the one you're proposing.
Ultimately, I think the expertise versus mass input question should be decided within the context of democracy. That is, a democratic "what's important, what's true" system that uses the knowledge of those who have it in its decisionmaking.
The hope for PWGD is that this sets up a communication system where learning is possible: those who are wrong don't get the same weight in public opinion the next time around.
You know, a cool way to handle this would be through some kind of bootstrapping... You talk about finding a way to identify proven experts in such a dynamic environment, but I believe the problem has already been solved. If you seed the network with a few known experts, hand selected, you can easily bootstrap your way up from there. Those editors review the work of new editors, and as gradually the new editors become more trusted and are able to help identify the next iteration, and so on. A lot of work has already been done in this area (look up trust networks or webs of trust).
A perfect example of what I'm proposing conveniently exists at the bottom of this very comment box. Take a look at that reCAPTCHA... do you know how it works? The system presents two non-computer-readable words to the user. If the user types them in correctly, he is allowed to post the comment. But how does the system know the user entered the correct text? The key is in the bootstrapping... two words are presented, but one of them has already been solved and confirmed by previous reCAPTCHA users. If the user solves the known word, the system assumes he solved the other one correctly too, and both can now go back into the pool of words with higher confidences as to their correct solution. By building off of this, the system is building ever-higher confidences in the validity of its pool of words.
Perhaps similar methods could be applied to the problem of democratic content moderation.
Hey tfe, that's a really cool idea.
Does it mean people can't know which is the test and which is the real? Because it'll be a little harder for people to provide news, analysis or editorial services not knowing if they will be used at all!
And that aspect of not knowing is key for the recaptcha approach I think.
Got another trusted web solution?