A few days ago I was snooping around Digg when I noticed a popular submission titled The Difference Between Digg and Reddit. I clicked, eager to learn, and was presented with an image juxtaposing two very distinct flavors of user-submitted comments surrounding the breaking news of Tony Snow’s death. The first comments shown at Digg offered generic words of respect that you might expect to hear about a public figure that passed away. The top comment at Reddit, however, was a bit more candid to say the least.

The discussion that followed ranged from folks saying “maybe I should join reddit…” to full blown conversations about what works and what doesn’t work in user-moderated comment systems. I have since been thinking a lot about what comments on media items are meant for, what makes a good system, and what tradeoffs exist.

The value of comments
I wasn’t around when the grand forces decided that comments were a must have for digital media outlets, but the conceptual benefits are pretty clear. From what I understand they provide an opportunity for readers to hear what others have to say, contribute their own two cents, show off their wit, and at times simply vent.

It seems like most organizations think of the feature in those terms, i.e. as a way to engage the individual and get some feedback. While that is nice, the real benefit of a comment system is the way it can drive collective intelligence. To put it in less geeky terms, comment systems should take the voices of a large group of people (the collective) and present what they have to say in a way that spreads knowledge and facilitates wisdom (the intelligence).

In particular a good comment system will…

  • Harbor insight – chances are there is an expert on the subject who is reading the article. It is also possible there are readers who are aware of valid points either on their own or via another source.
  • Reveal related information – there are probably resources out there where readers can learn more. Users will link to or otherwise share those resources.
  • Challenge assumptions – if there are false claims, incorrect citations, rhetorical gaffes, or questionable perspectives in the article or in other comments then someone will identify and address the potential mistake.
  • Answer questions – one article can rarely cover everything. Let the hive mind tie up loose ends.
  • Entertain – in the same way that there is room for funny headlines and Jon Stewart there is room for funny comments. It makes the whole news experience more enjoyable and increases the odds that people will pay attention.

Common pitfalls
The magical transition from noise to information is where most comment systems fail, but there are lots of reasons why. Here are a few that I could think of:

  • Ignorance – there will always be people who are outrageously closed minded or blatantly incorrect. I can’t tell you the number of times I have wanted to slam my head against the wall when reading comments on mainstream and local news sites alike. A system needs to account for this and prevent these types of comments from dominating.
  • Group think – It is hard for groups of people to tell the difference between “wrong opinion” and “other opinion.” This means that systems have to prevent the larger group from taking over the conversation and squelching valid alternate points of view.
  • Information overload – Once there are more than about 20 base-level posts it gets hard to take everything in. A good comment system will mitigate this effect and make sure that every useful contribution reaches somebody.
  • Inability to respond – without conversation there is no way for individual ideas to mix and match. This means that users sit around isolated and unchallenged. As a rule of thumb, strictly linear layouts like the kind you see here on the IdeaLab won’t work in settings with an average of more than four or five comments per item.

A Possible implementation
Like any other piece of technology, there is no “best system” here. A local paper that has never had more than 3 responses on any article might not care much about implementing a robust collective intelligence suite. That being said, a good system on a decently trafficked site will get used.

I want to throw out a few ideas that might help the situation. Some are quite old, some are new, but none are used everywhere just yet.

  • Threads – threads are standard on social media sites but they really ought to be standard everywhere. Users should always have the ability to reply to someone directly and have their replies appear in context. This naturally organizes information but it also helps to contain flame wars and hot debates that might otherwise take over and stifle new thoughts.
  • Rating – it is difficult to find a perfectly pure way to rate subjective content such as comments. Despite this it is a worthwhile addition even if it is only used to get a general picture about where community stands on the issues.
  • Randomization – you can’t view all of the posts all of the time, but by including an element of randomness when deciding on which comments to display at the top a system gives a chance to the ones that would have otherwise been lost in the crowd. This will help to prevent the lazy voting feedback loop where the top posts are rated higher simply because they are seen by more people.
  • Categorization – it is hard to compare comments. Some are informative, some are funny, some are insightful, some are insulting, etc. In order to account for this you can allow users to categorize them into predefined buckets. This could be combined with rating by having users select reasons why they rated something up or down.
  • Burial – some comments simply aren’t fit for print. Automatically hiding terrible comments helps clear away some of the noise, but how can you identify them? Ratings may be prone to group think but through categorization it becomes possible to see the difference between bad and unpopular.
  • Geotagging – I haven’t mentioned geotagging in a while, but comments should absolutely be linked to location. This provides some degree of context to individual statements and also gives a sense of scope when looking at conversations. Furthermore you can apply some of the targeting practices that I’ve talked about in the past in situations of information overload.

Now that I’ve said all that I charge you with the task of going out and fixing the way comments work on your sites. Just let me know if you need any help!