October 12 was a day of high emotion; it was finally time to thrive under pressure. I got home from work, rushed to my friend's house, and cracked open my laptop. The goal was to brainstorm like crazy, write up some solid project descriptions, and submit as many Knight News Challenge grant applications as possible over the three days I had left. Thank goodness fate had a better plan: the deadline was extended.

Now that we all have another two months, I'm going to take a few steps back and try to combine my formal education in information systems with my Knight-sponsored crash course of journalism's ongoing transformation and the lessons I've learned from three years of applications. I hope that the resulting guide will help you come up with some good ideas of your own. Remember, there is no limit to the number of applications you can submit. (Note: I won a 2007 Knight News Challenge grant for $15,000 to blog about
"connecting people, content and community.")

Well-designed solutions require well-understood problems. When brainstorming ideas for your News Challenge application, the first step is to write down a long list of complaints. (This also works if you already know what you want to do.) What do you want to change? What isn't working well today? What isn't happening that should? This list will give you a starting point, and will make sure that the ideas you come up with will have a real purpose.

Complaints might start from your own personal experience, but you need to expand from there because the true gems come from a fusion of imagination and research. Here are some ways to gather information:

• Talk to people. If you are looking for new problems, ask people you know what they would change. If you are looking to better understand one that you have already identified, ask if the problem exists for them and why. Their insights could provide the spark you need to turn a potentially good opportunity into a great one.
• Pay attention to buzz. Read what other people are saying about your issues. I can guarantee that as you read this there are members of the blogosphere discussing their own trials and tribulations with new media. This will help you get a general understanding of how people fit into the way things work, where they see opportunities for improvement, and which direction the crowd is moving.
• Know the current process. You can't change something without knowing what it is you are changing. Even if you plan to completely redefine the status quo, you need to appreciate and learn from the way things work now. There will always be something worth incorporating or maintaining.
• Explore the cutting edge. What are the front-runners doing, and what problems are they addressing? The cutting edge is known for being risky because nobody is sure of the best solution. Look at the problems that they are addressing and add those to your list. With any luck, you can think of a better idea.

The more you understand the tools available to you, the more effective you will be at finding creative applications for them. Ideally, you want to get to the point where you can have an intelligent conversation with a programmer, but for now it is enough to just have higher-level knowledge.

For each technology you think you might be able to use, figure out:

• How it works. While you don't have to be able to program, you really should have a general idea of how the magic happens. So long as you know what the tool is called, you will be able to find an accessible guide. Just remember that Google is your friend here.
• What it does. Tools tend to have an intended purpose, although often it's a very broad one. Be sure to understand what that purpose is. You can start by looking at its website and see what its creators say. Also try to find out which existing sites have used it, and examine what they have done.
• What it could do. Once you get a basic understanding of a tool, you can start to get creative and think of ways to use it that its creators never would have thought of. This task is all on your shoulders, but you can always scour tech blogs like TechCrunch or SlashDot to find examples of how people can push the limits.

If the technology is at all popular there is probably a community surrounding it. Once you find it, create an account and join the party; there will be people willing to help you learn.

You have a list of tools and a list of goals, now it is simply a matter of creative application: find out how to achieve those goals with the technologies available. Brainstorm as many solutions as you can for each problem, and be sure to dream a little bit here.

For each solution you need to be able to explain:

1. What it would do and how it would be used.
2. How it could fit in and what it could change.
3. How it would incorporate technology and/or people.
4. What assumptions would have to be met and how you would meet them.

You will probably find yourself coming up with new tools, new processes, or (more often) a combination of the two. If you're having trouble, try looking at how people are solving problems in completely different fields. Maybe you can learn from their work. Just remember that you don't need to know how everything fits together just yet.

This is when you descend from the land of the theory and optimism and take a close look at the world. You have some ideas already thanks to your list of solutions, but there are plenty of others to be found; plus, not all of what you have will work. What looks promising? Which ones can you cross off the list?

Some things to think about:

• Who are you dealing with? You need to understand your stakeholders (i.e. the people who would be impacted by your project). You want your solution to provide them so much value that they will be willing to donate time -- and maybe even money. At the very least, they need to be willing to try it.
• What is your competition? There will be direct and indirect competition, but you need to know about both. This will help you differentiate your idea from what is already out there, and it can also force you to further develop your solution into something even better.
• What resonates personally? You are going to need to explain why you are the person to take this idea and make it a reality, so figure out what you bring to the table and make sure you can get excited about it. It won't be enough to say, "I thought of it!"

To get funding from the Knight Foundation you need to be able to convince the world that what you have is a genuine opportunity. It should have the potential to redefine landscapes.

Take your most promising solutions and try to envision their implementations. What is going on behind the curtains? What kind of synergy can you create? Even though you are getting more concrete, don't get bogged down in unimportant details -- you are still brainstorming here.

Think in terms of process (how things get done), objects (e.g. "news article," "user contributed question," or "media clip") and user roles (e.g. "journalist," "editor," "consumer," or "judge"). How will the objects interact? How will each role fit into the system? Refine this line of thought with your previously researched understanding of your stakeholders, the current state of affairs, and the technologies available.

By now you've probably conjured up something spectacular, and filling out that first round application will be a breeze. With any luck you will have enough inspired thought to submit more than one. Good luck!

Last week I visited Carnegie Mellon University's website for the first time as an alumnus. The front page, often dedicated to highlighting faculty work, had a picture of an iPhone screen displaying brightly colored data visualizations. I didn't have to look past the first two words of the title -- "Citizen Scientists" -- before I knew that it would be worth my time to keep reading.

The article described how Eric Paulos, an assistant professor in the Human-Computer Interaction Institute, is equipping "everyday mobile devices" with sensors used to collect reliable scientific data. The point of all this effort is to create "a new generation of 'citizen scientists,' connected both to the environment and each other." Just in case that quote isn't clear enough: the study might as well have been funded by The Knight Foundation.

What's interesting about this project is that members of the scientific community, a professional group that arguably maintains higher standards for verification than journalism, are trying to harness the crowd in the same way that we are. In fact, they are actually going out of their way to do it. This demands an explanation!

At the end of July, an amateur astronomer found a giant hole on Jupiter. Some people were surprised that the ocean-sized crater was discovered by a hobbyist instead of a professional. I, however, wasn't shocked. My reaction could be the result of a personal perception that astronomy is relatively accessible to "the common man." Or maybe it's because looking to the sky is such a primal thing to do. Either way, the subsequent media dialogue exposed me to several new spins on the "amateur vs. professional" debate.

A few days later, I saw a follow-up story on Digg about another amateur observation. This time someone found a bright spot on Venus. It was admittedly less interesting than the news from earlier in the week. I scanned the comments anyway and saw a highly rated one that asked, "Why is everything first discovered by amateur astronomers? What the heck are we paying the professional astronomers for?"

Using my recently gained knowledge of the issue, I replied and pointed out that professionals have bigger and better things to do; it doesn't make sense for a PhD to use a million-dollar telescope to look at something that a hobbyist could view using a thousand-dollar one, especially when there is so much of the universe left to unlock.

Once again, there seems to be a lesson here for journalism.

It's important to know where amateur and citizen scientists fit in relation to professional ones. Here's how I define these three broad classes of scientist:

• Professional Scientists - These people make a living from science. They have the expertise, the patience, and the resources to handle the big stuff.

• Amateur Scientists - Folks who tackle science as a hobby. They enjoy participating and have varied levels of knowledge. They also have the capacity to make direct contributions to the scientific community because they're exploring the same reality as everyone else (regardless of what quantum physicists and philosophers might say).

• Citizen Scientists - These individuals are equipped to contribute to science when they are empowered by tools and networks. They aren't out conducting experiments or learning methods, but they are generally willing, with the help of professionals, to provide crowd power and reap the benefits of the resulting information.

Here's the interesting part: between the story of the amateur astronomers and the vision of Eric Paulos, all three types of scientist have beautifully compatible relationships.

Professionals can safely focus on daunting tasks, knowing that amateurs are ready and willing to take on the smaller stuff (like keeping tabs on Jupiter). The community standards are clear and ultimately bound by cold hard observable fact, so amateurs can make meaningful contributions without diluting the knowledge base. Meanwhile, citizens are being empowered by professionals to help the scientific cause in a way that informs individuals and improves their lives.

Now it's time to take the leap back into the land of journalism. If you buy my claim that scientists and journalists all care about informational integrity and the quest for truth, then several things can be extrapolated:

1. If professional journalists take the lead by clearly defining expectations, explaining best practices, and implementing an accessible infrastructure, then amateurs can contribute without disrupting the industry.
2. If amateur journalists do a good job of covering a smaller scope of topics or areas (e.g. the hyperlocal), then professionals can focus on the deeper, otherwise inaccessible issues.
3. Professional journalists are responsible for creating and maintaining the citizen network if they want it to meet their standards.
4. Citizen networks need more than a host. In order to reach full potential, they need to be explicitly empowered through tools and guidance.
5. A symbiotic relationship between the professional, the amateur, and the crowd is not just possible, it's socially optimal.

And there we have it: If the journalism industry can create an infrastructure that allows amateurs to contribute reliable information, then professionals will be able to dedicate more resources to epic reporting. If local papers can find the capacity to set up and empower meaningful citizen networks, they will establish a major foothold in the evolving domains of community and information. Man, science is useful.

Now we just need to define the standards (and explain how to meet them), create that infrastructure, and set up those networks.

iPhone image via Carnegie Mellon University.

But wait! It's a trick question, and not just because there are countless examples of all classes of reporting from both mainstream and creek media. The trick is epistemological: The existence or non-existence of these qualities on either side is practically meaningless if nobody can prove they exist to the audience.

What makes this trick important is that a person who doesn't believe in the benefits of mainstream media probably won't do much to support it, and I worry that too many people don't. That's the bad news. The good news is that technology can help newspapers, and all other deserving entities, earn trust among a generation that includes skeptics with access to a world of information.

Some Non-Representative Opinions

During the conference at MIT last month I decided to ask some back-at-home friends how much they value professional journalism. The results aren't exactly uplifting, but obviously they don't mean much -- I doubt Gallup would back my informal "methodology." I'm also betting a lot of the people I asked were thinking of network news when voting (one person commented "I voted for 5, but would vote for 1 if Wolf Blitzer [were] not included in that category.")

The reason I mention this private poll is because it reflects a small-but-still-too-large sentiment of mistrust toward the mainstream media that I've noticed among peers and Internet brethren. I'm not old enough to know if this is new or if it's been around since the dawn of time, although I assume the latter. The truth is, I also find myself finishing articles only to wonder what wasn't said; and why shouldn't I?

Fox News shows how easy it is for a media source to shape people's understanding of reality on a massive scale. Goodness knows I don't want to become one of those sheeple things I hear so much about.

The main issue is that most mainstream media is lumped together with political pundits, polarized reporting, editorial echo chambers, and infotainment-driven news cycles. Is it fair? Maybe, maybe not, but either way it's something that must be considered: In the eyes of some, The Watchdog has become The Man.

Defend Your Credibility

Clearly a business model or distribution channel that prevents free riders would be nice, and providing so much value that everyone starts subscribing again is also a great plan, but what about all the people out there who would vote for option #4 in that poll? It seems to me that since professional journalists are doing what the industry says they do anyway, it's time to use technology to leverage that effort and eliminate any room for doubt.

Here are some thought stubs about ways to show the audience why they can trust a specific news source. I may explore them in more detail in future posts.

• Make your source documents public - I was so excited when the Obama Administration made it easy to read all the official documents on whitehouse.gov, until I realized I didn't have 5 hours a day to read them. What I want is the ability to read a reporter's analysis and have the means to check their work and form opinions of my own. (Word on the street is that somebody awarded a grant to develop a technology that could help with this effort...)

• Provide Timelines and Notes - How much time was spent on this article? When did the research begin? What contextual factors influenced its development? Did anything get cut out that might still be worth showing in a footnote? The more information you can provide about the creation process for the audience the better.

• Share Author and Editor Information - I know that disclaimers are provided when appropriate, but the more information about the people who shaped the analysis, the better. What else has this author written? For who? Where does the editor fall on political issues? Maybe some of this is none of my business, but then again, maybe it is.

• Reference the Past - If you know the context of a new story, chances are you documented the context in the past, and maybe you even have some archived reports about it. Link to it. Even better, make a technology that allows for line item comments on old articles so that people can understand how it fits in. There is a lot of potential here.

• Track your News Trends - What if I could see a visual summary of the topics covered by a news organization over time? Then I could know for sure how much was spent on crime, entertainment, world news, politics, etc. Ideally this would also show me what the specific trend was about - did CNN spend more time on Michael Jackson than Iran? I don't really know for sure, but I'd like to.

Now that I look at it again, this might be the sort of added value that actually matters anyway. It differentiates those with time and resources from those without it while bettering the quality of the product. Sounds like a 2-birds-for-1-stone situation!

I hate to think that a newspaper can't take advantage of free, open source, low hanging fruit simply because it would create another system that has to be taught and maintained! At the same time, though, I very much appreciate the position of the incredibly stretched-thin tech guys. This post is about better understanding the "why nots" that discourage newspapers from adopting new technology and trying to figure out if there is any hope of getting around them.

The Barriers

In hobby-land, there aren't too many reasons not to play with technology. Nerdy people set up -- or program from scratch -- wikis, forums, and brand new systems just for fun all the time. Their peers are generally savvy enough to use the stuff without instruction and, although terribly sad, a "datapocalypse" won't cost anyone their job. That's probably why cutting edge is often discovered in the garage -- garage dwellers have much less to lose.

Newspapers need to innovate the same way garages do, or at the very least reap the benefits of innovation, so why is it so hard for them to do so?

• Cost of Setup -- I've mentioned legacy systems before; here is a place where they can constrain. If there is any chance that the new won't play nicely with the old, then systems administrators have to take extra precautions lest they break something that's being used. Worst of all, it's always possible something will go wrong anyway.

• Cost of Implementation -- Do databases or servers need to be set up? Does software have to be installed onto multiple computers? Do accounts have to be created for every user? What settings should be tweaked? Even if the software is free, getting it ready to use by a whole team of people might not be easy to do.

• Cost of Maintenance -- I'm told nothing works perfectly, although I hate to believe it. When things that employees use break, someone has to take the time to fix them. Every new tool hosted in-house is another thing that could go wrong and take a day to fix.

• Cost of Backup -- In the words of Tim Dunham, the CTO at the Post-Gazette, if a pet system becomes mission critical, it has to be treated like a mission critical system. This means that if a wiki containing organizational knowledge is set up and relied upon, it can't ever go down and the data can't ever be lost. Having a backup and recovery plan in place becomes essential, and that takes resources.

• Training and Learning Curves -- After hearing some horror stories about even the most minor software changes causing confusion, it seems reasonable to expect trouble when pushing people to use something completely different from what they're used to. How will everyone learn about it? Will they have to be taught? Is there documentation? Is the interface straightforward? And the answers to these questions will probably lead to more work.

• Functional Overlap -- It is possible that the need that will be addressed by new software can already be partially addressed by the software that is already set up? In some cases this means that the new idea really isn't worth the effort, but in others this line of thought might just be an inner excuse to avoid costs and risks associated with adopting new technology. Either way, you don't want to maintain two systems for the same task because it will create miscommunication and general confusion.

Some Workarounds

The above list may seem depressingly long, but never fear! There are ways to fake flexibility and nimbly try new things. Although I can't speak from experience, I'm going to throw a few ideas out there -- take 'em or leave 'em.

• Start Small -- Since the dawn of time, people have found ways to lessen the blow when dealing with large scale projects: developers make prototypes, web applications have closed betas, and cavemen probably made miniature wheels before trying full-sized ones. Not everything needs to be launched full-featured and full-scale up front. Pick a small group of people to try the new internal project first, or set up a smaller portion of the feature set, or just use default settings instead of spending hours tweaking to perfection. Doing these things will give you time to work out kinks, get feedback, and figure out how/if the new service might be used before you spin your wheels.

• Try a Web Service Instead -- Using an external product is a risky idea for many reasons, and I wouldn't suggest doing it for anything that involves trade secrets, information storage, or truly vital processes. But if it is possible to use a service someone already provides for free or for cheap, it's worth giving it a shot. The positive is that you can probably get it up and running in less than an hour and you don't have to worry about maintenance. The negative is that you lose control and increase risks -- what if the service dies off? What if someone steals data?

• Create an Experimental Environment -- You don't need much computing power to host an internal service. Heck, my 9-year-old laptop could probably serve a wiki or chat server without much trouble. Set up a place where techies can try new things without being worried about important tools breaking or data being lost.

• Run an Inventory and Cut out the Fat -- One thing that nobody wants is technological bloat -- i.e. having more services than are needed. It makes things confusing for everyone and creates unnecessary maintenance overhead. Take some time to go over the services currently available and what needs they address; try to find opportunities for improvement, consolidation, or system retirement. If this means you need to temporarily move backwards to open up resources for new opportunities, so be it.

• Centralized Documentation -- Having all documentation, guides, and general commentary in a central location is an incredibly useful way to share technical advice. If done right this will lower the time spent answering common questions and provide a nice way to communicate new information as new tools are launched. I would recommend a wiki for this, but there are definitely other ways.

• Automate and Streamline -- Menial tasks in the tech room are probably already automated. For instance, hopefully backups aren't being done each night by hand. If a chore is taking up too much time, try to automate it so that the whiz kids can work on bigger and better things. It is also worth thinking of ways to do this flexibly to save time down the road when additional steps need to be added.

Final Thoughts

The missing section of this post is a list of tools to try -- I'm hoping that you guys and gals could help me fix that in the comments. The point, though, is that newspapers need to have the freedom to try new things in a way that doesn't add much to their technical overhead. Regularly incorporating technology to improve day-to-day operations is incredibly important for the future of reporting.

It is also important to keep in mind that not everything attempted will work, and not everything will get used; just remember that if shots aren't taken, nothing gets hit. The risk of an individual project failing -- once again, many will -- isn't so dire if the setup costs are low. This is why starting small is so important -- it allows you to throw a lot at the wall and see what sticks.

Seriously, though, any suggestions, success stories, or tool ideas would be greatly appreciated!

The Spirit of Innovation Lives Strong

I have to start by commenting on the number of good and, maybe more importantly, innovative ideas that are being considered and pursued at the Post-Gazette. From day one it was exciting to see the plans that were already in motion. Put bluntly: many of the people at the Post-Gazette get it. They understand what the Internet is about and the potential that it brings for them and the communities they serve. Better yet, they even have some great homegrown ideas about to how to take advantage of that understanding.

I'm betting that they aren't the only ones, and if that bet is any good then neither the Post-Gazette nor the industry is suffering from a shortage of creativity, an inability to understand the potential of new technologies, or unwillingness to try new things. Yet if you look at the web sites of a lot of these papers, (yep, even the PP-G's), you would probably think that my previous claims of support were just an embarrassing attempt to kiss up.

You'll have to trust me when I say I'm telling the truth... Now the question is: what is preventing the changes from happening?

Some Major Disadvantages

Everyone reading this knows that newspapers are in a pretty uncomfortable position. Even after ignoring fiscal concerns they are under pressure to add substantial new features along with new processes and they can't afford to wait until precedents are set (since by then it could be too late). To make matters worse, they have to publish a paper every day.

In other words, they face almost all of the problems that tech startups have to tackle with a few added bonuses:

• Minimized Flexibility - Assuming the org already has a web presence it isn't working with a clean slate. The existing "legacy" system - be it in an in-house creation or some kind of purchased software - represents a past investment and provides vital functionality. The problem is that it also constrains improvement. Realistically this means that any new stuff will probably include awkward workarounds, an inability to integrate smoothly, or blatant loss of functionality.

• Doubly Limited Resources - From what I hear, software development is costly when you don't do it as a hobby, especially when you are trying to be cutting edge. This is restrictive enough, but it becomes even worse when these costs, combined with a tight budget, force shortcuts. For instance, having to buy existing software that only gets you part way there (and can't be easily extended), or creating one-time widgets instead of being able to spend extra time to make something reusable.

• Newspaper Overhead - It's easy for me to forget that papers don't exist to innovate; they exist to report the news and in doing so serve their community. As a result, tech people have to spend a lot of time on more mundane day-to-day support tasks instead of designing and developing big changes. Constantly looming newsroom deadlines also make it difficult to fully incorporate some aspects of good software design that require stakeholder (i.e. non-programmer) participation, such as focus groups, feedback sessions, and user tests.

Is There Hope?

Yes, yes, a thousand times yes. Despite these hurdles and the ones I'm not mentioning, papers actually have some pretty epic things going for them. For the same reason they have legacy systems, they also have legacy assets. Here are some things that money can't buy:

• Established Audience - I was told that the phrase "If you build it they will come" doesn't apply to the Internet. This is absolutely true for Dan the Programmer, but if you are a mainstream Newspaper you have the power to direct readers to check out new features. You get that all important shot at injecting life into a new-born system. In other words, if you build it they will come (that doesn't mean they will stay, of course).

• Community Allies - If a paper has played a positive role in their community for any significant period of time, it is not a lonely island. Local groups are out there that would love to join in on a mutually beneficial, innovative relationship. This means that if there is a new community service that the paper wants to provide, it can surely get some help.

• Content and Knowledge - Content may not sell, but it certainly adds value. Furthermore, papers have a tacit and deep knowledge of the communities they serve; something that stupid Facebook will never be able to snag. Since "know thy audience" is just as much a tenant for software design as it is for journalism, this is quite an advantage.

• A Very Dedicated Work Force - Newspapers are one of the few kinds of organizations that are literally awake 24/7. This speaks to the extent that the people there care about the paper's mission. Since the technological success of the paper is now a part of that mission, they presumably care about that too. Reporters and editors may not be able to program (which is perfectly fine, I might add), but that doesn't mean that they can't be involved in the improvement of the paper's online presence.

Some Final Thoughts

I'm curious to know what other disadvantages and advantages people see for papers - be it from experience or speculation. Please throw your $0.02 in the comments!

I'll wrap up with some random pieces of general advice, in no particular order:

1. Get feedback from everyone. If any significant development is made without actively soliciting feedback from everyone on staff, before, during, AND after implementation, then the paper is missing out on hugely important insights.
2. A great idea can fail miserably with a bad interface. You might have the best idea ever for a participatory system, but if the implemented U.I. isn't easy to use or is hard to find, then it will never reach full potential.
3. Find someone that has an eye for good on-screen visual design and do lots of user tests before releasing. This role could probably even be filled by a few current employees instead of an outside hire. Just remember, even if you think the product looks and feels amazing, chances are it is unattractive and difficult to use.
4. If you invest in out of the box software, make sure it can be easily extended to do what you want it to do. If it can't then you are doomed to either come short of your goals or spend more time finding a work around than it would have taken to code everything from scratch.
5. Open source doesn't mean inferior. I'll say it again: open source doesn't mean inferior. Part of the reason for this is that open source products often have awesome support and development communities by definition.
6. Don't let programming languages or similar concerns prevent you from adopting a better technology. Make your techies learn something new. The week or so that it takes for them to ride the learning curve will cost far less than a bad decision.

The Million Dollar Questions
The first thing you learn in an I.S. class at Carnegie Mellon is that before you can effectively design a system you need to understand three things:

• The People - Who will be affected by this system? What are their needs and desires? What are their current roles?
• The Process - How do things work now? What can we do better? What do we want to do? What has to be done? What are our limitations?
• The Technology - What technologies can we work with? What can they do naturally? What can we push them to do?

These are the high level questions you should be asking when planning to do anything technology related, and you should have the answers long before a dime is spent on programming or design. If you dart into a project carelessly and/or wildly flailing your arms then chances are high that you will end up with an expensive failure.

Six Steps for Success
Addressing those questions isn't easy; in fact, I'm sure it is much like writing an investigative news report. You enter the process with a high level concept and you need to dig down into the details, adapting your vision along the way. The key, though, is that it is much cheaper to adapt before there is code to change. With that in mind, here are the steps that will help you get ready to develop.

1. Identify stakeholders - Stakeholders are "people who will be affected by the project or can influence it but who are not directly involved with doing the project work." This might be readers, reporters, editors, community members, etc. The list will be different for every project and you will probably add more as the idea develops. Thinking in terms of stakeholders makes it easier to keep the needs and backgrounds of different kinds of users in mind, which in turn makes it easier to design an information system that will be helpful to everyone.
2. Gather requirements - Requirements are "statements that identify a necessary attribute, capability, characteristic, or quality of a system in order for it to have value and utility to a user." In other words, what does your system have to do? This includes functional requirements (e.g. users need to be able to upload and tag photos) but there are also non-functional requirements (e.g. the website needs to process all requests in less than 5 seconds). You'll have your own ideas, but never assume that you have all the answers; after interviewing stakeholders you will realize that their needs are different from what you would have expected. This is, by far, the most important part of a project's lifecycle since it will shape just about every decision; unfortunately, it is also the easiest to botch simply because people aren't good at clearly explaining what they want.
3. Understand constraints - Constraints are "restrictions on the degree of freedom you have in providing a solution." How much money can you spend? Do you have a strict deadline? Can you hire people with new skill sets? Is your website already using a particular piece of technology that you will have to keep supporting? All of these are limitations which need to be considered while deciding on a final solution.
4. Research technologies - There might be something out there that can help you do what you want to do; in fact, there might be something out there that does what you want to do. Each project will have its own unique technology needs based on the requirements and constraints you identified, and researching the landscape will enable you to select the best solution for your special case. This can save a lot of time and money down the road, so please, do your homework and avoid the kool-aided urge to skip this step and just jump to the latest buzz-on-the-street tech! Some things to consider while looking at a technology: What needs of yours will it meet? Is it overkill? How new is it? Is there a support community built around it? What are people saying about it? What are developers saying about it? What other sites are using it? How are they using it? Will you have to hire or train people?
5. Brainstorm solutions - By now you know what you need to do and you know the tools that are out there. Figure out all the different paths you can take to get where you want to be without violating any constraints. For each item on your list come up with pros, cons, and (if possible) estimates for the amount of time and money the solution would cost. For instance, when creating a brand new website one solution might be to hire an external firm that develops Drupal systems. Another might be to hire Ruby programmers and create the site in house using Ruby on Rails.
6. Pick the best solution - You're going to have to choose one to run with. Before you do, though, minimize unanswered questions. It is very inexpensive to learn more early on - it is much worse to realize 6 months down the line that you made the wrong choice and could have finished the job with half the price or in half the time. Do you know that the technology can do what you want it to do? Are you sure your staff can learn the skills required? Will your stakeholders actually approve of this solution and use the darn thing? To help this effort it might be helpful to do a prototype, which is a quick, cheap version of your system designed to answer your questions and, in project management jargon, "mitigate risks."

Once you have done all this you still have plenty of work to do before anyone should start churning out code. This should get you started, though, and it will make it much easier to talk with developers and contractors down the line. Now, go hire an Information Systems student so you don't have to battle through this brave new world alone!

The value of comments
I wasn't around when the grand forces decided that comments were a must have for digital media outlets, but the conceptual benefits are pretty clear. From what I understand they provide an opportunity for readers to hear what others have to say, contribute their own two cents, show off their wit, and at times simply vent.

It seems like most organizations think of the feature in those terms, i.e. as a way to engage the individual and get some feedback. While that is nice, the real benefit of a comment system is the way it can drive collective intelligence. To put it in less geeky terms, comment systems should take the voices of a large group of people (the collective) and present what they have to say in a way that spreads knowledge and facilitates wisdom (the intelligence).

In particular a good comment system will...

• Harbor insight - chances are there is an expert on the subject who is reading the article. It is also possible there are readers who are aware of valid points either on their own or via another source.
• Reveal related information - there are probably resources out there where readers can learn more. Users will link to or otherwise share those resources.
• Challenge assumptions - if there are false claims, incorrect citations, rhetorical gaffes, or questionable perspectives in the article or in other comments then someone will identify and address the potential mistake.
• Answer questions - one article can rarely cover everything. Let the hive mind tie up loose ends.
• Entertain - in the same way that there is room for funny headlines and Jon Stewart there is room for funny comments. It makes the whole news experience more enjoyable and increases the odds that people will pay attention.

Common pitfalls
The magical transition from noise to information is where most comment systems fail, but there are lots of reasons why. Here are a few that I could think of:

• Ignorance - there will always be people who are outrageously closed minded or blatantly incorrect. I can't tell you the number of times I have wanted to slam my head against the wall when reading comments on mainstream and local news sites alike. A system needs to account for this and prevent these types of comments from dominating.
• Group think - It is hard for groups of people to tell the difference between "wrong opinion" and "other opinion." This means that systems have to prevent the larger group from taking over the conversation and squelching valid alternate points of view.
• Information overload - Once there are more than about 20 base-level posts it gets hard to take everything in. A good comment system will mitigate this effect and make sure that every useful contribution reaches somebody.
• Inability to respond - without conversation there is no way for individual ideas to mix and match. This means that users sit around isolated and unchallenged. As a rule of thumb, strictly linear layouts like the kind you see here on the IdeaLab won't work in settings with an average of more than four or five comments per item.

A Possible implementation
Like any other piece of technology, there is no "best system" here. A local paper that has never had more than 3 responses on any article might not care much about implementing a robust collective intelligence suite. That being said, a good system on a decently trafficked site will get used.

I want to throw out a few ideas that might help the situation. Some are quite old, some are new, but none are used everywhere just yet.

• Threads - threads are standard on social media sites but they really ought to be standard everywhere. Users should always have the ability to reply to someone directly and have their replies appear in context. This naturally organizes information but it also helps to contain flame wars and hot debates that might otherwise take over and stifle new thoughts.
• Rating - it is difficult to find a perfectly pure way to rate subjective content such as comments. Despite this it is a worthwhile addition even if it is only used to get a general picture about where community stands on the issues.
• Randomization - you can't view all of the posts all of the time, but by including an element of randomness when deciding on which comments to display at the top a system gives a chance to the ones that would have otherwise been lost in the crowd. This will help to prevent the lazy voting feedback loop where the top posts are rated higher simply because they are seen by more people.
• Categorization - it is hard to compare comments. Some are informative, some are funny, some are insightful, some are insulting, etc. In order to account for this you can allow users to categorize them into predefined buckets. This could be combined with rating by having users select reasons why they rated something up or down.
• Burial - some comments simply aren't fit for print. Automatically hiding terrible comments helps clear away some of the noise, but how can you identify them? Ratings may be prone to group think but through categorization it becomes possible to see the difference between bad and unpopular.
• Geotagging - I haven't mentioned geotagging in a while, but comments should absolutely be linked to location. This provides some degree of context to individual statements and also gives a sense of scope when looking at conversations. Furthermore you can apply some of the targeting practices that I've talked about in the past in situations of information overload.

Now that I've said all that I charge you with the task of going out and fixing the way comments work on your sites. Just let me know if you need any help!

Obviously the game must be fun, but what makes it dangerously addictive is that the more you play the more you can do and the better you can do it. The result is an incredibly unproductive feedback loop. This post is about how to apply this formula to digital media in the hopes that it will improve participatory processes and maybe even offer some help when it comes to identifying journalists and citizens online.

The Idea: utilizing experience in a media conversation
We should all know by now that regular users of a media system need an internally provided sense of ownership over whatever content they produce. Really, though, the site should actually give a little bit back to the people who are providing free services and creating free content.

Kudos isn't really that satisfying, karma only goes so far, and cash is probably in short supply, so what's a system designer/local paper to do? Well, if we grant the owners of quality contributions slightly extra clout we simultaneously tap into the WoW feedback loop, reward the contributor, and use archived contributions in a way that actually provides added meaning to the current conversation.

Below are a few traits that the system could recognize along with examples of how such recognition might work. The examples use the context of the reader-driven media repository that I've been blogging about for oh so long.

1. Recognize quantity - The more a user ranks and judges new content, the more potent that user's vote could be. For instance, after voting on 100 articles that person's vote could count as 1.01 votes in future. As always, Newgrounds.com uses a system like this.
2. Recognize accuracy - The more accurate a user's judgment is when categorizing new content the more sway they could have over the categorization process in future.
3. Recognize quality - If a user has a track record of submitting valid journalism articles, maybe it could be slightly easier for them to submit future journalistic articles.
4. Recognize wisdom - If a user's contributions are regularly judged to particularly insightful or even just accurately reflect the attitude of a community then their future observations could have slightly more prominence.
5. Recognize roles- As a user performs acts that fit the role of a good journalist or good citizen the system will slowly start to associate their digital identity with these social roles. [More on this below]

There are a few risks you need to keep in mind: make sure the system isn't overcomplicated, make sure the rewards don't get in the way of journalistic ideals, and make sure users can't ever get unfairly powerful. These rules can be easily addressed by limiting the amount a person can grow in a given day and making sure that the rewards are not so influential that they could come close to dominating the system's natural order.

A Solved Problem: Identifying journalists and citizens
A long time ago I begged and pleaded for the industry to come up with a solution for developers who want an elegant way to identify journalists online. Although there were some great ideas in the comments, nobody went out and spent all their time working to fix the problem for me. Needless to say I'm heartbroken.

I have since decided that the best way to think of journalists and citizens on a digital system is in terms of their roles. As far as I'm concerned, if you act like a journalist online and you do a good job of it then you're a journalist; same goes for being a citizen. With a system that can recognize and analyze the quality and nature of a user's participation it becomes possible to identify journalists and citizens internally.

Depending on how you want to take it from here you can grant slight privileges as users become more developed in a particular role. Maybe system-recognized journalists will have a slightly easier time getting their content approved; maybe system-recognized citizens will be able to have more power over what items are considered more important to the community.

Final Thoughts and an Example Story
Imagine a user who uploads a personally written piece about the recent debate surrounding local zoning laws. By the time the article goes through the system it seems that community members feel the author got the facts right, offered great insight and fair analysis. The system and the community now have that information in their memory banks - they know that this person has produced one piece of quality journalism.

The author has this feedback and feels warm and fuzzy, and assuming the article quality wasn't a fluke there will be more good ones over time. Eventually the system sees that about 94% of this person's 30 submissions are flat out great and comes to a brilliant deduction: this user is a journalist.

We also have another user (or the same user) who enjoys perusing the purgatory section, which contains upcoming and un-vetted news about the local area and personalized interests. The person has judged about 500 articles over the months and based on the statistics he seems to have a pretty good eye for quality.

60% of the time he correctly identified good articles and 95% of the time he correctly identified bad articles. This user has also helped call out 20 pieces of content that went on to be incredibly well received by his community. It would seem he is pretty good at identifying community agenda. Egads, it's a citizen!

I'll close this post by saying that even if you don't care about role classification, leveraging a user's history can make your site more intelligent, more responsive, and more rewarding for everyone involved, so go figure out how to fit it in.

(This post pertains to a bullet point from Tying it All Together - User roles)

This post is about how the news system I've been blogging about can be driven by user generated content and collective intelligence. In a larger sense, however, it is about the way in which any news organization can make the move past the one-sided "audience" view of things and incorporate the voices and minds of its readers to better serve the public.

Collecting content
Users can contribute all sorts of information to a website (assuming they are willing). Understanding the nature of these potential contributions will help news organizations creatively incorporate them into current processes.

• Original reports - The system will provide a platform for journalists and citizens alike to add important, researched dialogue to the community agenda. This type of user generated content is meaningful, but it is also a way to broaden the conversation and guarantee that all issues are touched upon. The essential and difficult task of ensuring quality in original reporting will largely rely on the critical ability of the readers and is discussed later in this post.
• Opinion pieces - People will voice their opinions about issues in a way that contributes something valuable to everyone... or they will just want to rant. Either way these views make it possible for interested readers to get a sense of the overall attitude surrounding any given thread of conversation and potentially respond.
• Summary pieces - Readers can be passionate about information; maybe they are into politics, maybe they read everything they can about computer hardware, or maybe they are following all of the latest developments in local environmental policies. Regardless of the niche, there is probably somebody out there who is learning everything they can. Whoever they are, they can summarize the story and save time for those who aren't willing to find and read it all by hand.
• Facts and data - This is the category for first person accounts, observations, statistics, hearsay, or any other piece of information. Standalone the content might not be robust enough to be a story or an opinion piece, but these things could add value to an existing report or inspire a journalist or citizen to dig deeper.
• External content - There will already be relevant knowledge out there. By allowing users to suggest links of interest and linking to or otherwise ethically referencing that content the site helps people access the information they want. (Think Digg)

Keep in mind that the person making a contribution generally can't be trusted to know to which category it falls! In particular, I'm sure we have all made the mistake of presenting/believing opinion as fact.

Eliminating the bad
The news is a domain where informational integrity is incredibly important. In English 101 at Carnegie Mellon I read a few papers by Hadley Cantril about the idea of Critical Ability (the individual's ability to critically analyze new information before deciding whether or not to accept it). The moderation processes I propose will rely heavily on collective critical ability and will use computers to help increase the odds by providing some useful data.

• Technique 1: Purgatory - New articles of any type will start in a section of the site dedicated to unchecked information. This content will not be 'elevated' to the mainstream area until it has been collectively rated and categorized, and meets a certain quality threshold. By placing content here the users' critical abilities will be explicitly triggered, they will be reading the content specifically to judge it. [A previous post on this subject]
• Technique 2: Context - The system's tagging process will make it possible to display potentially related articles for curious readers. During article purgatory this will help inform critical ability; a lone report about a huge explosion in Montana might not be credible, but seeing that there are 500 of them alongside links to a breaking story from the AP would make the piece much more believable.
• Technique 3: User history - Has the user contributed anything in the past? What is the average quality of those contributions? Has the user tended to write opinion or report pieces? The system can provide this information to readers, once again in the name of empowering critical ability.
• Technique 4: Intelligent systems - Spam is automatically caught by mail and forum filters all the time. Although our situation will still require human input, the system could flag particularly suspicious-looking or particularly good-looking content in order to help guide purgatory readers.
• Technique 5: Targeted moderation - Since people will define topical and geographic interests, new articles can be targeted during the moderation process. This would mean that Philadelphians would have higher clout when judging a story that is relevant to Philadelphia and that those who like nanotechnology would be more trusted to review the latest report on the nano-bot 5000. [A previous post on this subject]

In general when there is collective doubt, the system will be pessimistic; this is necessary because the consequences of misinformation being presented as news are incredibly dire.

Delivering the good
When all is said and done, after the people have spoken, and [insert other terribly cliché phrase I can't think of right now], the judged and categorized content will be displayed for everyone to see. By now the article will be labeled as an opinion piece, or a report, or a summary, etc. It will also have some numerical measure of quality based on the user feedback so far. Finally, it will have the appropriate tags and metadata associated with it, since those who have looked at it so far would have been able to suggest changes to these things.

The process won't be over, of course. People will continue to rank and rate. They will also be able to flag articles for "moderation," which would entail a harsher and more targeted version of the initial judgment process.

This probably all sounds like a lot to ask of Joe User, but it actually isn't so bad. It will just involve spending a minute or two reading an amusingly bad or refreshingly good article about a topic that is likely to have been targeted (i.e. of interest) to them. Combine that with the incentive mechanisms I mentioned in my last post and we should have a fully functional process for facilitating and moderating a reader driven news agenda.

(This post pertains to a bullet point from Tying it All Together - User aggregated/moderated content)

Steve is right; a developer can't rely on user contribution unless he/she is sure users will contribute. Since one can never actually be sure about that, we are left with three simple tasks: hedge our bets, up our odds, and cross our fingers.

First things first, how exactly is this system relying on its users?

• Generating news content - the site will provide a platform for bloggers, citizen journalists, and professional journalists to distribute their work to a globally targeted audience. If that platform isn't used (i.e. users don't contribute opinion pieces, personal accounts, etc.) then that makes for a bored audience, and eventually, no audience at all.
• Performing moderation - democratic moderation is scalable and seems ideal for a community oriented system. Unfortunately, as you always hear around election time, democracy doesn't work unless people participate.
• Assigning metadata - machines will take the first stab at tagging new content, but it won't be effective (or accurate) unless people double check the computer's work.
• Participating in social process - social processes can't be facilitated without an interested society to use them. If users don't use the tools provided for conversation, activism planning, polling, etc. then it's as if the tools weren't provided in the first place.

Tactic 1: Collect a user-independent content base
Computers may not have the critical thinking capabilities of a human, but they are a lot more reliable. With that in mind there are a few ways we can use computers to create bare minimum levels of content and process. This way even if nobody else helps out at least the system offers something worthwhile.

Since a lot of relevant news is already on the Internet there is no reason we can't take a page out of Adrian Holovaty's book and start pulling it together automatically. This means that stories and articles from external blogs and news sources could be found, tagged, and linked to without the users having to lift a finger. This publicly displayed body should probably be limited to a short blurb rather than a complete mirror (for legal and ethical reasons), but the users would be pointed to the original source.

All types of external news content would be important for the global scale system I have in mind. Local news sites like a city paper, however, would want to link to contextually relevant content such as local blogs or peer analysis of local stories. These papers would also have the benefit of having paid staff doing original reporting; some articles are being generated in house already. In either case the system gets a nice content base before user contributions even start to enter the picture.

Tactic 2: Provide internal incentives
When thinking about a user's decision to participate on a website I go back to standard economics: if the benefits of contributing outweigh the costs then the site gets the much needed activity, otherwise the user goes his or her merry way. Even if we minimize the cost by providing a great user interface, there has to be something to entice the user. Below are a few examples.

• The megaphone effect - for some people simply being able to have a say in a public agenda is enough (i.e. "my comment will shape things" or "my vote will help decide the future of this post"). Larger sites have more voices to drown out the individual, but at the same time there are more people who will potentially see the results.
• Visible recognition - all user contributions must be formally recognized by the system in a way that the individual can see. This might be in the form of statistics, rankings, karma points, or personalized archives. Whatever the mechanism, the important part is that the user is acknowledged for providing time, thought, and content to the site. Without this, users will think of their contributions as just being dust in the wind...
• Gaining status - the system can take recognition to the next step by providing small amounts of earned status as a reward for quality involvement. Newgrounds is an example of a site that does this, giving people slightly more power the more they vote and providing access to "secrets" for more active users. I have heard warnings not to take this too far, since you don't want people to "game" the system.
• Potential for feedback - when I comment on Digg I spend the next day or so checking my profile to see what sort of response the community has given to my incredibly witty comments. (Digg is particularly brilliant because this feedback also serves as a moderation tool.) The potential to see how your perspective compares with the group's is often an attractive reward for contributing in itself.

You will notice I ignored money. This omission isn't necessarily a surprise, but I want to point it out since money might eventually be an attractive/necessary incentive technique where user contributions are on the scale of being entire articles. Newgrounds, for instance, offers a $500 prize every month to reward the creator of the best new animation.

Tactic 3: Personalize the user experience
There is a surprisingly accurate observation that in any particular social system 80% of the content is generated by 20% of the users. In other words, a few people are doing most of the talking/work. Intuition would imply that the 20% comes from the people who care enough to pass the "participation threshold."

In an earlier post I talked about how tagging can be used to better connect users to content and community. What if the system helped bring every user to the communities and content that they genuinely care about? This might be a local community, a professional community, or something else. My theory is that everyone would become a part of at least one "20%" - drastically increasing the overall site's participation.

The first tactic alone will result in a site with content. Providing incentives and personalization will take things to the next level by prompting users to help shape the public agenda. If you look at some of the more successful social news sites (Digg, The Huffington Post, Slashdot, etc.) I'm sure you will see most if not all of these trends. The next step is to take the personpower and use it to make sure the conversation doesn't devolve into spam, misinformation, flame wars, and overall noise.

(This post pertains to a bullet point from Tying it All Together - User aggregated/moderated content)

Step 1: Assigning Tags
This system will have tags for location, topic, and community, but where will they come from? Relying on people to some extent isn't the end of the world so long as you are clever about it; for instance, CMU professor Luis Van Ahn turned image tagging into a game. Nevertheless, it is very important to automate as much of the legwork as possible. With this in mind I see tagging coming from four places:

• The Content - When new content is added, the program will extract and analyze any clues buried within. For a news story this might involve phrases and terms that are parsed out of the body; for instance "yinz" could be recognized as Pitssburgh-ese or maybe the author mentions a location. There are plenty of algorithms that can make predictions based on this type of data extraction.
• The Context - All new content is being added by someone or something, be it a person or a feed from a website. That content provider has contextual metadata (i.e. a history); for instance the system could get hints from the tags of previous submissions. If the source is a user the system could take that user's specified areas of interest under account. Finally, if the story is being submitted as a response to existing content then it probably shares a lot of the same tags.
• The Author - Once the system makes its predictions the author will have an opportunity to fix mistakes. This means that he/she can add or remove metadata to the new content before it is officially submitted. It is very important that there be an opportunity for this to happen, although it is also important that the system does not rely on these corrections - the author might make mistakes or simply skip this step.
• The Swarm - I've come to trust the swarm effect, meaning I believe that a system can correct itself so long as every user can suggest changes in a non-obtrusive say (and they have motivation to take advantage of this ability). If the average reader is able to push for the addition or removal of tags this would mean that even if the initial tagging was wrong it would be corrected eventually.

None of these tag sources are failsafe, but since the swarm will correct everything over time the main concern is abuse. Some ideas to combat this: after the submission process no one user would be able to single-handedly add a tag, making it far more difficult to spoof tags or insert irrelevant tags. Also, different weights can be given to individual voices depending on their past interactions with the system (more on this in future posts).

Step 2: Getting User Preferences
This database of tagged content can be used as a way for people to find information through browsing and searching, but once again the system should do as much of the grunt work as possible. In fact, it should actively distribute targeted content to the users who would be most interested without forcing them to dig around.

Before any targeting can happen the system needs to know what the users are interested in. In order to do this it needs to somehow get the user to share what he or she cares about. I think that a lot of people find it creepy when computers start trying to guess their preferences based on what they look at, so here are a few alternate techniques:

• Blunt Elicitation - Asking the user for a lot of information when they join is a terrible idea; the more that users have to do when signing up, the more likely they will get bored and give up. However, you can still get the information, just defer it. Have a section of their profile dedicated to what locations/communities/topics they are interested in or identify with.
• Collecting Hints - It's hard for a person to sit down and exhaustively list all of his or her interests. In order to account for this the system can collect information over time by asking for feedback about the content they read. Simply asking "Was this interesting?" could yield a lot of information to help the system better serve its audience.

Users will, at some point, specify physical regions of interest. I'm picturing a visual interface where they drag circles or boxes around the locations they care about. They will also identify communities of interest by picking from a list of the communities contained in the areas they drew in the previous step (or from a lengthy universal list). Finally, users can associate topics of interest generally or specific to a particular location or community.

Step 3: Targeting and Adding Meaning
The result of all this tagging and preference elicitation is a semantic (i.e. programmer-friendly) bond between content, people, and communities. The obvious worth of this is that content can now be effectively routed to those who care. The data can also be used in more creative ways; here are a few ideas:

• Facilitate Social Process - One of the most powerful parts of this concept is that the tagged content doesn't just have to be news articles or blog posts. It could be polls, conversations, events, protests, advertisements, classifieds, real estate listings, etc. In other words, this structure makes it possible to add meaning through localized conversation and collective insight. (It also opens up a potentially effective business model).
• Connecting Related Content - Locations are inherently related, as are common interests. Having a detailed spectrum of tags makes it more likely that users will see related stories next to one another. By allowing them to explicitly identify these relationships the system can, in the words of Ben Melançon, lower the signal-to-noise ratio even more for readers interested in a particular thread of news.
• Identifying Trends - Being able to see comparative trends between topic, location, and community could be incredibly interesting. For instance, what locations in the United States are written about most often in Environmental news? Isn't it interesting that almost 60% of the users who have an interest in California also care about the topic of avocados? Why is there so much educational news in Philadelphia? You get the idea.

Like always, these ideas don't have to be applied to a global system; local news organizations could easily use them to improve their website. Some of it might be overkill depending on the amount of content put out by these groups, but that's why the next stop on the 'system design' train is a focus on collecting and generating content.

(This post pertains to a bullet point from Tying it All Together - Geotagging)

System Elements

• Geotagging - by tagging content to physical location it is possible to personalize it without losing the benefits of contextualization. The tagging can be done through intelligent algorithms and/or by humans during the content submission process. Consumers can then define regions of interest and receive a customized information feed.

• User aggregated/moderated content - the content on this system will be provided by users and democratically moderated. This means that although anybody will be able to propose new content through submission, it will only become a part of the media conversation if the members of the relevant communities deem it worthy.

• User roles - we hold these truths to be self evident that all users are created equal, but that doesn't mean that everyone will do the same thing. Users will naturally gain status within the role that they tend to fill. This means that journalists and citizens in real life will naturally become journalists and citizens on the system as they gain more clout within their areas of expertise.

• Incorporating multimedia - multimedia can always add meaning to a story and transparency is a vital part of trust. Being able to see the pictures, watch the videos, read the documents, see the full interview transcript, and hear the audio clips that informed the author of an article will enable the critical ability of a reader. It also makes it possible for a new reporter with no track record on the system to convincingly break a story. Mobile technology will allow users to have content uploaded directly to their accounts.
• Globally local scope - this system will contain a global array of content that is collected and presented at a local level. This method of organization makes it possible to adjust the granularity of any type of information and present different content to each user or group as appropriate.

Final Thoughts

When someone creates an account on this system they will have the potential to start, participate in, or simply observe the living breathing media conversations that are going on in their communities. Best of all, I think that it will play well with the industry. The moderation process will naturally reward quality journalism and properly identify opinion pieces as separate from genuine reporting. Just as important, though, is that it will complement rather than overthrow existing local media. I suppose this last point is something worth dedicating a post to as well!

Defining the Pieces
Before diving into roles, I want to draw a line between two different (yet inherently related) components of a media conversation:

• The agenda - what is being talked about? Individuals have a personal agenda, communities have a collective agenda, and reality imposes an agenda of its own.
  
• The information - what is being said? Any story involves facts and data, most topics inspire opinion and commentary, and information gains more meaning with additional perspective and context.

Obviously these are directly related concepts but they don't always go hand in hand. For instance, when an article supports comments it allows the public to contribute information, but the author already set the agenda. On the other hand, when someone Diggs an article or blog, they are helping to define an agenda without having produced the information.

Identifying Constants
So here we are with our tasks in front of us. Between the journalists and the citizens we need to define an agenda and present information. There are a few things that are true regardless of what we do (preconditions) so let's use that to figure out what needs to be done by the end of the day.

• There is a reality - something is actually happening; the information (as defined above) needs to be reported accurately.
• There is a community agenda - community members collectively care about certain issues and events; those things should all be addressed.
• There is a "service" agenda - there are some topics that communities should care about even if they don't actively demand it. This might be for the sake of "betterment of mankind" or maybe the story just fell under the collective radar; either way those issues need to be brought up.

Those bullets essentially say "we need to account for everything," but I'd be lying if I suggested otherwise. The good news is that nobody has to do it alone. If we recognize and facilitate the symbiotic (and overlapping) roles of citizens and journalists then we'll have a good shot.

Note: personal agenda is out of the scope for this discussion; people will read the things they are interested and ignore the rest. Helping users find personalized information is a topic for another time.

Classifying the Roles
It's finally time to divvy up the roles as I see them. Here goes nothing!

The Citizen...

• Identifies issues - each citizen will care about specific issues on their own. These individual contributions may or may not resonate with the community.
• Provides a community voice - the community is an aggregate of individuals. It only makes sense that these individuals will naturally shift the agenda and provide important information through their collective contributions.
• Adds personal commentary - each person will have a unique perspective and will be able to add commentary in the form of opinions, experiences, and observations.
• Is a part time journalist - (see below).

The Journalist...

• Is a Watch dog - when important issues go ignored the journalist is there to make some noise about it. This includes issues that fall under the radar, but also includes the "ideal agenda" that I mentioned before.
• Researches the facts - it takes hard work and time to find out what is going on. There is no room for opinions when facts are involved.
• Provides contextual commentary - events are rarely isolated and there is often a bigger picture behind the immediate. It takes someone who has made an effort to know the ins and outs of a given topic to be able to insightfully contextualize a story.
• Is a part time citizen - (see above).

Final Thoughts
The last bullets of each section are the most important, but even if you removed them you would see that agenda definition and information provision is divided between the citizen and the journalist. Put the bullets back and the divide between roles stay well defined, but now the individual gets to be fuzzy. In other words, there are some citizens who report and there are some reporters who voice opinions.

Some might raise their eyebrows at this. In particular, I know that professional journalists are expected to keep their opinions out of things; whenever a person is taking on the role of a journalist they absolutely should not let opinions alter facts and context. As individuals, though, everyone has different hats. So long as the hat being worn matches the actions being taken, there really shouldn't be a problem.

Between the two roles everything gets accounted for. The citizen influence pushes towards an accurate agenda, the journalistic influence makes sure no stone goes unturned, and between the two all the information (opinions, facts, and context) gets put onto the table for all to consume. There are other issues to be considered, such as product quality, and ability to differentiate hats, but I'll postpone that conversation for now.

After those last issues are addressed, we can apply this understanding into a concrete system so that needs of the journalist citizen, the citizen journalist, and the simple media consumer are met. The result will be glorious, and assuming I've managed to explain it correctly, I hope that it is something that everyone can agree upon.

This post is about where I think news organizations, especially local news organizations, need to take their digital presence. This is the conversation I hoped to seed with my analysis of the Anonymous activism against Scientology. It also turns out that this post will work nicely with the recent conversation on this blog about the need for news organizations to change the way they operate online.

Context and Clarification
In my posts about Anonymous I tried to identify some of the subtleties of online community coordination and pull out any lessons that could help us in our journalism-industry-wide quest to effectively utilize digital technology.

Based on a few of the comments to those posts it seems there was a little room for confusion. Some thought I was trying to provide a recipe for media outlets to take advantage of existing online communities or artificially manipulate the masses. In other words, not everyone understood what I believe the technologies should be utilized for. In a comment I wrote:

"[In these posts] I tried to look at what might have been a reason for [Anonymous] success and largely cite the fact that physical communities don't utilize the kinds of digital communication tools that you guys have. This is where (for instance) local newspapers, which are desperately trying to find their place on the internet, could fill a role. Not for profit, but instead to get back to the public service that they were supposed to be providing in the first place - an outlet for community voice and an amplification of community issues."

From what I understand, some of the original driving forces that inspired local news media were the demand for outlets of community voice and the need for amplification of important community issues. Ethics, practices, role, and tradition - i.e. hard news, public service journalism (which I will refer to as "hard journalism" from now on) - grew over time.

By focusing on those initial demands and drawing on "hard journalism" practices for reinforcement rather than direction, our adaptation to a new medium will hopefully becomes a little more manageable. That focus is what I wanted to develop with those posts (plus the whole Anonymous effort continues to fascinate me).

A New Community Medium
If my interpretation above is even partially accurate, it seems that local news operations are supposed to be information hubs for the communities they serve. When using a one-to-many medium such as Television or Print, reporters and editors try to represent their community by proxy. For old media that was fine because, realistically, it was the only way for the job to be done.

With digital media, as everyone seems to have figured out years ago, it isn't enough to just have an online newspaper. What people are realizing now is that it also isn't enough to simply enable comments, publish the occasional user-submitted-photo or blog, or incorporate a few pieces of interactive content. All of these things are small steps in the right direction, but small steps are slow and costly in the world of software.

This time around news organizations need to do more than just learn to use the media, they need to host a community with it - an idea that Richard Anderson put out there in his first post to this blog. People want a place, digital or otherwise, where they can gather and learn about the community in which they are a part, a place where they can get in touch with the issues, and a place where they can pick up on the "vibe." They want a modern Roman Forum.

If news orgs don't provide this then someone else will. What is troublesome is that the "someone else" won't necessarily incorporate hard journalism in their vision. What makes THAT troublesome is that such services directly compete with the news.

Facilitating Community Agenda
In the words of Paul Monaco, much of media's social influence comes from its ability to set agendas, not by "[telling] its readers and viewers what to think so much as it points them toward what to think about." Social Media, Digital Media, many-to-many conversations, and all those other phrases that are thrown around describe the tools being used to push that task of issue definition back to the community.

For news organizations moving online this may seem like a conflict of interest but in reality the situation isn't so terrible. All it means is that news organizations need to move past agenda definition and take on the job of agenda facilitation. In other words, they don't have to work to reflect the community any more, now they can provide the communication tools needed to let the community reflect itself.

To put it in slightly different terms - I'm suggesting that local news sites facilitate "bottom up" agenda. The individuals (the bottom) can suggest issues and those issues may be picked up, prompting natural investigative attention and a swarm response. Anonymous shows that existing communities can do this if they have the tools. They also showed what can happen when a real community issue is met with effective information technology (for interested parties, there was another round of protests on March 15th).

As I said earlier in my quoted response, one reason physical communities are suffering in the digital world right now is that they don't have the online hub set up for them. This is where local media (and global media too) should step in and start providing.

Incorporating Journalism
So far I have slighted an incredibly important part of the picture: the journalists. Pushing agenda definition onto the community does not mean we can forget about hard journalism. Communities still need the people who find the stories that have fallen under the radar, spend weeks researching the details, and double check the facts. Journalists should focus on what they have been trained to do best and news organizations need to invest in them or we will lose an invaluable part of the process.

I am suggesting we use the credibility of journalism to augment the democracy of social media and inform the interests of existing community. Paid professionals would still report on new issues to see if their community takes interest, but now the digital version of the physical community they serve can act as a living breathing tip-line.

Final Thoughts
This has been quite a mouthful and I probably I tried to say too much in one place. For those in Anonymous who are reading this (I'd be surprised if you made it this far down) I hope the topic was at least somewhat interesting to you. I would like to know what you think about "hard journalism" - and I don't mean the mainstream gossip you see on network news. I ask the same question to journalists concerning collectively defined agenda.

I have mentioned in the past that I want to see digital media facilitate local impact; to do that well we need to understand some of the nuances of many-to-many digital communication and look at how those nuances might change the way communities can plan, organize, and ultimately act on the issues they find important. This post lists a few traits of online communication and what they might mean for digitally driven movements, including the one being led by Anonymous.

Setting the Stage
I think it's worth making sure we're all on the same page since "digital communication" refers to a whole lot of things. In this post I'm talking about tools like forums, wikis, chat rooms, community systems, and larger scale systems such as Digg or YouTube. Ok, so maybe that didn't narrow it down much, but really any system that facilitates community communication and supports multiple conversations among multiple people can be a potential platform for organization.

So how does community organization using digital systems (I.S. Activism) compare to organization through more traditional means? Here is a far from comprehensive list; hopefully it gives some food for thought.

• Lower barrier to entry - it is a lot easier for an interested party to visit a web site than it is for them to travel to a meeting; since online involvement is less costly, people who have access to the community system are more likely to learn more about an issue and potentially get involved.
• Less commitment - these days digital has tended to mean impersonal. That may not always be the case, but whenever it is people won't have as many external/social pressures to stick around. It also suggests that comments and actions are more genuine.
• More efficient - Well implemented information systems help create knowledge. In this case help comes via lowered individual lookup costs, computer driven collective intelligence, and more minds solving more problems in a naturally coordinated format.
• Less controlled - If the system is democratic (note the "if") then community members, rather than community leaders, ultimately control the conversations and decide what issues are most important. Unfortunately, depending on the level of community openness, this means that it may be easier for malicious contributors to cause problems.

Those bullet points are all well and good, but I want to call out the most important point of all: I.S. activism relies 100% on communication. If the channels of communication are clogged with irrelevant noise, if the overall direction is derailed into uselessness, or if the system itself is compromised in some other way, the movement dies. Basically, if people rely on a system to communicate and organize, then that system had better work well.

Success and Failure
Traits aside, how will these factors come into play in deciding whether or not a tech driven movement ends up being a success or failure? I have thought of a few general pitfalls as well as some reasons why tech driven movements are something that our industry should strive to facilitate within the communities we serve.

[Boon] Worthwhile efforts naturally resonate and grow - Lower barrier to entry and the ability to provide information effectively means that movements surrounding community issues will be able pick up steam much more quickly through digital media. As community members learn more and care more, they will think about the issues and ultimately contribute to the conversation. If the system is worth a darn then the collective conversation will lead to collective action as thoughts turn into plans.

[Pitfall] More steam means more noise - Growth is good, until things get too big. In a traditional movement if an unexpectedly large number of people get involved the worst that would happen is that the extra resources would go unutilized - we don't have enough fliers to pass out; there isn't enough space in the meeting room; etc. In an online setting, having 10 times more people than your system can support means that nobody will be able to keep track of what is being said and therefore nothing will get done. This is where scalability and system quality come into play.

[Boon] Diverse ideas and cross pollination breed strength - Online settings give the concept of "many voices discussing many topics" a fighting chance, so multiple perspectives within the community will advise each community decision. Ideas and input from other communities can also be brought into the mix since the digital medium makes that type of cross posting and content sharing so easy. In the comments of my previous post DevNul also identified this as a way to prevent "group think" and in my opinion he/she has this absolutely right.

[Pitfall] Unclear direction results in stagnancy - Leaderless or weak-led movements could fizzle and amount to nothing without a well defined direction. Imagine how many millions of people join Facebook groups about the issues that they supposedly care about. Now think about how much activism those groups have triggered - hint: practically none. In this case the problem stems from the fact that Facebook is a terrible platform for group communication (yeah I said it.) With other tools the opposite problem might be true; there is so much communication that without a "leading voice" it is more difficult to move the masses in a coherent way.

Final Thoughts
For members of Anonymous I'm betting most of these things are already unspoken understandings. They have been using Internet tools for a while and their solutions for several digital coordination issues are tacit at this point, often in the form of memes (look up the phrase "tl;dr"). Visit the Enturbulation forums and you can see what they are doing well and what organizational problems they still face. For instance, forum moderators put important threads in more noticeable locations, yet there are plenty of individual missteps, plenty of accusations made between community members, and plenty of ways that the overall process could be made more effective.

I believe Anonymous has established their point well enough that their efforts won't stop in the near future, but keeping a high level of interest is vital for an online movement; genuine interest will always warrant genuine action. This reflects the way digital movements have changed how activism can work: participants are no longer just volunteers, now they define the issues and invent the direction.

To bring this post home I'll end with a final comment: I have a feeling a significant number of people learned about Anonymous' issues and the Enturbulation forums through some type of media. What if local news organizations could provide an outlet for the communities they serve to do what Anonymous did...?

We're not there yet, and it would take more than just setting up some forums, but as Richard Anderson said at the NAA conference last weekend, we are moving towards the business of community hosting. Facilitating, not leading, community organization for impact is exactly the kind of goal that makes that role meaningful.