Two recent developments suggest that we're starting to find answers to that question -- and more importantly, that there's a growing number of people trying to find these answers. Duke University has released an interesting report, and a group of journalists and technologists has begun meeting in Silicon Valley to address challenges that journalists and technologists might tackle together.

The February 2008 conference at Georgia Tech, entitled "Journalism 3G: The Future of Technology in the Field," introduced many of its 200+ attendees to the idea of computational journalism -- applying computer programming to the challenges facing journalism, journalists and a society that needs original reporting to provide information for citizens in a democracy. Two of the other attendees were the first Knight News Challenge "programmer-journalist" scholarship winners: computer programmers enrolled in the master's program at the Medill School of Journalism at Northwestern University.

When the John S. and James L. Knight Foundation awarded the scholarship grant to Medill in 2007, the idea of teaching journalism to technology professionals seemed odd to many people -- both journalists and technologists. But now there seem to be a lot of initiatives aimed at addressing the same set of issues.

Duke University, through its DeWitt Wallace Center for Media and Democracy, built on the ideas generated by the Georgia Tech conference in a couple of ways. First, the center created -- and has now filled -- a faculty position specializing in the field. The new Knight professor of the practice of journalism and public policy is an old friend, Sarah Cohen, previously database editor for The Washington Post, where she contributed to countless enterprise reporting projects, including a Pulitzer-winning investigation of child welfare agencies in the District of Columbia. Besides teaching courses, Cohen is expected to lead the development of open-source reporting tools designed to make it easier for journalists to discover and research stories.

Earlier this month, Duke released "Accountability Through Algorithm: Developing the Field of Computational Journalism," a report based on a workshop held in July. The report is full of interesting ideas for applying technology to journalists' challenges. Here are a few of them.

Information Extraction, Integration and Visualization

A new set of tools would help reporters find patterns in otherwise unstructured or unsearchable information. For instance, the Obama administration posted letters from dozens of interest groups providing advice on issues, but the letters were not searchable. A text-extraction tool would allow reporters to feed PDF documents into a Web service and return a version that could be indexed and searched. The software might also make it easy to tag documents with metadata such as people's names, places and dates. Another idea is to improve automatic transcription software for audio and video files, often available (but not transcribed) for government meetings and many court hearings.

The report also suggests developing "lightweight" templates that enable journalists to create data visualizations based on XML or spreadsheet files, and tools that help them organize their findings in a timeline. As the report points out, reporters working on in-depth projects often create chronologies in lengthy spreadsheets or text documents. A better tool would let journalists "zoom in, tag events for publication, turn on and off players or events and otherwise use them effectively," the report says.

The Journalist's Dashboard

Here the Duke report suggests that journalists need "a tool with which to spot what's new and what's important in the flow of daily information." A dashboard could include:

• A news alert system similar to Google News that scanned only the sources specified by a beat reporter,identifying the originating publisher and the number of other sites that linked to the item;
• A tool helping journalists keep track of their sources, including news items about that person and citations from the reporter's own archived stories mentioning him or her;
• A "trends and outliers" tool that might generate an alert any time a data source reveals a significant change in a piece of data -- say, a surge in monthly expenditures by a government agency, or a flurry of crime reports in a short period of time.
• A timeline generator that would display incidents related to a particular story as well as coverage on blogs and news sites.
• An annotator that would allow a reporter to see past stories, images and contextual information while writing -- for instance, by displaying background information about the person being written about. (This idea bears some similarity to the EasyWriter tool developed this spring by students in a Northwestern University journalism/technology class.)

Reader-Reporter Interaction

Philip Bennett, formerly managing editor of the Washington Post and now a professor at Duke, is quoted in the report describing a new approach to investigative projects that engages and taps into reader interest. Instead of seeing long-term investigative projects ending with publication of a package of stories, the initial investigation could serve as just the midpoint in the reporting process. Stories could be presented in ways that enabled each reader to explore the story in layers, giving each a "differentiated news experience depending on her interests." Bennett suggests that a series like the Post's Pulitzer-winning investigation of Walter Reed Army Medical Center could have become a focal point for readers interested in veterans' issues. "If the paper could nurture a community of interest around the story, readers might use the site as a discussion place for the action that follows from the investigation," the report says.

Applying 'Sensemaking' Approaches From Other Fields

The Duke report points out that academic researchers are wrestling with many of the same challenges that journalists face and suggests that their solutions could be helpful. For instance, Georgia Tech researchers have built a tool called Jigsaw that creates visualizations to display connections between individuals and entities mentioned in different documents -- something every investigative reporter would lust for. And the Muninn Project, an interdisciplinary research project focusing on World War I records, is seeking to convert images of handwritten forms into machine-readable databases -- a problem faced by journalists in many states that allow political candidates to file handwritten campaign contribution reports..

Another new development worth taking note of: a new "Hacks and Hackers" Meetup group formed in Silicon Valley by former Associated Press foreign correspondent Burt Herman, who is on leave from the AP and recently completed a Knight fellowship at Stanford University. The group -- billed as being "for hackers exploring technologies to filter and visualize information, and for journalists who use technology to find and tell stories" -- held its first meeting Nov. 19.

The first gathering attracted about 30 people, including people from Google and Google News, Yahoo, sfgate.com, the San Francisco Chronicle, Current TV, PARC (Palo Alto Research Center), and Topix.com, Herman reported. "It felt like the seeds of a movement, and the many lively conversations showed that everyone was able to find common ground," he wrote in an email to me.

Herman said his Knight fellowship -- during which he focused on innovation and entrepreneurship -- taught him that innovation requires bringing people from different disciplines together.

"I started the Hacks and Hackers meetup group to open a broader dialogue between technologists and journalists, so we can move past the endless hand-wringing about the future of news and get down to work building it," Herman said. "Technology and media come together here in Silicon Valley like nowhere else in the world, and there was no group yet focused on this. I'm hoping it will lead to better understanding and perhaps even spawn new ventures."

As some readers of this blog will remember, "Hacks and Hackers" is also the name that Aron Pilhofer and I came up with to describe a new organization and Web site for people working at the intersection of technology and journalism. At the Future of News and Civic Media Conference in June, Aron and I won a $2,000 prize to create an online community for people with these interests.

The Web community idea is still in the early stages of development, but Aron and I would welcome your ideas about how best to make it work. The original concept was to create a place where members can seek help solving problems and provide assistance to their peers by, for instance, sharing a tutorial for a project using Django or Ruby on Rails or Drupal. We know there are people -- in journalism and technology, in industry and academia, scattered through organizations such as the Online News Association, Investigative Reporters and Editors and the Society for News Design -- who can use each other's help and support. We like the idea of having some kind of reputation management system -- say, like Stack Overflow -- that would reward members based on the quality and quantity of their contributions to the community.

If you have ideas for the Hacks and Hackers site, please post them in the comments below or email me at richgor - at - northwestern.edu.

Manya Gupta, a software engineer for telecommunications companies in her native India, is the fourth winner of a Knight News Challenge "programmer-journalist" scholarship. She's now in her second quarter studying journalism at the Medill School at Northwestern University. She blogs occasionally at http://manya-myvoice.blogspot.com/.

Learn some more about Manya from the following edited Q&A.

Tell us about your background.

I am from India. I received a bachelor's degree in electrical engineering from JSS Academy of Technical Education in Noida, Uttar Pradesh.. While working on projects I realized my passion for programming and decided to make it a career.

So, I moved to Bangalore, the Silicon Valley of India, to work as a software engineer for Infosys Technologies and worked in the telecommunications domain. Three years later I decided to move to Ordyn Technologies, a small company, to gain some startup experience. My stint at Ordyn as a senior design engineer was very fruitful, and among many things I learned Python.

But I am not a complete geek. I am a traveler, a big sports buff, a trained dancer and an avid reader. I love playing football, tennis and volleyball and won a best player award for football in a tournament in Infosys. 

How did you get interested in journalism?

Four years ago I participated in a national level anti-reservation protest. [Editor's note: Here's a BBC article about the protests and the policy change that spurred them.] It was then that I realized the power of journalism to effect change. I experienced, for the first time, the positive impact journalism can make in creating a better society. What started as a small protest by a group of students in the national capital soon turned into a youth movement and it was because of effective, strong and powerful journalism. The reach to the youth through different media was amazing. There were traditional sources like the television and newspapers, but there was Twitter and Orkut and Web images and blogs. So, there was this curious mix of new and old and everyone, with whatever means he could, was participating in the movement.

That experience stirred me. It made me want to take the plunge into journalism and explore the new avenues that appeal to today's youth -- because the whole idea is to get the message to them, and adapting to their tools is important.

What have you learned by studying journalism so far? How has the experience changed your outlook?

So far I have thoroughly enjoyed the Medill experience. First and most important, I have learned to report, write and think like a journalist. I look for a story in everything around me! But it is not just old-style writing that I have learned. Medill is a place where the old meets the young -- because with every print story I also created a multimedia piece and that is how I learned the importance of storytelling in the most effective manner.

Beyond that, I have met amazing people, participated in some very intriguing discussions and learned from people with tremendous amount of experience. What I have liked most is that everyone is so willing to share what they have learned.

Moreover, it has given me the opportunity to explore; by interacting with people from different walks of life, by understanding their problems, issues and lives, and by telling stories through creative media.

The experience has enriched me. It has given me the power to bring people's day-to-day issues to light. At the same time, I have learned not to tie my emotions to one side and be balanced and fair by listening to other points of view. In short, I have learned to walk the tightrope.

The software, the first version of which was developed this spring by a team of computer science and journalism students at Northwestern University, has evolved significantly since then. John Templon and Nick Allen (a "programmer-journalist" attending the Medill School of Journalism on a Knight News Challenge scholarship) were two of the students who worked on the initial version of the software, which has been made available on an open-source basis. John and Nick, both Medill grad students, developed the software with Tian Huang of Medill and Thu Cung, a student in the McCormick School of Engineering and Applied Science.

The software, then called "Machine Generated Sports Stories," was one of five projects developed in an experimental collaboration with the McCormick School's Intelligent Information Laboratory, or InfoLab. The class brought together students from Medill's Interactive Innovation Project and from McCormick's practicum in intelligent information systems. Two professors from Medill (me and Jeremy Gilbert) and two from McCormick (Kris Hammond and Larry Birnbaum) led the collaboration.

If you want to know more about the class and the software the students developed, you can read the class blog, watch the students' final presentation, or download their comprehensive report that includes recommendations for journalists, media companies and journalism education.

Since June, Nick and John have kept working on the baseball project as paid interns at the InfoLab. They've reconstructed the code, built a greater variety of game narratives and begun to incorporate details about trends in player and team performance over time.

Earlier this month, an article in Medill's alumni magazine brought StatsMonkey to the attention of a lot of journalists. A couple of them didn't like it:

• "Soon enough, sports reporters could be obsolete," wrote Andrew Greiner at NBCChicago.com.
• Rick Green of the Hartford Courant, asked, "... isn't something lost when the reporter isn't there at the games, talking to players, paying attention to what's not said and feeling the mood?"

These weren't the first journalists to express concerns about StatsMonkey. Back in August, Gregory Hardy of CBSSports.com worried about what might happen "if robot sportswriters take over."

Given the turmoil in the news business these days, it's understandable that journalists -- especially sports journalists -- would be nervous about StatsMonkey. But I don't think sportswriters need to be worried -- if StatsMonkey becomes a commercial product, it is highly unlikely to put sports journalists out of work.

To understand why, let's start by explaining what StatsMonkey actually does:

• It imports the box score and play-by-play information, information routinely captured for games in professional leagues, college baseball and some lower levels (high school, youth leagues, etc.).
• It uses some baseball-geek stats (leverage index and win probability added) to identify high-stakes at-bats and key plays that significantly change the probability that one team will win.
• It determines a game narrative -- for instance, a come-from-behind win, a pitcher's duel, etc. -- from these key at bats and plays
• It constructs a headline and story from the options for game narratives and incorporates key events from the play by play
• It uses historical data -- about teams and players -- to add context (for instance, that a particular player's hit broke a 5-game hitless streak, or that this was the team's third win in a rwo).

Of course, the program will have a limited number of possible game narratives, and it cannot account for events that don't show up in the box score or play by play (for instance, the infamous play in a 2003 Chicago Cubs playoff game in which a fan caught a foul ball that might otherwise have been fielded for an out). A StatsMonkey story will be a very poor substitute for richly textured narrative by a professional sportswriter.

But think of a few ways StatsMonkey could add to what professional journalists do:

• It could instantly write a game story as soon as the last out is made, freeing a reporter to go down to the field or the locker room to do interviews
• It could generate a story about any game in progress, at any point during the game -- just what someone might want when checking on a favorite team during the work day.
• It could create stories about games -- for instance, college baseball -- that are not routinely covered by professional journalists.
• It could generate stories about each player in a game for whatever people are especially interested in particular players (not hard to imagine for college baseball)
• If Little League coaches start to enter game information through a mobile device (and there already is at least one "app for that"), it could generate game stories about Little League games, which have a passionate following but will never be covered by professional journalists.

Beyond that, even given my background and identity as a journalist, I would have to say to any sportswriter: If your game story CAN be generated by a computer, at some point it WILL be generated by a computer. Human journalists will do -- and should do -- the kind of reporting and storytelling that computers can't.

Beyond that, StatsMonkey is just a first experiment in identifying formulaic stories could conceivably be generated by software rather than people. Some other possible examples: corporate earnings reports, obituaries, even accounts of what City Council did last night. As with StatsMonkey, software that generates these kinds of stories most likely wouldn't replace journalists. The software would create stories that would otherwise not be written, or free up journalists to do more important work that can only be done by humans.

Got any other ideas for topics that would be a good fit for computer-generated stories? Post in the comments below.

News Mixer, the "conversations around news" site created by a team of master's students including the first two programmer-journalists, is one of four finalists for a new prize: the Gannett Foundation Award for Technical Innovation in the Service of Digital Journalism. The site is in some pretty good company; the other finalists are two important startup companies (Attributor and Publish2) and the Associated Press (for its AP Mobile initiative). As one of the two professors who directed the project (along with my colleague Jeremy Gilbert), I am incredibly excited and proud of what our students (Brian Boyer, Ryan Mark, Angela Nitzke, Joshua Pollock, Stuart Tiffen, Kayla Webley) accomplished. There's some wonderful student work represented in the contest's two academic categories, but the screening judges are saying that News Mixer is in a different league -- competitive with significant professional and commercial ventures.

Meanwhile, ChangeTracker, a project developed by Boyer during his internship at Pro Publica, won a finalist spot in the "Outstanding Use of Digital Technologies, Small Site" category. The award winners will be announced at the Online News Association conference in San Francisco, Oct. 1-3.

The awards announcement gave me an excuse to catch up with the News Mixer students (who dubbed themselves Team Crunchberry) to see what they're up to now. Here's a rundown:

• Brian Boyer is news applications editor for the Chicago Tribune, where he leads a team responsible for creating news-based applications for the Tribune's Web site..
• Ryan Mark, the other programmer-journalist on the News Mixer team, works with Brian at the Tribune as a news application developer.
• Angela Nitzke is Web content associate at the University of Chicago Medical Center, where she edits and publishes content and is helping redesign its Web site.
• Joshua Pollock is an information technology project manager for Granite Telecommunications, where he's using project management skills he learned with News Mixer.
• Stuart Tiffen is working in Bonn, Germany, at Deutsche Welle, the international broadcast news organization, where he writes, edits, creates Flash galleries and manages the organization's Facebook presence.
• Kayla Webley recently completed a 15-nation Asia trip, including an internship with Time magazine in Hong Kong.

I was particularly excited to get updates from Angela and Joshua, both of whom told me their innovation experience at Medill played a critical role in helping them get their current jobs, and gave them skills and knowledge they are applying daily. Their experiences speak to the value of having journalism students collaborate with computer programmers; both Angela and Joshua clearly learned a lot from working on a software-development project at Medill.

It's noteworthy, I think, that the five jobs listed above did not exist before the digital era. The Crunchberry students' career trajectories demonstrate clearly that while there are fewer jobs in some journalism categories (for instance, newspaper and TV reporters), there are rich digital opportunities for students who have both journalism skills and an understanding of the interactive world.

Meanwhile, Brian and Ryan are showing through their work at the Tribune (such as searchable databases on pollution caused by dry-cleaning businesses and political influence in college admissions) that programmer-journalists can play a significant role in the reinvention of journalism for the digital age.

News Mixer was one of the first Web sites to take advantage of Facebook Connect to build an engaging social experience around news. It won praise from people interested in conversations around news and, more recently, was recognized by the Association for Education in Journalism & Mass Communication (AEJMC) and the Knight-Batten Awards for Innovation in Journalism.

News Mixer, though, was just a demonstration Web site, a prototype designed to show the potential for increasing engagement and improving the caliber of news conversations. This week, one of the top news sites on the Web -- the Huffington Post -- launched a new service (HuffPost Social News) that delivers on that potential. I've been playing around with Social News this week, and it's quite compelling.

As you can see in the screenshot I've posted on the right, now that I've registered for Social News, every page of huffingtonpost.com displays recent activity by my Facebook friends on the site. I can see what my friends are reading and the most popular stories among members of my Facebook network. I can also choose to highlight my Facebook friends' comments on each story, rather than see comments from every user.

Furthermore, as you can see from my second screenshot at the end of the article, each user's profile page looks very much like the one the News Mixer team created. It displays recent activity by your Facebook friends and a list of friends who have accounts on the Huffington Post.

Why is all this important? Because the biggest problem facing content creators today is what I call the "awareness gap." This refers to the fact that every piece of online content reaches only a fraction of the people who would find it interesting or relevant. On the Web, because the volume of content continues to grow dramatically, many others who might be interested in a particular piece of content never know that it exists.

How can we close the awareness gap? For a long time we've been waiting for better personalization engines to produce the "Daily Me" predicted in the 1990s by Nicholas Negroponte of MIT. The problem is that it's hard to build a good personalization algorithm. Just because I found one article on a topic interesting doesn't mean I want another one on the same topic. But I'm very likely to be interested in content my friends are interested in, in part because my friends and I inevitably share some content interests, and in part because I'm inclined to keep up with what my friends are interested in so we have a common converational foundation.

Filtering news based on my friends' interests does raise some troubling issues -- especially the risk that my friends and I will build an echo chamber in which we close ourselves off from information that challenges our preconceptions, or that relates to topics that are important but not interesting to us. But the more I look at HuffPo Social News, the more I agree with Chadwick Matlin of Slate's The Big Money that it is in part "the future of journalism."

One product is a tool for working reporters, one is a new way of organizing content for mobile delivery, two leverage the growing power of Twitter and one generates baseball game accounts from box scores. All of the projects demonstrate what's possible when journalists and technologists collaborate.

Details of the new concepts will start rolling out with a final presentation Wednesday to faculty, students and invited media industry guests. We'll be live-streaming the presentation. If you're in the Chicago area, come on by: 3:30-5:30 p.m. on Wednesday, June 10, in the Forum of the McCormick Tribune Center, 1870 Campus Drive.

The five projects are the results of the first-ever collaboration between a Medill "innovation project" class, taught by me and my Medill colleague Jeremy Gilbert, and a computer science class run by Kris Hammond and Larry Birnbaum, who run Northwestern's Intelligent Information Laboratory. Here are descriptions of the projects:

Tweedia

Tweedia is a widget that presents tweets relevant to an article and displays those results adjacent to the article. It is positioned as a tool that publishers can use to enhance engagement with their content. In its appearance and functionality, Tweedia bears some resemblance to the short-format comments (or "quips") included in the News Mixer demonstration site created by the fall 2008 innovation project at Medill. But while quips were posted by users on the same page where an article appeared, Tweedia aggregates tweets on a topic automatically.

EasyWriter

EasyWriter is a tool for journalists to use while crafting an article using Microsoft Word. As the journalist writes, he or she can highlight text and EasyWriter automatically displays news articles and Web pages related to that topic. The idea is to make it easy for the journalist to check facts or do additional research without leaving Word.

News Feed

News Feed is a mobile Web site, optimized for iPhone users, that is designed to help users choose just the right amount of information for the amount of time they have available: 5 minutes, 10 minutes or 20 minutes. This is the project that most directly addresses a consumer need, I think.

Twitter Publishing

Another Twitter-based project, this is a tool designed to help publishers distribute their content to people who have demonstrated interest in a topic by tweeting about it. The basic concept is to identify a target audience based on people's tweeting behavior, then deliver them content they are likely to find relevant by analyzing the content of their tweets.

Machine-Generated Sports Stories

This project doesn't have a clever name (yet), but it is in some ways the most interesting -- and could be the most controversial -- of the projects. This team, which includes "programmer-journalist" scholarship winner Nick Allen, has built a system that reads the box score and play by play of a baseball game and automatically generates a story about the game. It's interesting because the team's software addresses a very real need among a variety of publishers -- from ESPN to college sports information directors to high school athletic directors -- who would like to produce baseball game stories quickly and inexpensively. It's potentially controversial because one might argue that this software will make it possible to eliminate journalists' jobs. I don't think that's the case, because most of the articles the team's software might write would not have been written by a human reporter at all. Furthermore, a tool that auto-generates game stories would free up beat reporters to focus on enterprise coverage rather than game accounts.

Over the next couple of weeks, I'll write a bit more about each of the five projects. If you want to know more now, check out the students' Web site, writeclick.org.

For Brian Boyer, the circle is complete.

Almost exactly two years ago, Boyer saw a posting on BoingBoing about scholarships for computer programmers interested in studying journalism. He was one of the first to apply for the "programmer-journalist" scholarships, and enrolled in the master's program at the Medill School in January 2008. In December, he was one of the first two scholarship winners to graduate.

This past week, Boyer announced that he has a new job, starting soon at the Chicago Tribune. And for good measure, he published a guest post on O'Reilly Radar blog, one of the world's most popular sites for technology professionals.

The headline on his blog post: "Hackers wanted! Scholarships available to coders who'll come to journalism and help save democracy." An excerpt:

Journalism is an info-geek's dream. You're constantly learning new topics, speaking with experts, and distilling real-world issues to their essence -- all in the mission of informing the folks who don't have time to soak up all that data. It's like being paid to write a new Wikipedia article every day.

Boyer's post has already generated more than two dozen comments, and it even made an appearance on the home page of Digg. Not all the comments have been positive. Some readers are questioning why a programmer would want to work in journalism; others wonder whether journalism has jobs sufficiently challenging and well-paying to be worth their while.

Boyer's new job might be the best answer to these concerns. He will be News Applications Editor for the Tribune, heading up a small team responsible for "a wide variety of data-driven web applications to visualize data and present investigative stories online."

Bill Adee, editor of digital media for the Tribune, was responsible for hiring Boyer. In an email interview last week, he provided more details.

1) Why was this position created?

Gerry Kern, from Day 1 as editor of the Tribune, made it clear he wanted the Tribune to serve as a watchdog in government and consumer affairs. To do so, he made it a priority to increase our digital capabilities in telling stories through data and making that data available to our online users.

2) What will Brian's responsibilities be, and who will report to him?

Brian and I have discussed two main responsibilities: (1) How do we collect, organize and display every piece of data -- photos, documents, etc. -- we touch? (2) How do we expand our ability to tell stories with data and display that information to fuel and inspire our journalism and our mission as Chicago's watchdog? Brian will lead a team of two "hacker journalist" types -- including Darnell Little, who has been doing great work here already -- plus a data visualization expert.

3) Why did you choose him for the job?

I picked Brian because of his experience as a developer and a project manager combined with his more recent journalism experiences. I hired him because I like people who are willing to pay a price for what they believe in. He walked away from his first career to become a "hacker-journalist" because he thought it was the best way to contribute to society. I also like the way he has connected with others in the hacker journalist community (with Twitter and his blog).

1. Journalists will learn technology development;
2. Technology developers will learn journalism;
3. Journalists and technology professionals will learn to collaborate.

The Pulitzer Prize awarded last week to the St. Petersburg Times for PolitiFact, a database-powered website assessing the truth of political statements, is proof that journalists can learn computer programming. The idea behind PolitiFact came from Times reporter Bill Adair; the database and software development under the hood was built by reporter-turned-developer Matt Waite, whose job title is news technologist.

The Knight News Challenge grant that financed scholarships for technology developers to study journalism at the Medill School at Northwestern University was a bet that programmers can learn journalism. At Medill, the first two scholarship winners helped develop News Mixer, a widely praised web application demonstrating new ideas for enhancing conversations around news.

The third option -- building communication and collaboration between journalists and technologists -- is being tested during the current academic term at Medill. The third Knight "programmer-journalist" scholarship winner, Nick Allen, is part of the newest experiment, in which a Medill class is collaborating with a concurrent class of computer science students from Northwestern's McCormick School of Engineering.

Before I tell you more about this new educational experiment, I'd like to celebrate PolitiFact. After all, who would have thought a few years ago that it would be possible to win a Pulitzer -- journalism's most prestigious prize -- for software development?

PolitiFact, of course, is software and a whole lot more. The core of the idea is not new: for years, news organizations have included regular features that assess the veracity of statements by political candidates. The St. Petersburg Times team that developed PolitiFact simply turned that idea into a database, backed it up with thorough reporting, wrapped it all together with an appealing design, and added a dash of entertainment (labeling the most egregious political fabrications as "pants on fire.")

Aron Pilhofer, who leads the New York Times technology team that produces software-backed journalism projects, nicely summarized the significance of the Pulitzer award as a "Color of Money" moment. He was referring to the Atlanta Journal-Constitution's groundbreaking 1988 investigative series written by Bill Dedman and documenting systematic discrimination against minorities in real estate lending. The project, and the prize, validated a growing movement among journalists to learn analytical techniques -- and software applications -- to find meaning in data in order to produce better journalism. In the years that followed, thousands of journalists learned these techniques, which journalists labeled "computer-assisted reporting" or "precision journalism."

In 1989, when the project won a Pulitzer, computer-assisted reporting was a novelty. By 2009, major investigative projects that didn't include data analysis were the exception rather than the rule. And the data analysis techniques used for that project had become widely used in day-to-day reporting as well.

Twenty years from now, I hope we'll see thousands of journalists developing online software applications that inform, engage and enlighten the way PolitiFact does. The question is how we'll get there.

This term's Medill "innovation project" -- directed by my colleague Jeremy Gilbert and me -- is trying to figure that out. We are collaborating with Kris Hammond and Larry Birnbaum, who run Northwestern's Intelligent Information Laboratory. For years now, they and their colleagues have focused on projects that deliver "frictionless information" -- that give people exactly the information they want, when and where they want it.

This quarter, they are running an Engineering School class seeking to apply their approaches to challenges and opportunities facing journalism and media. Meanwhile, Jeremy and I are directing a class of Medill master's students exploring new ways of bringing journalism and technology closer together. You can follow the journalism students' progress on their class blog, writeclick.org.

Student teams made up of journalism students and technology students are working on five discrete projects:

• A "smart editor" that helps journalists research and check facts as they are working on a story;
• An application that generates personalized sports articles from baseball box scores;
• A tool that would allow a person to request a personalized news feed based on exactly the amount of time he or she has to consume news;
• Two projects that leverage Twitter -- one to serve "tweets" relevant to an article you are reading, the other to deliver relevant news based on a user's "tweets."

These projects are experimental and exploratory. Whatever their direct results, I'm hoping that this combined class will turn out a small cadre of journalists who understand technology and the process of technology development, and a comparable group of programmer-developers who understand journalism and media.

1) Tell us about your background.

As an undergraduate, I studied computer science at Oregon State University. They have a couple degree options for CS majors there. You can stick to the theoretical track, or do work in a related field. I always knew that I was more interested in the practical side than the theoretical one, so I started studying new media after a brief flirtation with business. In the new media program, I took courses in video production, Web design, animation and writing for the media.

I found that I loved the expression that you are able to put into media production. For one class I spent about two solid days editing one minute of video. Oregon State also has a top-notch new media faculty.

I was also teaching myself the basics of Web development on the side, largely by doing projects for student groups. That was a slow process, with a lot of trial and error, but eventually I ended up being a teaching assistant for the introductory Web design course.

2) How did you get interested in journalism?

As I was wrapping up my undergraduate degree I knew I wanted to work with media, and I had broadened my news consumption quite a bit. I took a particular interest in podcasting and radio, and jumped at a chance to intern at the public radio show "Radio Open Source" after graduation.

Open Source was trying to convert a mass media channel into a truly two-way media system through techniques such as looking for and discussing show pitches from the audience on its blog. Even shows that take call-ins are essentially one-way conduits. On those shows, the vast majority of the audience still consists of non-participants. Genuine two-way communication was new, and intoxicatingly interesting.

Also, a couple of world events have defined my brief adult life and developed my interest in journalism. The first was the rise of the Internet. I doubt any technically-minded person of my generation could have looked at the Internet as an adolescent and not seen its huge potential. Journalism is one of the fields where this impact will be most interesting, and there is something alluring about the opportunity to embrace the zeitgeist.

The other transforming event was the rise of terrorism in the American consciousness.

I was fortunate to see David Ropeik speak at Medill on the news media's ability to analyze risk, which he says is not very good. The media's narrative about the risks of terrorism is an illustration of that problem. Your risk of being killed by a drunken driver is vastly higher than being killed by terrorism in the United States, particularly if you live outside of a major city. Yet we do not see media panic about drunken drivers, and advocates of policies designed to curb drunken driving at the expense of civil liberties would see much more heated objection than was raised to anti-terrorism measures earlier in this decade. Instead we get things like the duct-tape and plastic fiasco.

The framing of these issues has as much to do with pressure to monetize news as an objective evaluation of the story or any kind of institutional political bias. That isn't a very good way to engage in public policy debate. The terrorism story is just the most egregious example. You can read the news every day and see this happening. How many important stories are buried on A17 and chopped to 300 words while the triviality of the day leads? And of course a lot of papers don't even have an A17 page anymore.

We know that the economics of news are changing, and we know that economics have a big impact on news organizations. What we don't know is how these economic changes will change news decision-making. That is a question I am very interested in.

3) What have you learned by studying journalism so far? How has the experience changed your outlook?

Producing good journalism is a lot harder than it looks. I remember reading a few stories before coming to journalism school and thinking, "What a terrible story. That came straight from a press release." Reporting on a deadline makes you realize that finding stories is a real challenge.

Journalism school also makes you appreciate newspapers. As journalists argue that the death of papers will also kill most original reporting, I don't think many media consumers realize how hard that will be to replace. We talk a lot about the quality standards that "professional" journalists have, but the ability to find The Story is probably their most important role. To the extent that we can finance that kind of work in a digital media world is a critical question.

As a developer, you kind of regard the Web as a vehicle for new opportunities. The journalism world sees the Web as a leech, which is a viewpoint journalism school has made me come to terms with. Meanwhile, social responsibility is something that web companies ought to be thinking about more.  "Don't be evil" is not the same as "be beneficent."  The idea that Google or Microsoft might have a news department of some kind is not a whole lot stranger than GE owning the NBC Nightly News.

At the same time, there is no question that journalism organizations need to be thinking about the Web in a more opportunistic way. In the last few weeks we have seen a lot of different ideas for revenue models kicked around the web, which is a really positive sign. If newspapers start closing, we are going to see a lot of creatively funded entities stepping up to fill the news void in those markets.

Another great thing about studying journalism is that you get to refine a lot of valuable skills. Being a technologist, I tend to emphasize the value of technological tools, but there is no doubt that the value of basic journalism competencies, like strong writing skills or the ability to craft a narrative, is not going away. Studying journalism is, hands down, the best way to become a better writer.

4) What do you think are some interesting career opportunities for people who blend journalism and technology knowledge?

• Web Editor: A programmer-journalist is a perfect match for a Web editor position. The modern Web editor needs to be able to look over a news budget and instantly see what stories call for multimedia content, what that content will look like, and what tools should be used to produce it. Her or she should be able to hack things together on the fly. Your next content management system will be open source. Your next Web editor should know its guts.
• News Innovator/Web Evangelizer: Keeping a news organization ahead of the competition technically will be a constant battle. The news innovator's role will be to invent, scout, and borrow innovative tools and techniques.
• Multimedia Director: Bringing together reporters, graphic artists, and programming talent for multimedia project is a daunting task. Programmer-journalists understand enough of each of these disciplines to fit all the pieces together.

You can learn more about Nick at http://www.nicholasdallen.com and follow him on Twitter (@nicholasallen).

Here's one little example of why this premise was on target: ChangeTracker, a new service offered by Pro Publica and developed by Brian Boyer, who just graduated from the Medill School as one of the first two "programmer-journalist" scholarship winners.

ChangeTracker is a service that "watches pages on whitehouse.gov, recovery.gov and financialstability.gov so you don't have to." It identifies changes made to any of these pages and allows people to track the changes via a Web page, Twitter feed or email alert.

For policy wonks and others who want to keep track on what the Obama administration is up to, it's proving quite valuable. Just yesterday, the service was mentioned in a Washington Post article about the administration's technology efforts.

What I find most interesting about ChangeTracker is not its technological sophistication. No hard-core programming skills were needed to put it together. ChangeTracker uses a free Web service called Versionista to identify changes to the sites, then routes the output through Yahoo! Pipes to create an RSS feed.

Pro Publica doesn't try to argue that ChangeTracker is high-tech; in fact, the site provides instructions on how to create a similar service for any Web page you might want to monitor.

Still, I think ChangeTracker clearly demonstrates the value of bringing software developers into journalism. The idea for ChangeTracker came straight from Boyer, who just started an internship at Pro Publica. Back in July, Boyer wrote about Versionista when the McCain campaign used it to point out changes in then-candidate Obama's position on the Iraq War. He pointed out that Versionista is really a variant on the "version control" tools used by software developers to figure out "who broke something."

In a typical newsroom, a journalist might have recognized the newsworthiness of keeping tabs on changes made to key presidential Web sites. And a software developer would have understood the utility of Versionista. But working separately, they wouldn't have come up with ChangeTracker.

A journalist without Boyer's programming background could have done everything that he did in building ChangeTracker. But it took a "programmer-journalist" to come up with the idea.

Here's a new opportunity: BarCamp NewsInnovation, a series of user-generated conferences focusing on the future of journalism. The next conference in the series will be held Saturday, Feb. 21, at the Medill School newsroom space in downtown Chicago. Registration is capped at 50; as I write this there are 18 registration spots left -- sign up if you'd like to participate.

Jason Kristufek at the Gazette in Cedar Rapids is the lead organizer for the event. Beatrice Figueroa of Northwestern's Media Management Center is overseeing logistical arrangements. (Thanks, Jason and Beatrice!).

I've never attended at BarCamp so I've been reading up about them. It will be interesting to see how the BarCamp process works. I've been part of many traditional conferences, and when they are well organized, they can be great learning experiences. Still, many conferences end up being just an endless series of PowerPoint presentations that never get to the core questions and issues that I'm interested in learning more about.

The BarCamp process, which encourages participants to suggest topics and to volunteer to lead sessions, would seem to be dependent on the quality -- and insights -- of the participants themselves. So if you're interested in the future of media and journalism -- especially if you can bring the perspective of another discipline, such as software development -- please sign up. Journalism and our democratic society need your help!

The demonstration Web site, launched in December by a team of Medill students, shows off some interesting new ideas for engaging people in online conversations around news. The site has attracted quite a bit of attention from people interested in the future of journalism, social media and new technology.

More than just attention, in fact. There are now at least two separate organizations actively working with News Mixer's open-source code.

One is the (Knight News Challenge-funded) Populous Project, which announced recently that it will incorporate News Mixer's functionality into the Populous open-source publishing platform for collegiate newspapers.

And just this past week, e-Me Ventures (a Chicago-based technology firm affiliated with Gazette Communications, which sponsored the class that developed News Mixer) announced it had deployed a portion of the News Mixer code as an add-in to a test site, powered by WordPress.

"The News Mixer idea was huge. I was really blown away by the work that [the students] did," said Abe Abreu, CEO of e-Me. "We wanted to be the first to do something with it."

With these new developments, it seems like a good time to lay out some of the ways News Mixer -- and/or its functionality -- might be implemented on a production Web site.

Option 1: Deploy News Mixer as a complete publishing/interaction platform

News Mixer was developed, in partnership with Gazette Communications, as a standalone site geared to young adults in eastern Iowa. It was intended to achieve two goals: build connections with and among young adults, and improve the caliber of conversations around news. The combination of those those two goals explains why the site uses Facebook Connect to leverage the value of participants' existing online social networks, as well as why the site offers new commenting approaches (short format "quips" and paragraph-based questions and answers). Team Crunchberry (as the student team called themselves) hopes that this combination of features can address some of the well-established limitations of the open-ended comment box that is now the main vehicle for user interaction on news sites.

I think News Mixer remains a solid idea worth implementing as a stand-alone site. Online news sites, especially locally oriented ones, need to increase user engagement and to find ways to connect with young adults. But we won't really know whether the News Mixer approach achieves those goals until someone launches a site and promotes it to the target audience.

Gazette Communications, which owns the newspaper and ABC-TV affiliate in Cedar Rapids, Iowa, has applied for a Knight News Challenge grant to deploy a version of News Mixer, test it with users in eastern Iowa and refine it based on user experience and feedback. I've got my fingers crossed.

In terms of functionality, the site appears close to production-ready. In the class's final report (80 pages - 2.7MB PDF), the student team suggests a few desirable, and relatively simple, improvements before launch:

• Paginate or content on users' profile/feed pages, which list quips, questions and letters posted by your Facebook friends or others you follow on News Mixer. (Right now, new postings are added to the top of your profile/feed page, which just grows longer and longer.)
• Add photo handling capabilities to the content management system behind the site. The CMS, based on the Django Pressroom application, currently allows text content only. But there are other Django applications geared to photo handling, so incorporating this should be straightforward.
• Anticipating that at some point the site might have contain many articles on many topics, add the capacity to put articles and letters into categories. This would allow the site to, say, have separate sections for sports, business and local news. Again, this should be relatively simple.
• Consider adding "flag as offensive" options for answers and quips. (The capability is already in place for letters.)

In addition to these improvements, someone launching a version of News Mixer should expect to fix some bugs and load-test the site. If traffic grows, it may be necessary to make adjustments (in caching, for instance) to ensure good performance.

The student team also suggests consideration of several enhancements: (1) email notifications when your question or letter is answered; (2) addition of a site search engine; (3) some kind of highlighting for question answers provided by the news organization's staff.

Another issue to consider is browser compatibility. Right now the site is optimized for Firefox 3.0 and Internet Explorer 7.0. Facebook Connect (and News Mixer) rely on JavaScript, and there were significant JavaScript changes between IE6 and IE7. Making the site compatible with earlier browser versions will require testing and additional code.

Option 2: Incorporate one or more parts of the News Mixer code into an existing Web site

This is the approach taken by e-Me Ventures. The company engaged Brian Boyer and Ryan Mark, the two programmer-journalists who wrote the code that powers News Mixer, to migrate some of News Mixer's functionality to other sites. The first piece, programmed by Boyer and financed and supported by the e-Me team, is based on News Mixer code. It incorporates Facebook Connect and the quips into a test site, Iowa Content Experiment.

Some of the key functionality of News Mixer fits beautifully into the strategy being developed by e-Me Ventures, Abreu said. The company seeks to develop a content platform that will aggregate content, apply "semantic Web" technologies to increase relevance, then "make it all social." News Mixer is an example of "innovation around the integration of the content and the social," Abreu said.

E-Me Ventures is funded by Gazette Communications and other investors, and the company is seeking additional funding. The company doesn't want to create a News Mixer site, but rather is trying to "widgetize" interesting functionality that investors such as Gazette Communications can use on their existing sites. The Facebook Connect-powered "quips" are an example of a widget that could now be used by the Gazette or by other companies, Abreu said.

The quips widget will be made available on an open-source basis, as will other widgets based on the News Mixer code, Abreu said. He said e-Me will first make the widget code available for beta testing to his company's partners and investors, then release it generally. He said the code would be released on an open-source basis shortly after the launch, scheduled for March, of a site using the quips widget.

Regardless of what e-Me Ventures does, Boyer says the News Mixer (Python) code is "relatively modular," so a good Web developer should be able to use it to incorporate the site's functionality into a different site, even one based on something other than Python/Django.

Option 3: Take inspiration from News Mixer, write your own code

The News Mixer demonstration site applies four novel approaches -- novel, at least, for news sites -- to building engagement. They are:

1. Facebook Connect. While there are many sites using Facebook Connect to authenticate users, few are doing more than that. But Facebook Connect can be much more than a "single sign-on" service that makes establishing a new site-specific login ID unnecessary. All Facebook Connect applications allow users to sign with a Facebook ID, and have the option of cross-posting content to Facebook. News Mixer does more. Every user gets a page where contributions from Facebook friends are aggregated, and it's easy to invite your friends to participate on News Mixer. On the home page, contributions from your friends are highlighted.
2. Quips. This is the most visible new "commenting structure" on News Mixer, and people really seem to think they're cool. Allowing IM- or Twitter-like comments, and displaying them alongside the content (instead of at the bottom) is worth trying on other sites.
3. Questions and answers. This feature allows people to ask questions about the content in any individual paragraph in an article -- and then lets other users, including the journalist who wrote the article, respond to those questions. As with quips, the idea is to put some structure around the way users comment on the site, in hopes that this will lead to more substantive conversations. I'm particularly intrigued by the idea that this is a way of increasing the engagement between journalists and their readers -- and spawning follow-up reporting that the journalist might not otherwise have thought of.
4. The display of letters to the editor. The idea of letters to the editor is, of course, not novel at all. But their treatment on News Mixer is different than on other news sites. The key difference lies in the way News Mixer allows a site manager to designate letters as "editor's picks." Once a letter is chosen as an editor's pick, the site treats it equivalently to an article. So a site visitor will see a feed of articles intermingled with "editor's pick" letters. It's a way of rewarding letter writers who make particularly strong or cogent points.

Each of these ideas can be implemented separately, using technology platforms other than Python/Django. If your site relies on a different programming language or Web framework, you wouldn't be able to reuse the News Mixer code, but that shouldn't prevent you from using one or more of these ideas.

The vast majority of programmers wouldn't think the idea of studying journalism was even an interesting idea. Some would say journalism isn't worth their intellectual energy; some would be terrified by the prospect of doing what journalists do, like interviewing strangers or having to do a lot of writing. But Ryan and Brian dove right in. Not only that, they excelled in our program. I was half-expecting that at some point, one of my faculty colleagues would ring me up and say one of these guys just wasn't cutting it in our reporting and writing classes. Instead, I heard nothing but positive feedback.

They also left behind some really nice work: articles produced (by Ryan and Brian) in our Chicago reporting program, the EnviroVote site (read more about it here), and News Mixer, the "capstone" project they developed with fellow master's students Angela Nitzke, Joshua Pollock, Stuart Tiffen and Kayla Webley. I also really appreciate this post that Brian wrote for the Idealab blog, describing nine possible careers for programmers with journalism training.

I remember reading Tom Wolfe's "The Right Stuff," a book about the Gemini Mercury astronauts. The leaders of the U.S. space program recruited the nation's first astronauts from a pool of ace pilots, even though what they would be asked to do was nothing like piloting a plane. In choosing the first astronauts, NASA looked for a lot of other qualities: for instance, they had to be comfortable doing public speaking, because the leaders of the space program realized they needed to build public support.

The way I see it, Ryan and Brian were the Gemini Mercury astronauts for the programmer-journalist scholarship program. They've proven that programmers can succeed in a journalism master's program, and that people with programming skills can help develop solutions to journalism's challenges that people with more conventional journalism backgrounds wouldn't come up with.

Soon Ryan and Brian will be back in the working world, and it will be interesting to see what they do with their unusual blend of journalism and programming skills. The time seems to be right; programmer-journalists (even those not named Holovaty) are starting to get noticed. For instance, read this New York Magazine article on the team of developers working on Web applications for The New York Times, Mark Potts on Politifact.com, or Eric Ulken and Dan Honigman about the Times team. (By the way, coders: We still have scholarships available! More information here.)

Ryan is spending the next few weeks traveling, and will be looking for work when he returns in February. Brian has a 3-month gig at Pro Publica and hasn't figured out exactly what will happen after that. On their way out of Medill, they both agreed to answer a few questions about their experience:

What do you feel are the most important things you got out of your Medill experience?

Mark: I got a new skill. I learned how to write and report, something I had no idea how to do before I started. I feel confident enough about what I learned that I could go out and report for a living. Also, I got to work with great people. I have never worked with a group of such talented, driven, capable people before.

Boyer: In many ways, Medill is an old-school institution, and I learned the old-school way of practicing journalism: from the laws and ethics to slugs and nut grafs and inverted pyramids. This is the sort of knowledge software designers rarely have about their users. Now that I'm a journalist, I can write tools for myself -- and that's the easiest way to get it right. In addition, participating in the program has put me in touch with an amazing community of journogeeks. There are a mess of brilliant folks doing great work to further journalism online, and during my time at Medill, I've been able to connect to that community, in person at conferences and via blogging and Twittering.

Why should someone with solid programming skills consider a master's degree in journalism?

Mark: Because journalism needs them. There are so many tech-capable people in journalism, but few who have logged the time to understand computer science and software development. A person who does not want to just write code for whoever pays them, and actually come up with and execute interesting software projects, the journalism experience will help you. This program got me out of my element and gave me first hand experience that will help me relate to others in the field when i'm not elbow deep in code.

Boyer: Because it's fucking important. Cable television and the Web disrupted the business models of the big, important journalism organizations: newspapers. Now, the importance of a daily paper is debatable, but that democracy requires journalism to function is not. And so, for the sake of democracy itself, it is imperative that more nerds join the fight to save the news. We need to invent new business models, reinvent the newspaper, and create new forms of media. Plus, an all-expense-paid trip to graduate school in sunny Chicago, Illinois, is also a very nice way to weather a recession. And the smart, passionate classmates make for some pretty good parties and great conversation.

What contribution can programmers make to the future of journalism? Or, why does journalism need programmers?

Mark: How do we deliver news and information on the Internet? Access to timely, truthful and in-depth information is necessary for our open, educated and democratic society, and is the core goal of journalism. Newspapers, magazines and television are slowly being replaced by the Internet. The Internet is built by programmers, and only those with understanding of computer systems and programming know what the Internet is capable of, and how to make things happen.

Boyer: Journalism and democracy are facing problems that aren't going to be solved by MBAs, IT departments or bean counters. Journalism needs engineers, computer scientists, and usability wonks, badly.

What do you hope to do in your career, and how have your career aspirations changed since studying journalism at Medill?

Mark: Since I have tech skills, I can pick an industry to work in. Now, in addition to the tech skills, I have a deep understanding of media and what is actually involved in gathering and distributing the news. So I've picked an industry. And it's an industry that has meaning, something I can feel good about. This isn't telecom or sales. The work I can do in journalism has the potential to change people's lives for the better. This isn't a path to a fat paycheck. This is something that can give you opportunities to get excited about your job, and give you a sense that you've done something good and meaningful.

Boyer: I came to Medill because I wanted to make a real impact in people's lives. Instead of making software for investment groups or law firms, I wanted to do something good for the world. My time at Medill has clarified that vision. I want to use my skills as a technologist to fix journalism. As for career specifics, I'm less certain. I don't do five-year plans. My next stop is ProPublica, a fantastic new non-profit investigative journalism organization. Following that, it's less clear, but I plan to keep writing and designing software, and to do so as a journalist, at journalism organizations, and for the public to experience. Software may be my first love, but I'm for certain not planning to abandon my newfound writing skills. I enjoyed writing stories, and will continue to do so. And I've got an idea for a non-fiction book, about neither journalism nor programming, and I now feel qualified to make it happen.

News Mixer is the final project for six graduate journalism students, including two "programmer-journalists" attending Medill on Knight News Challenge scholarships. It melds three "commenting structures" -- question and answer, short-format "quips," and letters to the editor -- into a site that leverages users' social networks by using the newly released Facebook Connect system.

The class officially ended Dec. 12, but the students and I have remained busy. Stuart Tiffen prepared a nice video overview of how the site works. And with the students' help, I've finished editing the team's (79-page PDF) final report, which explains how the site was developed, lists additional features that the class didn't have time to implement, and includes findings and recommendations for journalists, newsrooms, media companies and journalism schools.

When my colleague Jeremy Gilbert and I were organizing the New Media Publishing Project class that created News Mixer, we had a sense that the time was right for some new thinking about news-story commenting. A few of the indicators that fresh ideas were welcome: Gawker's post in July contending that newspapers shouldn't allow online comments, Techdirt's reaction to the Gawker post, and even a story in the Onion ("Local Idiot To Post Comment on the Internet").

So it's been quite interesting -- and rewarding -- to see the reactions News Mixer has generated. Here's an overview of some of the best:

Mark Potts, an online news pioneer who now works as a consultant, wrote that "Anybody interested in smart ways to cover local news and building online communities around news coverage-in other words, anybody who wants to succeed in the news business going forward-should rush over to the new NewsMixer site. ... It crosses local news reporting with Facebook, encourages citizen participation and includes a very slick way of commenting and posing questions (for a reporter or the community to answer) about any particular facet of a story. Very neat."

"I interrupted writing this post to talk to the designer and developer of a little project I'm working on about ways that NewsMixer can inspire and influence what we're doing," Potts wrote. "Congrats to ... the team at Medill and the Gazette for a great piece of innovation. We need more like this."

Patrick Beeson, a Web project manager for E.W. Scripps Co., wrote that News Mixer could be a "a game-changing effort for news story comments," adding that "News Mixer has put together a great method of directing user participation on their site." Beeson added:

Traditional Web story comments are wrought with problems, especially when editors and content producers are hands-off in their management style. ... News Mixer seems to have solved these problems by limiting "comments" to a question/answer session on each paragraph of a given story.

News Mixer's "radical take on user participation is a great step forward for news sites," Beeson wrote. "And because News Mixer is built in Django, I plan on using their open-sourced code for my own project very soon, in fact. :)"

Editor and Publisher wrote an article about the site and followed up with a blog post praising the Knight Foundation as "the one ray of sunshine in this industry." Bloggers Mark Fitzgerald and Jennifer Saba said News Mixer is "a very cool new project ... [that] lets readers log on with their Facebook IDs and comment on the news in Cedar Rapids in unique ways."

Nick Gehring, an Ohio journalist and blogger who helps manage the Kent State University Web site, wrote that "Medill's tool takes news-story commenting out of the ghetto. You know you've seen it -- those awful, racist, and oftentimes off-topic comments made under some news articles. Newspaper Webmasters have been notoriously awful at moderating their communities."

Gehring wrote, "The icing on the cake, though, is the Facebook ID integration. This forces users to use their real identities -- although the users could fake a profile on Facebook, just like anywhere else, but I don't see this as likely as on-the-spot Web site registration." (If you're interested in Facebook Connect, check out this huge and growing list of sites that use this service.)

Gehring writes, "Perhaps Newsmixer will help end the debate over the value of story commenting. Yes, there is value! Blogs and other non-newspaper sites have proven this for the past few years." He added:

I once suggested -- and received a fantastic guffaw from an older journalist -- that we should treat stories online more and more like blogs. Does this mean dropping objectivity and providing more analysis than just-the-facts-m'am? I don't know, but I do think it means writing stories and directly engaging the people who comment below them. Aside this News Mixer system, reporters should be regularly responding to and commenting below their stories. Arguably, these same journalists, with some help, should be managing the online communities of their beats.

Ryan Sholin, who is developing the ReportingOn project funded by the Knight News Challenge, wrote that News Mixer "has a huge amount of potential as a conversation vertical, along the lines of the Guardian's Comment is Free.  I don't see Newsmixer running as a mainstream news site, but as a place to substitute for outdated message boards or underused staff blogs.  Populate it with content from your news and opinion sections, and let it stand as the forum for reader feedback, use it as your primary source for comments, letters, and other reader-authored content to run in print. Heck, if it gets big enough, print the letters and comments as a four page insert once a week, not just in a box on the opinion page."

In a comment at the Knight Digital Media Center blog, Aron Pilhofer of the New York Times wrote, "There are bits and pieces of it I'd like to steal right now." And Pilhofer is in a position to do so, because he leads a team of developer/journalists who develop applications for the Times' Web site.

Most of the reaction in the blogosphere has come from sites (and bloggers) involved with the journalism industry. That's why it was particularly interesting to discover that News Mixer got a lengthy writeup on Read/Write Web, one of my favorite technology sites and the 15th ranked blog on Technorati's list of the top 100 blogs based on inbound links. Under the headline, "NewsMixer: An Innovative Community News Framework," Sarah Perez praised News Mixer's "great features" and its use of Facebook Connect:

Instead of allowing for the creation of fake names or internet handles for use on the site, Facebook authentication means that people's real identities are being displayed. No more comments left by internet trolls hiding behind their mask of anonymity! Today's commenting systems are largely broken, as social media pundit Robert Scoble noted today on his blog. The main reason for his post was to share ideas about the state of commenting and interaction systems on the web. He wanted there to be a way that he, as the writer, could call attention to some comments as being more important than others. He had also said that he wished there was a way to see the social networks of the people commenting. As it turns out, News Mixer has introduced a great example of how that second request of his could work.

Perez urged newspapers to " borrow some of News Mixer's ideas as well. It's not too late to save the daily paper - it just takes some fresh ideas. Like Rupert Murdoch recently said, the time for doom and gloom is over - the internet is really just a huge new market ready to be tapped. We agree. Now is the time for innovation because...well, it's either innovate or die. Hopefully most will choose the former."

Gazette Communications, the Cedar Rapids, Iowa, company that sponsored the class that created News Mixer, is one local media business that seems to agree with the need to innovate. Led by CEO Chuck Peters, the company is a leader in rethinking how a local media company should operate in the digital age. The company, which owns the newspaper and ABC television affiliate in Cedar Rapids, plans to use the software that powers News Mixer to launch a site for eastern Iowa in 2009. It also has applied for a Knight News Challenge grant to continue developing the software.

International attention

News Mixer has also attracted considerable interest from outside the United States.

Richard Kendall, Web editor for a British newspaper, wrote that News Mixer shows "all the freedom and open-ended opportunity the online world bring to news media. ... Bringing the public and journalists closer - certainly not ground-breaking, but by making it the focus of the site rather than an optional element it does help to seed the sense of community?" Noting that the software powering News Mixer is open source, he added, "And if it all works out, we'll all be doing it."

In Sweden, Citizen Media Watch, a site operated by online journalists Lotta Holmström and Gitta Wilén, described News Mixer as "a great new tool for news discussion and fact-checking." Holmström described the site as "quite cleverly set up," adding:

What I like most about it is the way that any story can be scrutinized paragraph by paragraph by adding questions and answers, thus providing a tool for collaborative fact-checking and discussion about the validity of statements. It is also a social tool, letting me know when my contacts have been active on the site. And it flattens the news hierarchy ...

Also in Sweden, the CEO of Mindpark.se, a Web development business owned by several Swedish newspaper companies, also wrote about News Mixer. I don't read Swedish, and Google's translation is only somewhat helpful. But I can pick out that author Joakim Jardenberg thinks News Mixer is "hyper-interesting, well thought out and timely."

In Spain, the Web site periodismociudadano.com (CitizenJournalism.com) described News Mixer as "giving a new dimension to the usual system of comments."

Michael Kowalski, who runs London-based Web development firm Kitsite and previously worked at the Guardian, described News Mixer as "an interesting spin on news commenting." He found the Facebook Connect integration "very cute. No choosing passwords, waiting around for emails, or any of that. Though it does also feel kind of terrifying."

Other international sites that have written about News Mixer:

• Andy Dickinson, who teaches online journalism at the University of Central Lancashire, linked to Sholin's post and added, "I agree that this should be a model that is explored further."
• Reffos.com, the blog of an Indian software developer, linked to the Read/Write Web post with a succinct response: "Steal the idea."
• Dirk von Gehlen, editorial director of jetzt.de (Süddeutsche Zeitung) in Germany, also linked to Potts' post.
• Anton van Elberg, a Dutch online editor, posted about News Mixer on the site www.rethinkingmedia.nl.

Other links

For the most up-to-date set of links to sites commenting about News Mixer, check out my News Mixer tag set on Delicious. Here are a few recent additions:

• Deborah Potter, executive director of Newslab and co-author of the journalism textbook "Advancing the Story: Broadcast Journalism in a Multimedia World," wrote about News Mixer under the headline of "News mixed with Facebook."
• At Journalism.co.uk, Laura Oliver interviewed Brian Boyer about the site.
• On the Society of News Design site, my new Medill colleague, Matt Mansfield, wrote about News Mixer as an example of "a brave new world of storytelling that includes programmers who understand journalism."
• Daniel Bachhuber of CoPress, a group trying to improve content management systems for college student media, posted to FriendFeed about News Mixer: "Really cool light-weight Django application that allows you to "mix up" the news. Most notable for me is that comments under 140 characters (i.e. Twitter-length) are called "quips" and those under 250 characters (actually, up to 250 words - RG) are "Letters to the Editor." Login is seamless with Facebook Connect, and I think the project has tremendous potential as a community building exercise (although I'm not sure how I feel only limiting myself to the comments my friends make, if that's the case)."
• Mark Briggs, a consultant and entrepreneur who formerly worked at the Tacoma News Tribune, described "identity management" as one of the three key new technologies for journalists. "See Newsmixer.us for an excellent example of how to implement this game-changing opportunity," he wrote.
• Joseph Cizek, a California-based Web producer, was concerned about the idea of associating his online comments with his Facebook identity. "The problem for me is that I've kept Facebook as my 'off-the-clock' social networking, and use Twitter, LinkedIn, et al., as my professional social networking. So, if by keeping up on the social media 'scene,' I have to use my Facebook identity professionally, I'm now concerned about the mixing of my business online presence with my personal online presence."

Here's what they came up with: News Mixer. It melds three "commenting structures" -- question and answer, short-format "quips," and letters to the editor -- into a site that leverages users' social networks by using the newly released Facebook Connect system.

News Mixer is already getting some positive buzz thanks to some Twittering last week after Team Crunchberry presented the site to Medill faculty and to the class's sponsor, Gazette Communications in Cedar Rapids, Iowa.

Patrick Beeson, a content manager for E.W. Scripps Interactive Newspaper Group, wrote that News Mixer "could be a game-changing effort for news story comments." Blogger Nick Gehring wrote that News Mixer "takes news-story commenting out of the ghetto."

What's online now is a demonstration site. Log in with your Facebook ID and you can see how it works (and help us load-test the software). Invite your Facebook friends to use the site and you can get a sense of the interesting possibilities. Here are some of the highlights:

Facebook Connect. Using a Facebook ID means you don't need to establish a new login and password to use News Mixer. Beyond that, Facebook Connect allows the site to display comments from your social network, meaning that every user has a different -- and personalized -- experience. We're thinking that this will stimulate more intelligent discussion than generally occurs via the open-ended comment box that appears at the end of articles on most news sites. Also, every time you post to News Mixer, you are given the option of cross-posting that comment to your Facebook feed, which exposes it to friends not using our site and potentially draws them to participate as well.

Three options to comment. Team Crunchberry decided to offer three very different options for reader response:

• Questions and Answers: Displayed like annotations in the margin of an article, readers can ask a question about any paragraph of the article -- or answer questions left behind by other people.

• Quips: Displayed as a small talk-bubble in a live feed on the home page and on article pages, quips are short-form comments that allow people to leave feedback in a quick, to-the-point form. They're modeled after Twitter and instant-messaging.

• Letters to the Editor: A very old idea, but with a few new twists. News Mixer calls on letter writers to "Add your voice to the marketplace of ideas. Offer a thoughtful point of view in 250 words or less." Once written, letters are treated equivalently to articles in News Mixer. Each letter gets its own page, and people are allowed to write letters in response. When a letter is particularly insightful, an editor can use the News Mixer content management system to designate it as an "editor highlight." The "editor highlight" letters are given prominence on the main Letters to the Editor page, and also appear on the home page, intermingled with news articles. The idea is to encourage and reward the most thoughtful responses.

User profiles. All users of News Mixer get their own profile page. On News Mixer, users are allowed to follow each other's activity on the site, and view the activity in their news feed. Along with your own contributions, recent comments from your Facebook friends and people you're following on News Mixer are aggregated and quantified in your user profile, which serves as the nexus for the News Mixer social community.

A personalized home page. The News Mixer home page highlights recent comments and "quips" from your social network. It also highlights a question that has recently generated a lot of activity.

Gazette Communications is interested in launching a version of the site, geared to young adults in eastern Iowa, in 2009. The code that powers the site is available on Google Code, and we've already gotten some interest in using it for other sites.

I'm really proud of the student team that developed this site in just 11 weeks (about six weeks of hard-core coding): Andrea Nitzke, Joshua Pollock, Stuart Tiffen, Kayla Webley and "programmer-journalists" Brian Boyer and Ryan Mark. We've never done original software development in a Medill "innovation project" class before, so the students (and their instructors, me and Jeremy Gilbert) learned a lot. And I think it really does offer some provocative new ideas for approaching conversations around news.

If nothing else, I think News Mixer is "proof of concept" that enrolling programmers in journalism schools -- and teaming them up with journalism students to develop an innovation relevant to the future of journalism -- is a good idea. We're still looking for applicants for the remaining Knight News Challenge scholarships -- if you're a coder interested in applying your skills to inventing the future of journalism, please check us out.

In the coming weeks, I'll be writing more about the lessons learned from the News Mixer project, and about other features that Team Crunchberry would add if they had more time. We welcome your feedback on News Mixer.