CONTENT VS. FORM

The survey was conducted November through December and distributed online via Twitter and our Facebook page, and was sent to Chilean journalists and the 904 people who were subscribed to our newsletter at the time; 463 people answered the questionnaire.

A summary of profiles and likes and dislikes showed two type of users very clearly: a) journalists interested in the content and b) developers and designers who are in it for the data visualization and how the site will be built.

In the first group of users who are on-board for the content, we discovered that only 26% work in media and content development. Meanwhile, 43% work on strategic communications agencies and PR agencies that are hired by private companies and do lobbying and monitoring of legislative work and government regulation. This could mean that they want to: 1. See what we publish about their clients. 2. Read what we publish about their clients' competition. 3. Find what information we publish about their agency or their agencies competitor. 4. Check information about elected government officials. 5. All of those combined. (That's gonna be fun).

TYPES OF USERS

As you can see in the slides above (in Spanish):

• 90% of our users are from Chile.

• 80% of them are men; 20% are women. They come from two big user segments: 54% are between 30 and 44 years of age, and 38% are between 18 and 29.

• Most are working (83%) and educated: 41% have completed university studies; 20% have a master's degree; 14% have enrolled in graduate studies; and 5% have a Ph.D.

• Most work in communications (24%), followed by technology (14%), consulting and advisory services (14%), education and training (12%), and government (6%).

• If you break it down in the communications sector, 43% work in communications & PR, and only 26% work in media and content development.

HOW THEY FIND DATA ABOUT PEOPLE, ORGANIZATIONS

• Most use Google (99%), followed by Wikipedia (80%), Facebook (69%), and LinkedIn (58%). This is followed by digital archives of news websites (50%), global databases (22%), government databases (20%), and private databases (10%).

• Of those who claim to obtain information from government databases, 37% said they use "Gobierno Transparente," a section in each one of the Chilean government websites which list the names of public officials and their salaries and purchases made by each agency.

THEY WANT IT ALL

We learned that for most users, all of the features proposed by Poderopedia are important: 97% want faceted search and advanced filters; 97% approve of the idea of having profiles; 96% want to view relationships; 70% say they want to collaborate with crowdsourcing, want more transparency, and to understand social networks and possible conflicts of interest; 63% want an API (application programming interface) for developers. And, here is a nice surprise: 88% want Poderopedia to publish journalistic special investigations.

SCHOOL DATA

Asked about which kind of data would be the most important to include in a Poderopedia profile of a person, most of the users said everything. But where a person studied was the most important thing for them (81%), followed by income (67%), family (67%), friends (65%) and classmates (54%).

Schools are relevant because they impact everything else. According to a 2008 study by La Tercera, 50% of Chile's top CEOs and executives come from five private schools: Verbo Divino, Sagrados Corazones, Saint George, San Ignacio and Tabancura. Three of the 15 richest families in the Forbes ranking studied in Verbo Divino, where Chilean President Sebastián Piñera and many members of his cabinet studied. In total, those five schools represent Chile's 18.5% GNP (gross national product) and 0.04% of total country's schools.

Social, business, religious and political membership data is also key to our users: 94% want to know about a person's links to public offices and NGOs (non-governmental organizations); 92% are interested in the data of business partners and companies; 81% about social private clubs; and 80% about think tank relations.

All this reflects something that we see every day in countries like Chile and many others in the region: Where you're born, your last name, and where you went to school and with whom, very much define your chances in life and reflect on who becomes part of the 0.1% who call the shots in Chile.

PERSONAS

The user answers also allowed us to create Poderopedia's first "personas" and to conduct several user observations for more insight (as you can see from our messy white board pictured below) in order to start paper prototyping.

In this whole process, good work methodologies have been crucial, especially those involving user-centered design by fellow Knight News Challenger winner Jesse James Garret, and the agile-development-no-meetings-zero-email workshop we took with the guys at Menlo Innovations, thanks to the Knight Foundation.

We'll be at SXSW this year, so if you're there, come see us. Here's the schedule of our presentation on civic media in Latin America (hashtag #sxswlatino). We'll have a few surprises. If you want more information or you're interested in getting involved with Poderopedia, please contact us at info (at) poderopedia.com or follow us on Twitter or Facebook.

In our ongoing quest to trace the outline of the phrase "civic media," we began the Center for Civic Media's 2012 lunch series with Paul Wright, editor and co-founder of Prison Legal News, and executive director of the Human Rights Defense Center, the non-profit umbrella which publishes PLN.

PLN operates in a unique media environment, where the very act of distributing a magazine to their customers might first require winning a lawsuit. You see, their primary audience is made up of prisoners themselves. Prison Legal News is the longest-running publication put together with the help of people who are incarcerated, and since its first issue in 1990, it has become a critical resource for discussing issues facing these populations. It's an independent, monthly magazine that reviews and analyzes prisoner rights, court rulings, and news about prison issues. PLN focuses on state and federal U.S. prisons, as well as some international coverage. Paul himself has become a distinguished advocate on behalf of the U.S. population. Asked whether we could blog his talk, Paul responded, "Secrecy is the antithesis of publishing."

From Newsletter to National Publication

Prison Legal News started as a newsletter, in 1990, covering only Washington state's prisons. It was 10 pages and hand-typed for 75 subscribers. It launched into the publishing world with a $50 budget. The organization was completely volunteer-run until 1996. The first run of six issues ended up becoming a 22-year, 224-issue run (and still going). Some of their earliest subscribers are still with them -- a great sign for the publication's longevity, but a less great reflection of these subscribers' sentences.

PLN's perseverance has paid off: In 1990, there were 30 or 40 prisoners' rights news publications, but many have since ceased publishing. Prison Legal News has expanded its coverage as its subscriber base expanded. At one point, they realized they had more subscribers in California than in Washington, and that they had graduated to a national publication. Yet Paul considers himself one of the few people in print publishing these days who welcomes competition. He wishes there were other publications and institutions engaged in this work.

Prison Legal News is not light reading -- there's no horoscope, no advice column, just hard news and information. But that's what their customers want. An annual reader survey draws a 30-40% reader survey response, and the feedback is consistently asking for more useful information rather than lighter fare. There was a publication in the 1990s called "Prison Life," which covered prison life and the prison experience, and they were somehow surprised when they were unsuccessful, because prisoners would rather not read about this in their leisure time.

An expansion into book titles has focused on self-help and non-fiction reference books for prisoners, especially titles that aren't viable for traditional book publishers. Paul mentions books including "How to File a Lawsuit and Win," and books on hepatitis C (a dangerous health threat within the incarcerated population). There's great interest in books on health, including "Our Bodies, Ourselves," which Paul notes has been banned in some prison systems. They also provide "radical critiques of the criminal justice system", including edited volumes titled "The Celling of America," "Prison Nation" and
"Prison Profiteers." Paul notes that the books reach a different audience than the magazine, that there are people who prefer reading the long form of arguments.

Who Reads Prison News?

Prison Legal News is a niche publication. It's not trying to reach the whole incarcerated population of the U.S. It's targeting activists and lifers interested in improving prisons. Paul said they want to reach the activists, the 1% of people who make change. Men are 95% of the U.S. prison population, and make up a higher percentage of PLN's readership compared with women. Paul attributed this to the fact that women generally receive shorter sentences, and their subscribers tend to have long sentences ahead of them. Paul has found that it's the people who are in prison for a long period of time that make things happen. These are the lifers, the ones filing the lawsuits and organizing other prisoners. These are people who have accepted that prison is their life now, and who are working to do something to improve it.

There are around 7,000 subscribers to the print publication, but the reach is much broader. Reader surveys suggest that copies reach more than 10 prisoners each -- Paul estimates a readership of 80,000-90,000 readers. Additionally, the website gets around 100,000 visitors per month. The subscriber base includes judges, court officers, lawyers, journalists and academics, including Noam Chomsky, who Paul told us proudly was one of the first subscribers. All the big investment banks subscribe, Paul told us, because they follow news on the private prison industry. "I was happy when Lehman Brothers went under, but we lost a subscriber," he said. Lehman Brothers had been one of the biggest bankrollers of the private prison industry, so it was a happy day when they went down.

Publication Litigation

A big focus these days is making sure the target audience in prisons can actually receive the magazine. This requires extensive litigation. Prison Legal News has obtained consent decrees in nine states, ordering state prisons to deliver the magazine. PLN is currently litigating in New York and Florida to enable subscribers to receive their publication, both the magazine and the books they publish.

Almost every state's prison system has censored and banned the magazine at one point or another, Paul told us. The organization has won nine lawsuits, receiving consent decrees that order state prison systems to deliver the publications. The bans are generally pretextual. They're bans based on postal rates used to deliver magazines, or whether prisoners are allowed to pay for the magazine from their trust accounts. Sometimes there are arbitrary blocks on sending publications to prisoners in certain types of custody. In Washington, PLN discovered they needed to become an "approved vendor" and had a very difficult time figuring out "who's brother-in-law we had to work with" to gain "approved vendor" status, Paul said.

It's not just PLN getting banned. In one case, in South Carolina, the American Civil Liberties Union had to sue when a prison banned all books except the Bible. These pretextual excuses can get pretty absurd -- Paul is currently facing an argument that the staples used to bind the magazine might be used as dangerous weapons. While we think it's funny, these are the issues PLN is forced to litigate (marshal the resources to sue the government, and win). "Think of every magazine held together by staples, delivered by mail. TIME, Newsweek. We're the only publisher in America who routinely challenges this censorship," he said.

Many of these rules are designed to prevent prisoners from having material to read, far beyond PLN's magazine. It would help if other American publishers would join in the fight to ensure publications are able to reach prison populations. When an Indiana judge upheld a ban on gay publications "Out" and "The Advocate," Paul asked the publishers to file suit, because it would stand up better in court than a suit from a prisoner. But publishers aren't seeking the prison population. "They tell us that they're not part of our targeted advertising demographic," he said. For PLN, the core audience is prisoners, and there's no point in publishing if the core audience can't get it. In recognition of this, they realized that funding staff attorney positions was a priority.

I noted that some critics of PLN have argued that it's as much a litigation platform as it is a publication. Paul countered that "our initial goal was always just to publish the magazine. But we got to to the point where we're just consuming ever greater amounts of organizational resources just getting the magazine into prisons." Paul estimated that he can spend as much as 40% of his time focusing on being able to distribute the publication, rather than producing and editing it. "The editor should be worried about being (an) editor, not worrying about why one prison system or another is censoring content," he said. For there to be any litigation, the government has to illegally censor the magazine, then PLN has to sue, and then they have to win. "If you don't like the consequences, don't break the law," Paul said.

Isolation from Society

Restrictions on what can be sent in and out of prison harm PLN in another way: It makes it very hard to hear from the incarcerated. In some prisons, prisoners can no longer send or receive information beyond what fits on a postcard. Other layers of draconian restriction include rules that postcard communication has to be in ink, can't use a label, etc. These mechanisms tend to be arbitrary and are designed, Paul argued, to prevent prisoners from having communication to and from the outside world. His organization has challenged a couple of these successfully, with a couple more pending. Paul told us that they are trying to nip this trend in the bud before it gets entrenched.

"Part of the goal is to get prisoners information. But conversely, we want to hear from them," he said. The bulk of the magazine's content is provided by contributing writers, who are mostly prisoners, some of whom have been working with PLN for over a decade. In the hopes of ensuring widespread distribution of the information, PLN doesn't demand exclusive publishing rights -- and people are free to copy and disseminate the information.

This is an area of close overlap with one of the Center for Civic Media's projects, "Between the Bars." BTB is a blogging platform for prisoners that gets around the lack of Internet access by scanning and publishing letters to a blog, and then mailing comments back to the authors on postcards. In addition to helping the incarcerated publish to the web, it helps the rest of the U.S. population by ensuring that we are able to hear from these voices, who comprise 1% of our entire populace.

Prison News Online

The Internet has greatly improved the visibility of Prison Legal News. Paul told us he conducts 3-4 interviews a week about the publication and the issues it raises. He's fluent in Spanish and noted that there's a great deal of interest in these issues from programs in Colombia and Venezuela. One of his associate gives interviews in Russian media, which seems to have an endless appetite for stories about the U.S. prison system. Some have observed that the U.S. prison system must be pretty bad when the Russians enjoy making fun of it.

The online presence of the magazine has allowed PLN to build a publication library online, with more than 6,000 documents available in its Brief Bank. "We've got the biggest, and I would say, the best, repository of prison documents online," Paul said. As a result, PLN generally shows up in Google's first page for prison-related queries, except sometimes when the "Prison Break" program is on TV.

Prisons can be a bit of a timeless place, said Paul, where the equipment you see is 50-60 years old. PLN's print publishing business still thrives here (advertising levels for the print magazine are actually going up), and web publishing is almost nonexistent. PLN hasn't figured out how to make money online, like other publishers. Its content performs poorly with online advertising. On the site, the news content is free, legal content is paid, and these fees cover basic staff time put into the site. Advertising and subscription income and book distribution bring in about the same amount. Payroll is the biggest expense. They get some foundation funding and donations, and when all of this revenue is cobbled together, it's enough to move forward.

Staying Human

The acts of reading and writing are core to helping prisoners maintain their humanity, especially when everything else in these punitive systems is working to degrade that humanity. A publication like PLN lets prisoners connect with others, when the rest of the system is designed to isolate and alienate.

Paul is wary of the dehumanization that takes place before genocides and in prisons. We lose sight of the people in prison. We need to keep in mind that they're someone's father, someone's son, regardless of what they've done. When someone's been murdered in a prison, it's almost always that person's mother who calls PLN.

Paul closed his presentation by noting that he's now 264 issues into this project, and that since 1990, "everything to do with the criminal justice system, by objective or subjective standard, has gotten worse."

Over the course of the next several posts, I'm going to give you an X-ray view into the guts of the OpenBlock application. Together, we're going to watch how data gets ingested and processed into information and insights that residents of rural communities can use to make decisions about their daily lives.

We knew basically two things when we started this project. First, we knew that public data went into OpenBlock. And we knew that digital public data is for the most part in a poor condition to be easily digested. Second, we knew that the meaningful geographies of rural areas were going to be different than the geographies of urban areas. But beyond that, the anatomy of OpenBlock remained almost completely obscured. No instances of the application lived in the wild, and the code itself was still missing significant documentation.

Step 1: Getting data into OpenBlock

Our first step was to figure out how to get data into OpenBlock. And while many of us probably think about data as being some sort of news event -- a transaction, creation, deletion, inspection, election, rejection, incarceration or some other function of government that takes place at a specific time -- the initial data you need for OpenBlock is about a location. You have to tell it where it is. And you do that by ingesting data from the U.S. Census Bureau.

The Census Bureau is an amazing resource of geographic data, which it calls TIGER/Line files. The bureau provides TIGER/Line files that show information about various "layers" of geography -- most of which aren't relevant to OpenBlock. Each layer actually consists of several files that you download from the Census website as a single zip file. And inside that zip file is a file with a .shp extension -- that's a shapefile, and it is the only one that has nutritious value as far as OpenBlock is concerned.

Several layers are important -- one file contains data about the boundaries of all ZIP codes in a state. Even though most ZIP codes aren't going to be relevant, you need them anyway. And while loading ZIP code shapefiles into OpenBlock may not be simple for people who don't have at least some familiarity with Django, it doesn't require a lot of human judgment. Plug in some code and you're done.

But nobody lives their lives by ZIP codes. We care about geographies like counties and cities and streets. And knowing which geographic data to load next does require some editorial thinking. Most of us are familiar with counties as political entities that have some meaning. But counties are different in each state. For example, in Virginia there is a City of Fairfax and County of Fairfax. Fairfax City isn't in Fairfax County or any county. They touch each other, but one does not have jurisdiction over the other. Now, here in North Carolina we have Durham County and Durham City. Durham City is inside Durham County -- mostly -- except for the part of Durham City that is in Orange County. Orange County is the home of Chapel Hill, except for the part of Chapel Hill that's in Durham County. And both Durham and Orange counties contain large parts of land that aren't in any city at all.

teaching journalism to programmers and vice versa

So when we talk about teaching journalism to programmers and programming to journalists -- this is really the kind of thing we're talking about. Somewhere here we have to have someone who knows the political geography of North Carolina and who can also describe the rules of that geography to a computer program so that it doesn't leave out anything it shouldn't and also includes everything it should.

Let's say we want to show the locations of all new business that have been incorporated in Orange County. The N.C. Secretary of State, which records new businesses, may have the address of the business, but not the county. We're going to have to tell the OpenBlock application which addresses are inside Orange County -- regardless of whether they are in Chapel Hill, Durham City or no city. If we tell it just to grab the Chapel Hill addresses, we will erroneously include the businesses that are in the part of Chapel Hill that's in Durham County. Or, since most people think of Chapel Hill as being an Orange County city and they might be confused if they know a business has opened but isn't listed on our website, we may want to tell OpenBlock to include all Orange County addresses, but not the ones that are in Durham City, and also include the Durham County addresses if they are in Chapel Hill.

And then we have to write for the reader an explanation of whatever assumptions we're making in our data, and it has to be brief and clear.

Once you've made your editorial decisions, county bounders can also be downloaded from the Census Bureau as well. Just to be proper, Census calls this layer "County and Equivalent."

Cities, however, are a more delicate matter that again require some knowledge of Census terminology and local political geography. There is no Census layer called "cities."

Until recently, the practice among the OpenBlock community had been to look for geographic information about city boundaries from local governments. Most county governments are pretty good about publishing their geographic data on the web. Many large public universities have a GIS (geographic information system) section in their library, such as this one at N.C. State University that indexes links to the geographic data source for many of the state's county governments.

lesson learned: local data varies

The problem with local data is that poor and rural counties are less likely to have the online GIS data. And you will also find yourself dealing with a wide variety of standards. We spent a few days flummoxed by our inability to load Columbus County data. For whatever reason, the county decided not to include the necessary .prj projection file you need to make a shapefile work in OpenBlock. It's a lesson that's going to be important for us to remember throughout this project -- local data varies widely in quality. And sometimes it's not obvious to the layperson's eye what is missing.

Our thinking right now is that we're going to be able to turn back to the Census Bureau for city information. But we're not using the Census layer called "Consolidated Cities." Nor the one called "County subdivisions." We're using the "Places" layer.

You can read more about our "city" solution on the OpenBlock discussion group. But there are a few bits worth mentioning here:

• Places do not cover 100 percent of a county. So we're going to need to be on guard that events happening outside the boundaries of one of our places doesn't somehow get left out of the search results.
• Some of the places in Columbus County probably have little or no meaning to the audience there. In the "Government" navigation of our partner site, Whiteville.com, not all of the Census places are listed. That said, several of them show up occasionally as the location of obituaries that run on the site.

Geography is hardly the dynamic data we think of as news. But we've already seen several road bumps that are big enough to deter almost all small news organizations from using the OpenBlock application. So post this as one of our goals: to automate the process of loading basic ZIP, county and city geographic data about your community. Again, you can read more in the OpenBlock discussion group about how we might use FIPS codes -- the unique number given by the Census Bureau to each state, county and "place" -- to do that.

And there's still a big elephant in the room -- the geography of "blocks" that is one of the core concepts of OpenBlock. That topic is big enough for its own post a few weeks from now. Before then, I'm going to walk you through some of our experiences loading "news" data into OpenBlock and how we're hoping we might be able to work with fellow Knight News Challenge winner ScraperWiki to build our own API for North Carolina state government.

Understanding each step of the set-up, production and editing process involved with OpenBlock is critical to our ability to describe the expense side of the equation, which we hope and expect will lead to the financial viability of the application as a tool to fill the information needs of rural communities.

Marco Kremerman, an investigator from SOL, a Chilean foundation that conducts research about the labor market, recently published a column in which he stated that 4,000 families run Chile.

Kremerman's post was highly controversial among Chile's elite, an endogamous group of power players, little accustomed to public scrutiny and not fond of being forced out in the open. It also caused some deep rumblings among the middle class, during a time when issues such as inequality and the extreme gap between the rich and poor have become a matter of national and international public interest around the world.

One of the things we loved about Kremerman's work is that he came to his conclusions using public data from the latest National Socioeconomic Characterization Survey (CASEN in Spanish), one of the public data sets we're using in Poderopedia to establish a database of who's who in business and politics (based on the fact that in countries like ours, where you are born, your last name, and which school you attended very much define who you are in the scale of power).

WHAT IT MEANS TO BE PART OF THE 0.1%

Four thousand families might seem like a lot, but that 0.1 percent of the richest households (in a country of 17,094,270 people) is part of the 10 percent that runs practically everything in Chile.

According to the 2009 CASEN data extracted by Kremerman for his post: "4,459 families have a monthly average income of $19 million pesos ($37,300). This is 0.1 percent of the richest households, who generally tend to under-report their income in such surveys."

Being part of the 0.1 percent means that you might be our president or in the annual Forbes billionaires list, and that you either own one of the top 100 Business Groups or work for one of them, according to the Superintendency of Securities and Insurance (our SEC).

It also means, as Kremerman put it, that you probably own banks, insurance companies, retail stores, pharmacies, private pension funds, private health insurance companies, mining companies (the heart of Chile's economy), soccer clubs, forestry companies, fishing companies, or big media.

So what if we go deeper and see what being part of the 0.1 percent means to the rest of Chilean citizens? How do the 0.1 percent decisions affect people's everyday life?

Kremerman did the exercise. Here goes -- Chile's 0.1 percent decide:

• The interest rate you end up paying with a bank for consumer credit.
• The excessive charges on electricity, water, telephone or gas bills.
• The supermarket account you pay each month.
• The return of your pension fund (during the first half of August 2011 the Pension Funds had lost more than $8,000 billion pesos, or $15.4 billion, equivalent to more than 6 percent of the total funds).
• The value of your private health care insurance plan.
• The credit card interest charged by retail stores when you have to buy clothes or any appliance.
• The harsh conditions for small businesses that operate as a supplier, contractor or part of the business chain of large companies (sometimes waiting 120 days or more to receive pay).
• Bus fare and plane ticket costs.
• The percentage of fish available for artisanal fishermen.
• The shows and news you see and hear on TV and most radio stations.
• The editorial line of the biggest media companies.
• The fee you pay in a private school or university.
• The players that are hired by your favorite soccer team.
• And, of course, the possibility of not having universal quality public education and health because they don't want to pay more taxes (one Warren Buffett idea they failed to like).

The 0.1 percent make up the core of Poderopedia's who's who database. But in our research, we're discovering that the power map of civic, business and political leaders, as well as companies and organizations, covers the top 20 percent of the country with links in each one of the sectors mentioned above, similar to the well-known Pareto principle.

UNDERSTANDING OCCUPY CHILE

The Chilean economy is often applauded by the media and think tanks. But you don't read as much about the gap between the richest 10 percent and the poorest 10 percent.

In a recent book, however, economist Cristóbal Huneeus and Andrés Velasco, former Harvard teacher and ex-minister of Finance, demonstrate that the income of the richest 10 percent is 78.5 times greater than that of the poorest 10 percent today.

If you break down the numbers, like Kremerman did, it means that "Chile has an average per capita GDP of $32 million pesos ($64,000) a year for a household of four people, but over 80 percent of Chileans live in a household with an annual income not exceeding $10 million pesos ($19,000)."

It also means that "the richest 5 percent of the population generates income 830 times more than the poorest 5 percent," Kremerman wrote. And, according to the 2009 CASEN, the poorest 10 percent live with 1.5 percent of the country's total revenues, and the richest 10 percent represent 39.2 percent of revenues.

In addition, Chileans on average owe 7.5 times more money than what they make each month, so many are afraid of losing their jobs or becoming ill. That makes it easy to understand why long before Occupy Wall Street began, we had thousands of students and citizens protesting for free quality public education, with nine out of 10 Chileans supporting their protests.

But on the other side of the food chain, many business associations, lobbyists and large funders of political campaigns continue to reject a tax increase on the rich and oppose structural changes that would break the asymmetry of power between the major economic groups and ordinary citizens. 

Many problems arise when your country's legislature is consistently more responsive to its donors than its constituents. One of these problems is that simple good ideas can't just be left alone to bask in their goodness.

The Internet is clearly a good idea -- not tautologically good, but certainly one of the better things that's happened to human communication and the spread of knowledge in recent centuries. But now some people in Congress who didn't know what an MP3 was until their granddaughter got an iPod a few years ago, want to go and ruin the web to benefit a few reactionary trade groups who would prefer censorship to innovation. A bill that was introduced into the House last month, called the Stop Online Piracy Act (SOPA), aims to penalize or eliminate websites that have pirated content, and the repercussions for Internet users could be far-reaching.

The reaction online has been one of the largest upswells of traditional advocacy by web-native organizations in recent memory. Ever heard of Facebook, Google, Twitter, eBay, LinkedIn, Mozilla, KickStarter, Yahoo, AOL or Zynga? They're all opposed to the bill.

Google's fighting the good fight within the halls of Congress, where its representative was the only opposition witness allowed before the House Judiciary Committee. 4Chan, BoingBoing, and other top web properties converted their home pages to CENSORED home page takeovers, offering priceless in-kind advertising to the cause.

The result? Six thousand websites participated. One million emails were sent to Congress -- and 3,000 handwritten letters.

Tumblr takes it up a notch

Tumblr took even more dramatic action as far as getting users' attention, and redirected the roughly 500,000 daily unique visitors to Tumblr.com to a slick "call Congress" tool that dialed users, prompted them with talking points, and connected them to their representatives.

I need to take a minute and let you marinate on Tumblr's part in all of this. The service combines Twitter and blogging and has grown 900 percent in the last year. With 30 minutes' notice, Tumblr got hooked up with Mobile Commons, another New York-based start-up. And then they delivered an average of 3.6 calls per second to Congress. Because Tumblr is a blogging platform, its action also produced a sharp uptick in blog posts about SOPA.

The tool the Tumblr team built made me a little happy and a little sad. I was happy because it was perfectly executed. The interface was nice; it was blatantly clear what I was supposed to do; and it got my complete attention until the task was completed. But it also made me sad because I've been watching political technology for 10 years and have never seen anything nearly this good from the industry vendors who charge campaigns and non-profits significant sums of money for their clunky click-to-call tools.

It appears that Tumblr built in a day or two what no D.C.-based technology supplier could come up with in the last five years. The closest I've seen to Tumblr's tool was a short-lived but great "Whip Congress" tool built by Change Congress and a couple of Google employees. It provided a nice overview of which members of Congress to thank and which to spank for their stance on a bill.

Tumblr ended up routing 87,834 calls to representatives, for a total of 1,293 hours on the phone. For those of you who haven't worked in advocacy at the federal level, members of Congress pay a lot more attention to phone calls from constituents than emails or petitions.

But the bill lives on.

protecting free speech

Even if this bill is defeated, what has happened this week is really important for the protection of free speech online (which is, in spirit if not in law, identical to the protection of free speech in general).

We take for granted that great things like the Internet exist (and frankly, we should be able to). The problem with the U.S. Congress is that if a tiny, tiny minority of people doesn't like something (like the open Internet), and they give lots of money to key members of Congress, their opinions suddenly trump the vast majority of citizens, who didn't realize they'd have to fight for something that's so obviously great and well-loved. In this case, pro-SOPA groups like Pfizer and the MPAA have given 12 times the amount of money to members of the House of Representatives as web companies and consumer groups:

So, in addition to paying more attention and donating LOTS more money, the people who want to ruin the Internet also have the advantage of surprise. Net-savvy individuals first have to find out about the threat to the web, and then they have to overcome the counterintuitive logic that something as brilliant as the Internet is being fundamentally threatened by the people who just got around to figuring out that Facebook and its 800 million users might be a good place to rent out Batman movies.

It's vital that even consumer-level websites are getting political right now. We need them to win this battle. And we need everyone who loves the open web to be relatively engaged in protecting it, at least when ideas as terrible as SOPA gain traction.

The same problem occurs on issue after issue. No one stands up for great things we all take for granted, because who would mess with great things? There's no real money or lobby behind protecting free speech, just some ideals and financially struggling non-profits. If it weren't for the courts, free speech would be a distant memory at the mercy of some industry inconvenienced by it.

It's really, really helpful when major and relatively apolitical sections of American society suddenly pay attention and push back on Congress. Many terrible ideas are advanced in the halls of Congress all the time, but they depend on the majority of us not paying attention until it's too late and they've already become law.

The battle lines have been drawn, and those on the side of an open Internet, and free speech in general, need to stand up. This week, they have. But the opposition is heavily funded and well-organized. Preventing the sabotage of the Internet will take more than some clever Javascript site takeover code. It'll take phone calls and long-term organizing and building support and paying attention during non-crisis moments.

Fortunately, some groups have started this process. I'd recommend at least joining their email lists and following them on Twitter, as these groups are working very hard to keep the Internet the Internet:

• Electronic Frontier Foundation
• Public Knowledge
• Center for Democracy & Technology

A video to reward you for reading all of that text:

PROTECT IP Act Breaks The Internet from Fight for the Future on Vimeo.

Web results screenshots via americancensorship.org. Bar chart via MAPlight. (More on who in Congress got that money here.)

"Sure, you can buy the bird," I told him. "But what are going to feed it?"

With the launch of our OpenBlock project in North Carolina, rural newspapers from across the state have called or emailed to express their interest in getting our help installing and using the application. Installing the application isn't much of a challenge, I tell them, but what are you going to feed it?

OpenBlock, a "hyper-local news" platform, is a beast that eats data. So before we can make the Tar Heel State a good breeding ground for the application, we're setting out on a digital public records census. We aim to figure out how well city and county government agencies are living up to the recommendation of the Knight Commission on the Information Needs of Communities that "governments at all levels should ordinarily collect data electronically and in standardized formats."

Unlike many similar audits of public records that have been done by the Associated Press and others, this isn't some sort of exercise to see how governments comply with state and federal open records laws when they think they aren't being watched, so we're happy to describe here how we're going to go about gathering our data. In my dream world, the N.C. Association of County Commissioners sends out a link to this article to its members and implores them to help.

We're focusing our census on a few of the public records that rural newspaper publishers and editors have told us will be most valuable to their readers and advertisers -- births, deaths, land transactions, crime reports and health inspections.

Crime reports are particularly interesting. We know that people love police blotters, but also have real concerns about the safety of victims and the fairness of the criminal process. We know that state and federal agencies collect crime information in digital formats, but it's old and aggregated so it no longer has news value by the time it reaches that place in the information food chain.

To properly gauge the state of digital police records, we have to go to the city and county level. So our first step was to try to find or create a comprehensive list of every law enforcement agency in North Carolina that might generate incident or arrest reports. Thanks to a great report that a state agency submitted to the legislature earlier this year, we have the names of 569 police agencies.

From there, we're in the process of tracking down the website addresses of each agency to examine whether they publish incident and arrest reports there. (We will publish that list shortly, and may ask for your help filling in the blanks.)

Taking an 'Element' State of Mind

The bad news is that there's no indication we'll find a single agency that produces reports in a GeoRSS feed. The good news is that most police departments in the state appear to use a relatively standardized paper form to record police incidents and arrests.

In most cases, we can at least get those pieces of paper. But we've already run into cases in which police departments are unwilling to turn over standard incident reports without first heavily redacting them with misused citation of the state's open records law.

We're interested in the financial viability of OpenBlock, and paper records raise the cost. We'd have to pay people or recruit reliable volunteers to gather the paper records, scan them, and upload them to a DocumentCloud service that could use the layout of the page to extract editorially meaningful elements such as the date, time, location, and description of each document. That becomes almost impossible if we run into handwritten paper reports, which would force us to re-key the documents using local volunteers or perhaps something like Mechanical Turk.

For our census, it is not going to be enough to report that police records are online or offline, or that they are digital or not digital. We really need to be able to describe the format, location and timeliness of each data element. Taking a look at the website for the Winston-Salem Police Department, gives a good idea why we have to get more granular than the "documents state of mind" of traditional investigative reporters.

Winston-Salem publishes to its site what amounts to an index of incident and arrest reports. Each record includes the date, time, "type," case number, "primary offense" and "location." But for incident reports, it also links to a fuller record that provides information that's important for readers and reporters who want to determine the relative news value of each event -- data elements such as whether a weapon was used; the name, age, race and gender of the victim; whether drugs and alcohol were involved; whether anyone was injured; the amount of time the crime went unreported; and descriptions of the items that were stolen.

But missing from even those fuller records are data elements that would be useful for journalists who want to report trends and patterns rather than simple events. Some of the data is omitted with claims of too vague "information security purposes" and other data is omitted because of technical limitations of the departments' digital records management systems.

Each element brings with it a different cost of transforming it into a complete and current digital public record.

The variety of formats that our initial tests have already turned up seem to be limited. We've come across PDFs on the web, Word documents delivered daily via email, HTML tables, CSV file on the web, CD and via email, and the Mac-proof SNP filetype courtesy of Microsoft Access.

Most of these digital formats are created for police departments by one of three vendors that have a corner on 95 percent of the market. But we also know that 15 percent of the state's police agencies -- covering 1 percent of the population -- maintain no digital records.

police records play a key role

Police records are far from the most important -- and have proven throughout the history of this and other similar applications to be the hardest to get. But they play a key role in determining the viability of OpenBlock at rural papers. When compared to other interesting public records such as real estate or health inspections, there are simply more police reports that come out more often than other record types. Volume and frequency drive most common measures of audience engagement such as time on site and return visits.

OpenBlock is a hungry animal, and we've got to find a way to help rural papers feed it without going broke. That's the whole point of our census.

As we set off on our survey, we'll report findings and failings here. We're beginning to imagine some interesting things we'll be able to measure once we have a fuller picture of the state of records in North Carolina.

In the meantime, let me know what your experiences have been gathering digital public records at the state, county and city level. Share your experiences with @OpenRural Twitter and I'll re-tweet them. I've got lots to learn from you as well.

I asked one high school social studies teacher we got in touch with, Jack Buckley at nearby Cohasset High School in Cohasset, Mass., how he would use OpenCourt in his classes. He teaches an elective called "Intro to Law" that he says is like a traditional civics class -- save for the fact that he brings police officers and legislators into the classroom and takes his students out on field trips to the local prison and court.

In the past, he's brought his students down to Quincy District Court to observe. With the live-stream, the court can come straight into his classroom.

After looking through our website, he also said that First Justice Mark Coven's interview was a "gold mine" for a public policy assignment he does during the second half of the course, after students have a basic grasp of the branches of government. He asks students to select a public policy issue and to research the government's response.

From his email (edited for clarity):

Students find the assignment to be difficult; I think I can now make it a heck of lot easier to understand by listening to Judge Coven's explanation of a district court as a place where needed human services can be accessed in balance with dictates of the law and the needs or considerations of victims. For students, he makes it pretty clear that the bulk of legal problems are rooted in social ills -- drugs, alcohol, violence, and financial problems. That's pretty easy for all students to understand, and sadly, for too many students, their personal lives are touched by these issues.

So, in short, my assignment on public policy and government response will be to listen to Judge Coven's interview, select an issue he mentions (or a related issue such as possession of a handgun) and research the problem as it manifests itself in our county, i.e., Quincy District Court! Then I ask students to begin to piece together local instances, MGL [Massachusetts General Laws] statutes, federal laws, acts of Congress, etc.

This is just what we want to hear. We'll be very happy if OpenCourt can make the abstract ideas of justice, public policy, and social services more real to local high school students. We are also working with the organization Discovering Justice, whose educational programs have reached almost 100,000 students.

If you have any ideas or feedback for our site on how we can be more of a resource for schools, please drop us a line.

This is an America where people still read and trust the local newspaper, where print advertising hasn't completely migrated online. But it's also an America where reporters keep their crime database in a 300-page Word document. It's a place where the inspections department doesn't have a live XML feed, and where they may even be a little reluctant to give you paper copies if they don't recognize your face. And it's a place where a good Python developer is darn hard to find.

OpenBlock Rural is going to help these rural newspapers get ahead of the oncoming wave of digital interlopers by lowering the cost of deploying OpenBlock and using it as a tool to engage younger audiences, as well as increase advertising revenue.

Ultimately, we need to help figure out how to give rural North Carolinians -- about a third of the state's 9 million people -- the news and information they need to take advantage of life's opportunities and to participate fully in our system of self-government. This is but one of the ways that the project's goals overlap nicely with the Knight Commission on the Information Needs of Communities in a Democracy.

For us and our focus on OpenBlock, that's going to mean finding a way -- or, more likely, many ways -- to acquire, organize and produce relevant government data. While getting data into OpenBlock, and publishing it in a way that makes sense for rural areas, may have some unique technical hurdles, I don't expect the technology of scraping a site in Washington, D.C., to be much different than scraping a site in Washington, N.C.

how to produce high-quality public records

The real challenge is going to be to open government data inexpensively. Small newspaper staffs do not have access to $100-per-hour programmers. Done well, our project will show both rural journalists and county governments a way to produce high-quality -- more in a later post on what that means -- public records. My expectation is that we'll be able to recycle one or two efficient methods of getting data into the OpenBlock application. For example, just three vendors supply the record management systems to 95 percent of all police agencies in the state. So rather than finding 500 different ways to gather incident and arrest reports, we should be able to come up with three templates.

An inventory of digital public records will be one of the first things you will see from this project. Eliza Kern, a senior journalism major at the University of North Carolina and the student leader of ReeseNews.org, is already hard at work on a research project that will eventually yield a website with the locations and descriptions of local digital public records in North Carolina, as well as a report in which we're going to put a dollar figure on what it will cost private publishers to acquire and convert records.

Whatever the cost of records management, it's going to need to come in far below the amount of revenue we aim to help rural newspapers generate from this product. As I'll describe later, OpenBlock is but one piece of a digital revenue strategy for rural newspapers that my colleague Penny Muse Abernathy, the Knight chair in Digital Media Economics here at UNC, has developed with the help of her undergraduate and graduate students. (Read more about it in the textbook and workbook.) As part of this Knight News Challenge grant, she will be training the ad sales staff at our partner newspapers on how to create and sell opportunities for local advertisers to sponsor this application.

With revenues outpacing the operational costs of each rural OpenBlock installation, profits will be available to produce analytical reporting that builds and sustains an informed community. The data we free up for use in this project should allow both journalists and their readers to ask themselves questions that are vital to economic and cultural development -- how are we doing compared to other communities like us, and are we heading in the right direction?

Using hard data to answer hard questions

And just like those communities, we're going to use hard data to find the answers to difficult questions we'll face throughout this project.

Questions such as:

• How does OpenBlock deal with a rural county that has 17 local governments, including a tribal government?
• What geographies are important in sparsely populated rural communities, and how do we display them in a meaningful way?
• In communities where everyone knows everyone, does "public information" take on a different meaning when it gets published online?
• Can a relevant dataset foster technical innovation and STEM education in rural communities?
• What role will broadband penetration and mobile Internet play in the user experience of rural OpenBlock?
• How can UNC's Journalism & Mass Communication students continue to be cross-discliplinary leaders in editorial product development?

I'm looking forward to your questions as well. Send them my way at @rtburg or @openrural, and watch this space as we learn and share.

As Knight News Challenge winners FrontlineSMS, Ushahidi and NextDrop have shown, mobile communications are crucial for citizens living in rural areas, where being able to reach other people and access relevant news and public services information make a huge improvement in people's lives. Plus, cell phones are tools that most already have.

THE PITCH

What if, apart from efforts to widen connectivity in isolated areas and government programs to provide computers for schools in rural areas (which has been a very good, but slow, undertaking, and not an attractive business for telecom companies), governments of underdeveloped countries create and provide easy ways to access public information and services on mobile phones with an application or open-source web app that could be downloaded from government websites (in Chile it's Gob.cl)? Or cellular service providers could pre-install an app or direct access to a web app on every smartphone or other devices?

This could mean a great deal for people, particularly in rural and impoverished areas where the biggest news is not what's happening in Congress or the presidential palace, but what is happening to you and your community (something Facebook understood very well in its latest change that challenges the notion of what is newsworthy -- but that's a topic for a separate post).

People could do things like schedule a doctor's appointment or receive notice that a doctor won't be available; find out about grants to improve water conditions in their sector; receive direct information about training programs for growing organic food and the market prices for products they might sell; find out how their kids are doing in a school they attend in the city or if the rural bus system will go this week to the nearest town or not. These are just a few very straightforward examples of useful public services information that could be available on people's phones. Such availability of information could save time and money for those who lack both things.

I know it because I saw it as a boy growing up in a small town -- and as the son of a farmer who still hasn't gotten around to the idea of using a computer, despite having the chance to use one. But because my father owns a mobile phone, he's become an expert user of SMS and applications that allow him to check weather conditions.

WHAT'S IN IT FOR THE TELECOM COMPANIES

At the same time, telecom companies could support this initiative by providing mobile Internet connection packages and a free SMS service for rural areas by which citizens could specify their information searches or requests (a kind of help desk). Why would they do it for free? Because with each free transaction, there might be another one that has nothing to do with the government or public services information, which may produce additional income. It might also improve the companies' public image.

Another way of getting support from these companies consists of giving them a
tax reduction for providing the service and automatic updates of information. Thus, rural citizens living in small towns and cities would be able to access the data they need (pension reforms, hospital appointments, housing benefits, food grants, etc).

IN SIMPLE WORDS

To do what we're talking about, we need clean and intuitive interfaces with super-simple steps and strong government websites or apps that learn from the end users' needs, systematizing:

• Databases containing questions and answers made by ministries and government staff.

• Services citizens can access in order to ask for all kinds of information: subsidies, hours of service, etc.

• Simple and complex procedures, so that answers can be delivered accurately and in the shortest amount of time.

This reduces the margin of error, maximizes human resources -- decreasing the man-hours needed for searching for requested information -- allows specific departments to detect questions which are more usual, and meets the needs of users and citizens.

However, in order to make citizens understand the information, it has to be written in a simple way, with no illegible technical or legal terms. For such a purpose, there are citizen language manuals that standardize response criteria issued by the state. (A good example of this in Spanish is the Mexican Lenguaje Ciudadano government guide.)

This is a small civic proposal to start a wider conversation and brainstorming and discover projects and ideas that may already be addressing this issue. Please feel free to post your tips and thoughts in the comments section.

Image of Santiago, Chile by Flickr user Cleanie.

NextDrop, which informs residents in India via cell phone about the availability of piped water, has been fortunate enough to have the full and sincere cooperation of Chandru, one of the best valvemen in Hubli. It's incredibly helpful to work with someone so willing to share the know-hows of the water system here, because the on-paper description of the process is much different than what actually goes on.

Chandru let us tag along as he opened and closed a myriad of valves in his water areas, providing service to residents who (after more than 20 years of seeing him around) know him by name. As usual, we learned much more than we expected -- and we're sharing that process of discovery because it's interesting, and even lots of fun.

A day in the life

From what we could tell, Chandru's service period starts with a call from the section officer and lasts around 48 hours. The section officer will tell Chandru that it's his turn to provide water to his residents. This call is actually pretty important, because different valvemen in Hubli take turns providing service to their areas to maintain ample water pressure -- or no one will get water. After this call, Chandru has around 40 valve areas to give water to, each with its own valve. He'll usually open valves for three or four areas at a time, and leave them on for 4-5 hours each.

As we followed Chandru on his rounds, we learned that water pressure is incredibly important for water delivery, and is one of the reasons service is provided erratically at times.

Lingaraj Nagar North, for example, is located uphill, so Chandru leaves this valve open for 7-8 hours instead of four. But, because of the pressure needed for the water to reach up the slope, residents in Lingaraj Nagar North only get 3-4 good hours of water (when some other valves have been closed). The distribution within Lingaraj Nagar North itself is also varied -- people living more uphill get less water than those living downhill near the supply. In cases like these, NextDrop can send a notification that the valve has been open, but some residents won't actually get water because of insufficient pressure.

In the opposite case when an area is located more downhill, instead of shortening the time a valve is open, Chandru will only open a valve two-thirds of the way for the same amount of time. This is also to regulate the amount of pressure so that the pipes don't burst due to excess pressure. The pipes are really old. If a customer calls Chandru to complain about lack of water, he can open the valve a little bit more or longer.

We should mention that Chandru doesn't use any fancy instruments to measure the pressure in the pipes underground. He depends on none other than the rod used to open valves -- and his ears! The valve itself is at the end of a small 2-foot deep shaft, reachable only by a rod.

The sound of water

Chandru will put his ear to the end of the rod and gauge the pressure by the sound of water rushing by. It's actually quite loud when the sound travels to the end of the rod. We got to have a listen ourselves.

We also got a tour of the old water tank in between valves.

The tanks have supplied water to all of Hubli for 99 years. Its 100th birthday is next year. The tanks are a pair of 20-foot-deep underground structures that each have a huge valve opening at the bottom, where lake water pushes upwards. The valve openings are about the size of sewage openings in the United States.

These two wheels open the valves, which currently work to drain nearby Unkal Lake.

The goal is to reach the mud at the bottom of the lake for construction, and next monsoon season will fill the lake back up.

Along for the ride

We weren't the only ones shadowing Chandru.

Chandru's been a valveman for over 20 years now -- he's one of the best. So far, he's also only taken two sick days -- in his entire career of working for the water board. The water board makes it a point to have someone shadow Chandru for when he won't be around to run the water delivery for a huge number of people. Ravi, the water board employee who usually repairs aged pipes, was also along for the ride watching Chandru open and close valves for different areas.

There's a lot of information to be learned for this job, but no formal training program. The day-to-day details of running water deliveries often isn't known by those who aren't out in the field. A lot of information gets lost in between. Watching is the best way to learn, which is why we're trying to go out and watch the valvemen ourselves.

What we need to understand is how things currently work, so that we can make it as easy as possible for valvemen to adopt the NextDrop system. To us, it looks like organized chaos. Somehow (we're not quite sure how) progress is being made with text messages and everyday processes. But there's definitely a method to the madness, and we're just starting to understand it. Days like today grant us a lot of insight into how to move forward.

You can read about more of these efforts on our NextDrop blog.

Since it looks like I'm going to be one of the lucky ducks who gets to hack away on my proposed idea in Berlin this September, I wanted to share it here. I would love feedback, of course, but also if you know anybody who might be interested in incorporating their content or using the platform, I would love to get in touch.

Part 1: Introduction

Scraping, Slicing, and Scrubbing C-SPAN

How do you get from a TV channel to a rich video archive, and how do you get there automatically? The goal of my project, called ATTN-SPAN, is to convert C-SPAN into a series of overlapping video segments that are identified in terms of state, politician, topic, party, action, and legislative item. Some of this is straightforward, and some of it might be impossible, but here's an overview of the planned nuts and bolts:

1. DirecTV offers TV content in a format that's easy to record digitally, and VLC is a free tool that can do that recording. Combine the two, and we can download C-SPAN streams into individual files that are primed and ready for analysis.
2. Once a video file is in our clutches, we can use VLC once again to separate out the video from the Closed Captioning transcript.
3. Now we have a transcript and a raw video file. Next, we register all of this information (in a database) so that we can look it all up later, and then convert the video file into streaming-friendly formats and store it alongside the original recording.
4. C-SPAN consistently shows a graphic on the bottom of the screen that says who is talking, their state, their party, and what is being debated. By using a technique called Optical Character Recognition (OCR) we can pull this text out of the video image. Once pulled, we can add that to our database so that we can access all of this information for any moment in the video.
5. At this point, we have most of the information we need, but there's still room for fine-tuning. We can use audio levels and the closed-captioning transcripts to try to identify moments of inactivity, normal dialogue, and heated dialogue.

These steps are enough to split up and categorize C-SPAN footage into an organized video database, but there are still more ways to flag special moments in the footage. For example, we may want to identify changes in speaker emotion in order to give our algorithms the ability to craft more engaging episodes. This is possible through the work of Affective Computing at the MIT Media Lab, a group which has developed several tools that perform emotional analysis using facial recognition.

We may also want to identify specific legislative action (e.g., "calling a vote"). This could be accomplished by looking for key words in the transcript (e.g. "call a vote") and possibly through common patterns in the audio signal. (Maybe there are identifiable sounds, such as a gavel hitting the table.) Both of these concepts require additional research.

Creating a Profile and Constructing an Episode

If video events are the building blocks, then viewer interests are the glue. The creation of a personalized episode requires two things: a user account and a context. The user account provides general information like where you live, what issues you have identified as important, and (if you are willing to connect with Twitter or Facebook) what issues your circles have been discussing lately.

The context comes from time and cyberspace. Every night, after Congress closes their gates, your profile is used to create a short, rich video experience designed to contain as much relevant content from that day as possible. At this point, you might get an email begging you to watch, or maybe you log in on your own because you're addicted to badges and points and you want as much ATTN-SPAN karma as you can get.

There's another way to access this content though, and that is through the websites you visit anyway. Imagine if you could read an article about the national debt on The New York Times (or in a chain email) and actually see quotes from your own senators in the report. What if you could supplement the national report with a video widget that lets you browse what your House members had to say when they controlled the floor during the debt debates?

From a technical perspective, this isn't that far-fetched. Truth Goggles, one of my other projects, is a bookmarklet that will analyze the web page you are viewing, fact-check it, and rewrite the content to highlight truths and lies. This impossible feat is fairly similar to what I'm proposing here.

Adding Rich Information

Once an episode is pieced together, we can look up the information surrounding the video to know who is talking and what they're talking about. What else can be added, and how do we get it? Existing APIs (application programming interfaces) offer some good options:

• Contact Information - Thanks to the Sunlight Labs Congress API, it's possible to get the contact information for any member of Congress on the fly. Thanks to VoIP services, it's possible to create web-based hooks to call those people with the click of a button.
• Campaign Contributions - The New York Times offers a Campaign Finance API which can help you understand where the person onscreen gets his or her money.
• Voting Records - The New York Times also offers a Congress API that will make it possible to know vote outcomes from related bills as well as information about the active speaker's voting records.
• Truth and Lie Identification - My Truth Goggles project can be easily adapted to work with snippets from video transcripts. This will allow ATTN-SPAN to take advantage of fact-checking services like PolitiFact and NewsTrust.

This is a good start, but I would also like to show links to related news coverage and create socially driven events based on community sentiment (for instance, to track moments that caused people to get upset or happy). This won't come for free, but it should be accessible given the right interface design.

Part 3: A Note to the Newsies

So that's the idea and the plan. What's the value?

It seems plausible that ATTN-SPAN, a system that analyzes primary source footage and pulls out any content that's related to a particular beat, could be useful as a reporter's tool, but what about your subscribers? ATTN-SPAN can augment an individual article so that it hits everybody close to home. Suddenly, one article becomes as effective as two dozen. Moving past text, for larger organizations with a significant amount of video footage, ATTN-SPAN can be tweaked to use your programming instead of (or in addition to) C-SPAN.

At this point, I have to warn you that this isn't the first, nor will it be the last, project to work with C-SPAN. A 2003 demo out of the Media Lab used C-SPAN as one of several sources of information in a platform aimed to provide citizens with Total Government Awareness. Metavid, the platform I used in my initial prototype, already makes C-SPAN more accessible by enabling searches and filters. The list surely goes on.

So why is this a more powerful project? Well, the real goal of ATTN-SPAN isn't to get more people watching C-SPAN. In fact, I tricked you: This project isn't about government awareness at all. It's actually part of an effort to make indisputable fact ("blunt reality" and "primary source footage") a more prominent part of the media experience without requiring additional effort from the audience.

Newsrooms do an amazing job of reporting events and providing insight, but for deeper stories, there simply isn't enough time or money to cover everybody's niche without going beyond the average person's attention span.

Thus ends my pitch.

The code for both prototypes mentioned in this post can be found on GitHub: ATTN-SPAN and Truth Goggles. Please forgive any dirty hacks. I would be thrilled if anybody wants to offer suggestions or even collaborate. On that note, please get in touch on Twitter @slifty. This proposal was originally posted on slifty.com.

This tight group, which the media has dubbed "Piñera's business West Wing," was composed of Piñera's brother Pablo, CEO of the state bank BancoEstado; his top adviser and current Interior Minister Rodrigo Hinzpeter; and seven high-profile businessmen who own some of the biggest investment banks, pension funds, private health care companies, airlines, and TV stations, among other things. Many are also board members in some of the biggest economic groups in Chile. Most went to school together, spend summer vacations with each other, and are in the same gentlemen clubs and think tanks.

When Piñera was elected president in January 2010, his friends were there to greet him. And despite now being entrenched in politics, the president's ties to the business class remained almost untouched. Since then, conflicts of interest have become one of the government's biggest headaches, and Piñera's credibility, according to surveys, is in tatters.

Piñera's business West Wing is just one example of who and what kind of connections Poderopedia intends to keep track of in order to promote greater transparency and accountability in Chile.

Poderopedia will be an editorial and crowdsourced database that visualizes the relationships among Chile's political, civic and business leaders, shedding light on any conflicts of interests, so you can understand why things happen. Crowdsourced information will be vetted by professional journalists before it's posted. Entries will include an editorial overview, a relationship map, and links to sources of information.

SHOW, DON'T TELL

In countries like Chile, as you can see in the slides above, your last name and where you are born define your chances in life, because the elite are very much inbred. All this determines who becomes powerful and influential in the business and political world, usually affecting (for better or worse) the course of public affairs, government decisions, and bills approved in Congress.

Journalism has always been about asking the right questions at the right moment with the best contextual information in order to dig deeper and discover new things that are relevant to the public. But with an overflow of information, it's hard to distinguish between what's urgent and important. Sometimes media companies don't have the time or money for it. Many don`t care. In other cases, editors are just worried about tomorrow and think of developers as the guys who fix their computers.

In this mega real-time data ecosystem, there is a big opportunity for curating, sharing, hacking and linking, as writers Clay Shirky and Jeff Jarvis and others have said so many times. It's a chance to provide filtered content, meta-content and context-driven content to help people understand what happened, why, and who is responsible for it.

Everything has to do with who's connected with whom and how, whether it's a company merger that leaves thousands out of work, the approval of a hydroelectric project rejected by the community, or the decision to pass a bill that will neglect our children's education but will serve corporate interests.

All this data is out there. It's in articles written by journalists, in documents, wikis, and public databases provided by government offices, in studies made by scholars or NGO reports, in information given by whistle-blowers that reaches the hands of citizens. These are the sources Poderopedia feeds on. 

WHAT'S NEXT

Poderopedia is being developed by journalists, programmers, librarians and citizens who aim to publish the most accurate and fact-checked information in order for it to become a daily reference for the media and a helpful resource for academics, NGOs and concerned citizens.

The project is starting in Chile -- thanks to a grant from the 2011 Knight News Challenge -- and after its 2012 launch, it will grow to other countries.

In its initial phase, an editorial team will validate each published entry. But once we open the platform for everyone, collective intelligence will help us validate and detect errors.

Sound too much like a Wikipedia spinoff? Well, sort of. It's Wikipedia on speed with a journalistic twist. Poderopedia double-checks every piece of information from both public and anonymous contributors before publishing it. And besides including typical basic information, Poderopedia focuses on the key newsworthy aspects and historical records of a person, company or institution, digging deep into their networks, with the purpose of letting you have a profound understanding of who's who.

The Poderopedia website and its content will be free of charge, open source, and use Creative Commons license Attribution Non-commercial Share Alike (by-nc-sa).

When we launch Poderopedia in 2012, you'll be able to grab the content and put it in your website using Poderopedia widgets. You'll also be able to quote Poderopedia in any way you want and republish its content on your website or blog, following our Creative Commons license guidelines.

Our next move will be to release Poderopedia's API (application programming interface) for developers interested in making new programs with it. Every line of Poderopedia's code will be released under the open-source GNU General Public License and will be published in our GitHub repository. APIs and documentation will be released on a regular basis to provide you and the news ecosystem with new, exciting tools (by that we mean Information Retrieval tools, Relationship logic software, and more). You'll need to sign up to access our services, telling us basic information, such as how you'll use our platform, and accept our Terms of Service.

A LITTLE Q&A

We've been receiving a number of queries about Poderopedia, so we've put together a brief Q&A to answer some of the most common questions for our Idea Lab readers.

Q: I am developer, journalist, designer, UX/IA, mad scientist, publisher, scholar, citizen, soccer mom, can I help Poderopedia?

A: Yes, please! Send us an email. We always need a hand, and we're really interested in crowdsourcing ideas to make Poderopedia better. If you truly rock at what you do, we'll definitely try to hire you as well.

Q: Will I be able to use your results as a source in my investigation/article?
A: Of course! If you have more questions about it, please email us.

Q: What kind of services will Poderopedia offer for news organizations?
A: Free updated and validated profile databases and several development toolkits oriented to build new storytelling experiences, which allows you to save time and put your energy into other important tasks.

Poderopedia will also have a paid, custom plug-and-play database platform that you'll be able to use with your own content and databases to provide context and new data for your users, increasing traffic and revenue from your own advertisements.

Additionally, our recommendation engine will give you the power to connect your existing databases into our software platform, enhancing your documentation centers or knowledge bases. 

Interested? Write to us or follow @poderopedia for new updates.

Three days after its launch, Valdis Zatlers, Latvia's outgoing president, made a public appeal to use Manabalss.lv. Within a week, Saeima, the parliament of Latvia, decided to vote on the first of Manabalss.lv initiatives, and it was adopted in the first reading. Four weeks later, the same initiative passed the second reading. Thus we can proudly say that in a month's time, Latvia had passed a new law on the online request of the nation. Additionally, a second initiative hosted on the EuroSay.eu framework is presently being deliberated in the parliament.

We believe these unique results are due to four basic principles that the EuroSay.eu creators developed over a 12-month period of research and planning:

• Development of crowdsource initiatives -- Authors of new initiatives are given council by a network of voluntary experts. Their advice helps create thought-out and useful initiatives from the representatives of the people.

• 100 percent legitimacy -- The framework interacts with Latvia's Internet banks, providing a free and reliable identity check. Saeima members can be absolutely sure that the number of petitioners is a real representation of the civic will. Additionally, the list of petitioners can be compared to the national voters' register. Personal data accessed by the EuroSay.eu framework is completely protected; its creators consulted with the best Latvian hackers on the matters of Internet safety.

• Initiative tracking -- The framework has the built-in ability to allow the media or NGOs (non-government organizations) to report on the initiatives' progress. This pushes participants to follow through on their commitment, and helps them garner more votes if greater support for a particular initiative is necessary.

• Built for the user -- The creators worked with leading marketing and technology experts in Latvia to create the ultimate synthesis of functionality and form and an interface that's attractive and easy to use. Website http://manabalss.lv/ scores high on both functionality and user interface quality.

We have figured out how to launch this project in your country and in your organistaion! If you're interested, please let me know either in the comments or e-mail me at kristofs [at] creo [dot] mobi We're looking for people elsewhere who would be interested in collaboration on this non-profit project to spread direct-democracy over the world.

It started like a side project, when sitting with colleagues we were brainstorming about our country. We noticed that even though Latvians are considered to be politically very inactive and reluctant to engage with anything, we actually are the contrary - we have done the Baltic Way and once 4 years we do Latvian Song and Dace Festival and every year we do the Big Cleanup. What is interesting, we do all those things volutarily, nobody is paid for them. So, we are active nation after all. Its just that we don't believe in Politics and we feel that we have no say, no voice, no power to get heard. That is where EuroSay.com comes in - to let every individual and every organisation get heard, gather like-minded, show the public need to authorities and get their ideas done!

In the beginning of this Summer, our president initiatied the procedure of firing our parliament, because, it didn't let investigate one of the Latvian oligarchs who is also an MP. In the end of July, people had a referendum and 95% voted pro firing the parliament, so we have new election coming this September.

The central issue at stake is a First Amendment question of whether the court can order a news organization to redact material that has been presented to the public in an open courtroom.

On July 8, WBUR, a public radio station, filed a response memo as well as a supplemental affidavit of our executive producer to the state's Supreme Judicial Court.

The documents are the latest in a lengthy legal exchange between the Norfolk County District Attorney's office and Quincy District Court judges over the redaction from the public record of the name of an underage alleged victim of sexual abuse which was accidentally blurted during a suspect's dangerousness hearing two months ago.

Two-Day Delay

OpenCourt publicly live-streams daily video of the court's First Session proceedings and posts the footage after an interim of two days. This delay is to allow reasonable room for redaction requests and to edit video in extraordinary circumstances, according to WBUR's journalistic standards and as outlined in OpenCourt's initial archiving guidelines.

We have not posted the May 27 archive episode at issue, pending the upcoming appeal hearing on Aug. 4 before a single justice of the Supreme Judicial Court, namely Justice Margot Botsford.

As mentioned in our filings, we would have removed from footage the name of the underage alleged victim and any information in court that would identify her, regardless of a court order. Such an order, however, represents a challenge to basic First Amendment press rights, specifically relating to issues of prior restraint. We are obligated as a press entity to clarify that our actions are voluntary and not mandated by the state.

Perhaps the most famous prior restraint case was the New York Times publishing of the Pentagon Papers in 1971. The leaked secret Department of Defense study extensively documented the U.S. government's Vietnam War history. The federal government sought to suppress the information in the documents. However, the Times' argument triumphed when the U.S. Supreme Court ruled that the press had a First Amendment right to publish information important to citizens' understanding of its government's policies.

Preventing Harm

In another case more relevant to ours, our lawyers write that in Nebraska Press Assn. v. Stuart in 1979:

The U.S. Supreme Court reasoned a prior restraint was not appropriate because there were no express findings that harm would occur upon publication. Moreover, there was no demonstrative evidence that other measures would be unable to prevent those harms ... as Mr. Davidow's affidavit sets forth, OpenCourt has taken other measures to prevent exactly the harm that concerns the Commonwealth.

We have every intention of protecting the latter, and over months have constructed guidelines with our Advisory Board, the public, and an open "working group" at the court. The guidelines are a living document.

The outcome of this case will set important guidance for the future operation of this project and others like it. More importantly, it could also significantly shape the legal lens through which the First Amendment is viewed when it comes to emerging technology in general, and specifically towards live Internet video-streaming.

Photo by of gavel by bloomsberries via Flickr.