I'm pretty thrilled to be joining them: I care a lot about software freedom, improving access to information, and making great software accessible to small organizations. DocumentCloud gives me a great opportunity to approach access to information from a different angle, and to have a hand in developing undeniably excellent tools that will be (some already are) accessible to large and small news organizations alike.

I just started this week, but you'll be hearing more from me as we proceed, about both our challenges and successes. The first challenge was realizing that it was time to bring someone on board to work with our document partners and help Jeremy Ashkenas, our lead developer, find beta testers to help keep him moving forward. I like to think we handled that one well, and I'm looking forwarded to more challenges to come.

With gadgets guzzling evermore energy, New York City faces a looming energy gap. New Yorkers will have to cut back on our electric use or start generating a lot more power. Our game lets people explore the options that are on the table, along with a few that aren't. Should the city ban air conditioning? Harness the tides? Go nuclear? Warning: the game is addictive.

Switch is a concentration-style game that deals each player 18 pairs of cards, each representing an opportunity for the city to conserve or produce electricity. As players match pairs, they're asked to decide whether each policy initiative is a good fit for New York City. At the end (or whenever the player grows bored!) players "flip the switch" to see how the measures they've accepted would add up against the city's predicted 2030 energy needs.

We worked with Will James of Tekimaki, whom we met through his very cool subway map project at onNYTurf which, in addition to being both early and awesome, is the only online NYC map I know of that is available in Estonian.

We've learned a lot about gaming and news games over the last two years, and a lot about building them on the cheap. More on that after you've all played Switch!

If you want to take a look at distribution of discretionary funds by the New York City Council, you have to start with a 400-page PDF full of tables of information. And then you need someone on hand who knows how to pull tables from a PDF into a workable spreadsheet. That, or you need a pencil sharpener and a calculator. And while highlighters and pencil sharpeners are not blowing holes in anyone's reporting budget, the hours required to process this information certainly are. The situation is absurd: this information started out in a database and there's no reason that anyone -- whether they're a reporter, civic gadfly or deli manager -- should have to jump through hoops to put it back into a database.

Of course, those hoops are just for information the city already makes public. If you want to know where pedestrians are being hit by cars, or how parking placards are distributed in a city where curbside space is valuable and abuse of parking privileges is well documented, you'd better know who has that data and have someone on hand who can write an iron tight FOIL request. Want to know about the distribution of lead poisoning cases in the city? For that you'll need lawyers.

FOILs take time, which means money. Lawyers, too, tend to want money for their time. One way to make information cheaper is to step up the data requirements in local transparency laws. New York City is considering legislation that would amend existing public records laws to require that information be made available and that it "be presented and structured in a format that permits automated processing." That is to say, raw data. Just publish it -- don't make us ask.

With the law itself lingering in committee, the mayor's office announced a competition, NYC Big Apps, for applications that will use city data. Perhaps the idea is to deflect attention from the bill, which the mayor is no fan of. The contest, which offers a prize that includes dinner with the mayor, is not really a substitute for making data available.

Steve Romalewski, a pioneer of web-based GIS and community mapping projects, is also skeptical of the contest. He notes that it offers no explicit guarantee that any datasets will be fully available for the long haul, and that no one has offered any explanation of why just 80 data sets are included.

Romalewski also rattles off a good list of datasets that are currently only available on a per-request basis -- which means, among other things, that you need to know they are there. His list includes the types and locations of small businesses, green spaces, recreational spaces and housing violations, as well as interim multiple dwellings (aka lofts) throughout the city. He also points out that land use data currently must be licensed from the city at a rate of $1,500 per year if you want all five boroughs: not a trivial expense to small projects like Gotham Gazette.

Romalewski argues that we shouldn't have to ask for data--that most of what city agencies aggregate belongs in the public domain. I'm with him there, and curious as I am to see what comes out of NYC Big Apps, I'm not convinced that the contest going to help put city data in the public domain in New York City.

I don't know whether or not the legislation currently sitting in committee is the answer we need, but I do know that New York City is not alone in needing far better access to the data that civil servants use and aggregate in the course of their work. I also don't think that simply providing us with the raw data is enough -- but at least it's the bare minimum we need to fill the role of government watchdog.

By the way, if you want that list of under-publicized city data, skip to the comments in Romalewski's post.

Since the code is under a GPL3, doesn't MSNBC.com have to also keep it under the same license if modified? Or can they take the code base since Everyblock is a web-based service?

We at Gotham Gazette had been wondering just about the same thing, albeit for different reasons. We're working on our final Knight-funded game and the programmer we're working with thinks the GPL is too restrictive which got us wondering what it would look like to release source code according to the terms of our agreement with Knight but also allow our programmer (who's hardly getting rich off of this development project) to use the code under a different, less viral license.

And, James Vasile at Hacker Visions has an answer. It is a complex answer, and worth a read. Loosely? The holder of the copyright is not necessarily bound by the license a project was released under.

Assuming we haven't made any terrible mistakes (I already spotted one little error. If you spot it too you can buy me a beer!), we'll post cleaned out versions of the other games we've developed in the next week or so.

One thing we didn't really budget for in these games was really good documentation. What we've got is pretty bare bones. And we certainly don't have the resources to support the software. That said, if you're genuinely interested in getting a Balance!-style game running yourself, I'll do what I can to lead you through it.

Two new projects I'm watching?

Adopt-a-Stimulus -- which I first caught wind of on Twitter -- asks individuals to pick one TARP project and track it. Steve Katz tried and hit a few walls, but it sounds like the folks at ProPublica are committed to staying on the project and working out the kinks.

Cranes on Pause -- WNYC is asking listeners to map stalled construction projects. This one is one that Gotham Gazette has been brainstorming for a while, too long, I guess. I'm glad someone else had the same idea and the wherewithal to get it online! There are rumors that many, many developments are bankrupt or near bankrupt or were banking on fast-rising real estate prices that are decidedly not rising any more, but there hasn't yet been any clear reporting on whether the rumors are all just rumors. Cranes on Pause is a great first step towards a much clearer picture of what is happening to the city's building boom.

Seriously: we knew we wanted to do two things: to build a game that would stay relevant through the New York City campaign season and to find a topic that would fit nicely into the existing code base for one of our earlier games. Ballot access is an important and relatively obtuse issue in New York City that definitely lent itself to the quiz-y nature of the budget maze game, and when candidates file their nominating petitions later this month, there will be a lot of bumping activity, as candidates are forced off the ballot for failing to number the pages on their nominating petitions.

This new game reflects a lot of the lessons we learned from our first maze game. First, we really toned down the "maze" end of things after too many people complained that it was too challenging. Second, we made it a whole lot harder to loose after our high school interns complained that they kept losing (and getting kicked out of the game) because they actually don't know all the answers off the tops of their youthful heads. Since the point of our games is to engage people in civics and civic participation, penalizing players who don't know the system inside and out would seem to defeat the point.

We hope you'll enjoy the game!

Take a look at both if you haven't already. The Post's works a lot (a lot!) like our game, but I'd say the layout that they came up with is a far more effective way to display the available information. On the other hand, I really like the way we handled revenue, by creating a menu of real and specific proposals like a bag tax or sales tax changes. Their menu of options is much more limited in that regard.

So there you have it. One more budget game -- my question (as ever) is ... is this really a game? As I check that category box for "Games and Virtual Worlds" a budget balancing widget seems a bit faint.

Unlike the Post's game, though, ours is available for re-use and re-purposing. Let me know if you'd like to tinker with the skeleton of it!

All the finalists in the contest were invited to share their games at the Games for Change Expo where I watched a handful of people play our game for the first time, an edifying experience to say the least. The Budget Maze, for those of you who have forgotten since you played it last May and told all your friends about it, asks players to navigate a dreary maze of a dungeon that is the city budget process, in search of the key to the youth program they want funded. In keeping with Knight's preference for free and open source software under the hood (all our games are available as free and open source software, but some are built on non-free tools) the game uses exclusively PHP, AJAX and jQuery to provide interactivity.

When we were designing the game, we were committed to keeping it "game-y" -- a response to complaints about earlier games. Watching people navigate the maze for the first time, and chatting with the experts in game design as well as in pedagogy who stopped by our table, I realized that we had gotten perhaps too caught up in our ideas about what a "game" should be, maybe at the expense of making a good learning experience.

One thing I saw while watching people play is that the maze is far too hard to navigate. We wanted the maze component to make the game more challenging and more fun, but watching people play it I could see that, while it might be challenging, it was taking too much of the fun out of our game to have to puzzle through the maze.

Another realization I came to while chatting with the young intern who'd come with me to the expo, and who had a very tough time getting all the way through the game, was that we were sacrificing pedagogy to game-y-ness, as well. We designed the scoring to ensure that at each level it gets harder and harder to get to the end. Too many missteps along the way and you'll run out of time: the city will pass a budget without your input.

That is nice enough in theory, and it sure is the way the world works, but the very people we wanted to reach with the game -- New Yorkers who are genuinely interested in understanding the budget process but don't know it by heart yet -- are the ones who'll have the hardest time getting the answers right. So they're the ones that get kicked back to start over and over as they screw up. The experts have no trouble getting through the game, but they're experts. They don't need us to show them how government works!

It was good timing, the expo was, because we're actively developing our next game, which will follow the same format as our last maze but with some notable improvements. We'd already agreed we want to make it easier to snake through the maze towards the good stuff. We realized at Games for Change that we should re-think the idea of "losing" and let everyone get to the end of the game.

I've been hearing about Calais lately. I at least got the memo that I'm supposed to know what it is. I gather that Calais has something to do with the sources in Media Cloud. So over to Calais:

"Calais is a rapidly growing toolkit of capabilities that allow you to readily incorporate state-of-the-art semantic functionality within your blog, content management system, website or application."

That clears everything up for you, doesn't it? There's more on their about page but there is still a lot of applause for the semantic web and very little in the way of "register for a free account and we'll help you parse your content in search of semantic keywords" (which, is, roughly, what I was looking for.)

So: Media Cloud is a cool tool and data junkies (and everyone, really. everyone) should take a look. But I don't want to be left out of the project, either. So file this as my mildly unhinged rant against insiderism. The Berkman Center is the last place (okay, not the last) I'd expect to find the perspective that only the big kids matter.

The benefit of the doubt I can offer is that this project is clearly still under production. They want some ideas about what people might use Media Cloud for, maybe just to keep them inspired, maybe to get people talking about it, maybe so they can plan feature rollout accordingly. So maybe they just need to hear from small publications?

What do you want from them?

And: back to Calais for a moment. As a publisher, we should be wanting to use Calais to index our archival content. They don't actually have tools to do that with, they have an API -- that means that if you aren't using an out of the box content management system (we aren't) you'll need to write your own interface to Calais. Which brings me to my next question: anyone else writing Calais hooks (is "hook" even the right word?) for their content? How are you structuring it?

This week, though, I made an interesting discovery. After a few tests and a bit of wrangling to establish canonical URLs we implemented commenting with Disqus this week. The move runs altogether contrary to my D.I.Y. instincts and my free software instincts but the tools we were using (multiple instances of Phorum ) were spambait and I wasn't ever able to reign them in and still get other work done. So I let go a little bit and farmed out our comment capacity.

This week, we ran a (excellent, but that isn't the point) article on City Council attendance records and we're seeing a lot of comments on it. We've never run this story at this moment in time before, so I can't really tell you whether or not we would have gotten as many comments (five, so far. Sadly, that is a lot for us) if readers were stuck with our old forums.

I'm pretty sure we wouldn't have. So I'm going to go out on a limb and say that people do want to participate (and yes: participation is about a lot more than just reacting) and will if they don't have to leap technical hurdles to do so.

So I'm wondering: how are other small sites (and big ones?) facilitating reader comments? Are you happy with it? Looking for better tools?

Luckily, ChangeTracker had a few wolf cries left with me so I peeked again when I got this one:
bq. Biography of George W. Bush changed on 3/5 http://tinyurl.com/az8jzp

Which turned out to be quite a bit richer in content. Among the details removed from the 43rd President's biography were observations about commitment to bipartisanship and his work "to create an ownership society and build a future of security, prosperity, and opportunity for all Americans."

That at least gave me a chuckle. What would you want to follow with a Change Tracker?

Fair warning: I wasn't trying to capture the essence of the day so much as taking notes that struck me as relevant to my own work, with an emphasis on questions I've been asking myself lately. I drifted off during the segments on the book and film review APIs, which, while neat-o and potentially of interest to lots of people, have little bearing on Gotham Gazette's public policy coverage.

Despite a few unsuccessful attempts to inspire
participants to identify themselves, it was hard to get a real handle on who was in the
room. Male, mostly. 30ish, generally. It turned out I was sitting next to a designer from USA Today. There seemed to be a handful of other publications at the hackday, as well as people from data driven projects from a very cool dictionary effort (though with my collected business cards in another coat pocket and no attendee list in evidence, I can't tell you which dictionary effort. Alphabetasomething.) to OpenCongress and Sunlight.

First up was Tim O'Reilly, who had some wise words about our own data as publishers. Or, about the NY Times' data as a publisher. And about finding meaning in user generated content and looking for ways to turn that meaning into something useful. More precisely, turn that meaning into real-time user-facing services but that only makes sense if you're going to sit through his whole talk. This is crowd sourcing, he's talking about. It is inviting your readers to participate in reporting in a way that is meaningful. So the question is: Could we do a better job of anticipating how readers want to participate and giving them a framework for that?

O'Reilly had a lot of good questions about the business of publishing. The questions I'm asking now are less about reader participation than about who our readers are and what they're doing:
Looking our "Most Emailed" stories: do we know more about who is emailing stories? Should people you've sent stories to makeup a social network? Are they your web? Can we tell a story about who is blogging articles? What articles are being blogged? We can answer that question on a purrely objective level, but we don't aggregate or compare data about articles that are or aren't being blogged. When a popular blog links to a Gotham Gazette story the post can get dozens of comments while our own forums sit quiet. We don't have a good way to document or measure that conversation. I'm not as worried about eyeballs on ads as about being able to demonstrate that we're making a positive contribution to civic conversations.

How much of that did Tim O'Reilly say? I wasn't taking those kind of notes. I still think these are good questions to ask ourselves.

There were pieces of Tim's talk that troubled me, too, though. Pieces that reflect some of my frustration with social media standards in general. He suggested that the TImes could do more to celebrate frequent commenters, but there are good reasons not to: you encourages people who want that attention to comment, but don't necessarily foster conversations. Moreover, I'm not sure that focussing on the loudest voice encourages soft-spoken readers to say their piece. And I think that encouraging discussion should be at least partly about making sure that quieter voices get heard.

O'Reilly also asked for more "not again" features, more ways to ensure that you don't see an article you've already read. Again: I'm not convinced this is the best way to foster civic engagement. Sure, when you're reading the paper the first time you might want to skip straight to the newstuff, but I'd really like you to be able to come back to Gotham Gazette later in the day to read that story again or point it out to a friend. I think there is real value in things staying in the same place, at least for a little while.

This is my list of websites to look at again, gathered over the course of the day: MediaCzar's maps of twitter tweeters, USA Spending.gov picking up what OMB Watch's Fedspending.org has been doing for a while (with their blessing and their toolset). Stimulus Watch for more community budget monitoring and blprnt:http://blog.blprnt.com/ for gorgeous visualizations of NYT content based on data from their feeds. And some tools, Apture which I'd heard of but didn't know how to spell, bit.ly which adds a lot of analytics to your shortend URLs, Semantalyzer for more structured keywords and OpenSearch for search result standardization, a concept I need to noodle a little more.

The whole point of the day was to get us all using the Times APIs but the ones I really really want are still under development. Like plans to extend the Campaign Finance API to New York State data or to open up the Represent to outside lookups. The latter, http://www.nypirg.org/ used to do, but they dropped their Who Represents Me project when maintaining the GIS servers finally overwhelmed them. They'd already spun off most of their mapping work, leaving Who Represents Me something like orphaned.

I wouldn't say I live tweeted the event (secret: I loath live tweets of events, especially events I'm not at.) But I did my share of whispering from the back of the room.

Original game in hand, we worked with a Flash programmer to build out a whole new version of the budget game, this time with text and numbers stored in an external XML document. This change means that anyone with an eye for detail (XML is an unforgiving standard -- a tiny error can trash the whole thing) can edit the XML file and produce their own game of balance.

That turned out to be a good thing when the game got delayed by several weeks and we had to scramble to re-organize the content to reflect a dramatically changing budget landscape!

Speaking of EveryBlock, they recently announced that they're working with the New York Times to track Times reporting on political districts. Presumably they'll be taking advantage of the Times' new API -- now if only they could tell the rest of the media how to flag our stories as discussing a particular district. As the local election season gets rolling, Gotham Gazette will be covering many underreported races along side the ones everyone is talking about, and we'll be aggregating coverage of every race as it appears in other papers and news outlets around the city.

And, speaking of news outlets, local NPR station WNYC has launched a very cool crowdsourcing project tracking Uncommon Economic Indicators to mixed results so far (there's a tweet up right now about the rising cost of heroin) but it should be interesting to see what listeners come up with.