Arguments about paywalls around news content are becoming increasingly dogmatic and ideological. As a result, lots of sensible ideas about how to make money from new models of journalism are being obscured. Not least, how to add value to existing content so it becomes more identifiable, more searchable, and helps lead people “back home” (that’s where the Hansel and Gretel theory comes in).

On one side of the fence you have pro-pay-wallers, led by the Murdochs, for whom pay walls seem to answer the question, “How are we going to solve the economic crisis in news?” They’re in the process of trying to convince a great swath of big news organizations to stop providing their content free at the point of delivery. By doing this, the theory goes, they will enhance the value of news content by reviving scarcity and convince a new generation to start paying for news. The “freeniacs are wrong,” writes Nicholas Carr, who adds that “Charging people for news, even online, is by no means an impossible dream.”

On the other side you have the anti-pay-wallers, led by a growing and increasingly coherent group of technologists, liberal educationalists, and bloggers. They see pay walls as a complete misunderstanding of the new era of information abundance. To them, the construction of pay walls is a frantic attempt to recover a 20th century era of constrained media by a generation that “just doesn’t get it.”

Building pay walls is “desperate stuff,” writes Stephen Foley. “It won’t work, and if newspaper executives on both sides of the Atlantic follow Mr. Murdoch’s apparent lead, I predict we will witness the collective suicide of scores of news organizations in the U.S. and elsewhere.”

A debate that needs to refocus

Both sides are becoming more and more trenchant in their beliefs and are ramping up the rhetoric. But, as with the fight between the Big-Endians and Little-Endians in Gulliver’s Travels (about which end to crack open your egg), this ideological dogmatism is distracting us from the more difficult questions. And it doesn’t get us much closer to working out long term ways that will enable journalism to pay for itself.

The pro-pay-wallers need to acknowledge that pay walls are not the Holy Grail that will solve all their economic woes. They should listen to polls — like the PaidContent UK/Harris poll from this week — indicating that most people would leave their favorite news site for a free site elsewhere on the web if a pay wall was built. They should accept that it will not be possible to close the digital Pandora’s Box that is the Internet and recreate the constrained published content environment of the 20th century.

The anti-pay-wallers should concede that there will be areas of content where pay walls work. Pay walls do not have to cordon off all — or even the majority — of information on a site. The Racing Post has a smart and sustainable hybrid strategy of offering significant amounts of content free, and only charging for that which it knows its readers highly value (as reported in the Independent earlier this week).

For £7.50 a month, members get a horse racing TV channel streamed live to their computer (for which 3,000 people signed up in the first week). For £9.50 a month, members can receive a “premium tipping service,” and for £199.95 a year they can get “ultimate membership” with access to tips, races and the Racing Post database. Equally, the anti’s should acknowledge that journalism, as we’ve grown to understand it, is far from free to produce.

Mired in ideological silos, the pro and anti-pay-wallers are also missing some of the most important aspects of the debate. How do you add value to the content itself, such that people will be more willing to pay for it? It’s a question made more urgent for the Murdoch camp by the fact that most content becomes “invisible” as soon as it goes behind a pay wall.

The Hansel and Gretel theory

Here’s where my Hansel and Gretel theory comes in. For those that don’t remember the Grimm fairy tale, it goes something like this: Woodcutter’s wife convinces woodcutter they can’t afford to feed the children. Woodcutter therefore dumps children in the forest. But clever children find their way back by leaving a trail of pebbles. So woodcutter dumps them in forest again. This time, with only a breadcrumb trail, they can’t get home. They then get imprisoned in a gingerbread house by an old witch and… you can read the rest here.

News stories have been, up untill now, a little like Hansel and Gretel’s breadcrumbs. Spread by news organizations round the web, they quickly attract an audience, but that audience gobbles them up and rarely follows the breadcrumbs back home. What if instead of breadcrumbs, journalists and news organizations dropped pebbles? That way people wouldn’t eat them and there would be more chance they could lead them back home.

The difference between a breadcrumb story and a pebble story is metadata. Embed some good consistent metadata in a story and it turns something ephemeral into something much more solid. People suddenly know, for example, where it came from. A story can have the equivalent of an address and a zip code built into it, so people follow it back by whatever trail they want.

Metadata has the significant added benefits that it is visible and malleable. It can be identified and picked up by search engines and aggregators. It can then be displayed so that people have enough information to know if they want more. It’s a little like seeing the front page headlines on the newsstand before deciding to put your hand in your pocket for some change. It can also be used for cross-referencing stories, for digging through the archive, for building mash-ups.

Google has an example of how, using metadata, it can display more information about a site in its “search snippets.” Similarly, we (the Media Standards Trust) have been working out how to best integrate metadata in news through our Knight / MacArthur Transparency Initiative.

The great pay wall debate is not going to end anytime soon — but needs to be a little less polarized than it has been to date. Working out how to leave a trail of pebbles would be a good start.

[For more on this debate, see an excellent two-part series, The Great Debate Over Micropayments and Paid Content at MediaShift.]