RSS Damage
© 2002, 2003 Jamie Zawinski <jwz@jwz.org>

LiveJournal is one of many online web-log hosting sites; it is also an RSS aggregator, meaning, it can be made to poll other web sites to you, and when they change, show you the new additions. This is extremely useful, and a big time-savings.

As people add such polled sites to LiveJournal, a recent trend has been for people to think, "when we add RSS feeds of sites, we'd better make sure the author is ok with that first!" This because, in late 2002, some doofus had a hissy fit when he found that -- gasp! -- someone was actually making use of the RSS feed that he himself provided, in exactly the way that RSS was intended to be used. He later retracted his hissy fit, but since then, everyone became paranoid about checking first before adding feeds.

I think this is completely silly, but I also suspect that this attitude isn't going to go away: as time goes on, and as RSS gets more popular, more people are going to be bitchy about who aggregates it for whom, and are going to bend over backwards to break things (like the oh-so-many sites that already provide crippled RSS feeds that include only the first few words of the entry, making you click through to actually get any coherent content.)

If this attitude persists, RSS-aggregation sites (and applications) will cease to be useful as a tool for combining the updates from many different sites. It's the late-90s "portal" game all over again, but in reverse: where it used to be that people like Netscape and Yahoo and Alta Vista were trying to prevent you from ever visiting anyone else's server, now all the little guys are trying to get you to hit their server instead of viewing a cached copy elsewhere (which is mystifying, in these days of zero banner-ad revenue.)

And yet, it's really convenient to have one place to go to read all of your subscriptions.

So when that happens, people like me (those with the time and skill to do so) will stop using publicly-available web sites like LJ to aggregate things, and will end up setting our home pages to be local files instead. When it comes to it -- when the only way to get a decent RSS feed of a site is to roll your own and keep it secret -- those of us who are able will use tools like Cheesegrater and Portalizer (or their hypothetical less-halfassed descendants) to screen-scrape other sites and combine them locally, forging user-agent strings as necessary so that our cron jobs don't look like robots, but just look like eager humans wastefully hitting reload. Then we'll have our aggregation and convenience. We'll also have the aggravation of chasing the tail-lights of HTML changes, but we'll do it anyway, because the short-sighted will have left us no better choice.

Shame for everyone else, though: the folks who don't have the time or skill to do this kind of thing.

When one puts something on the web for people to read, people will read it in whatever way they find most convenient. If they don't want to load your banner ads, they won't. If they don't want to obey your Expires: header, they won't. If they want to read it by screen-scraping your HTML and reformatting it locally, they will.

If authors get huffy over RSS aggregator sites like LiveJournal, then readers will just move to other more hidden mechanisms. It will be more work for the readers, but the authors still won't get what they want.

Publishers ignore these facts at their own peril.