Archive for the 'Architecture' Category

Searching for transparency in the ping network

Tuesday, June 27th, 2006

The “just do it” philosophy of the blogosphere has its advantages, but ironically transparency is not one of them. You would think that a group of people who obsessively document their daily and professional lives would leave a clear written trail that could be followed later, but that ignores the fact that the people involved in the blogosphere’s infrastructure also maintain back channel communications through email and in person at conferences. The result of this unwritten layer of communication is that beginnings of new projects receive a huge amount of postings and links, and failures or tapering off of projects are largely undocumented. By the time people fail to follow up on commitments, or companies involved in a project die off, everybody is so exasperated that they just walk away without posting an announcement.

The specific project I’m referring to here is the FeedMesh network. This was an attempt to coordinate the efforts of various independent ping services. The idea of a ping server is that blogs can send a message to a central server to announce when they have been updated. The ping server then makes this information available to other sites, such as blog search engines, so they can be more efficient and timely in updating their database. Rather than requesting a copy of every feed a search engine is tracking on a regular basis, the search engine only has to read the feed to gather new posts when a ping has been received. In theory, everyone involved with blog feeds would benefit from sharing ping notifications. In practice, established companies that are already being pinged directly by major blogging tools such as Wordpress are reluctant to share this information with less established competitors.

FeedMesh seems to have followed this trajectory from an excited, highly linked start to a slow death due to political and corporate in-fighting. John Keegan wrote a detailed primer on the history and motivations behind FeedMesh’s origin a year ago, and there was a Yahoo group and wiki. Unfortunately, the Yahoo group has been abandoned to the spammers and the wiki is now closed. What doesn’t seem to have been written is a clear post-mortem of its death. Is FeedMesh dead? Is it a casualty of Pubsub’s recent implosion? I know everyone who was building FeedMesh knows why it has gone silent. Maybe one of them could write a definitive post on its current state.

Is a feed item an atomic unit?

Saturday, June 24th, 2006

The model of a feed economy is based on the assumption that we will see multiple levels of aggregation, where feeds are aggregated, split into items, and then reaggregated into composite feeds. This is already happening today at one level with blog search engines that return their results as feeds. Mashups can add multiple layers of aggregation as they consume the output of other mashups, and their output is in turn consumed again by even more mashups. Throughout this production-consumption cycle there is the implicit assumption that the individual feed items are atomic units. This not only means that it is the smallest recognizable component of a feed. It also assumes that the contents of an item will remain intact throughout the consumption cycle, sort of like a kernel of corn when it is, uhm, digested. But is this assumption correct?

FeedBurner is on the leading edge of the wave that is splitting the feed item, releasing the potential economic value contained within. If you look inside a feed published through FeedBurner, you will see that the <link> element has been rewritten to point to the feedburner server. Here is a simple example using the feed from Stowe Boyd’s /Message blog. His most recent post has a URL of http://www.stoweboyd.com/message/ 2006/06/widget_marketin.html. The <link> URL for this item in Stowe’s FeedBurner feed has been rewritten as http://feeds.feedburner.com/stoweboyd/wpeL?m=495. If you try clicking on these two links, you will find that you end up on the same post on Stowe’s blog. FeedBurner is obviously redirecting the new URL through its own servers and then sending the browser to Stowe’s blog. There is nothing nefarious about this. In fact, it is one of the reasons why Stowe and many other bloggers use FeedBurner. In return for letting FeedBurner publish his feed, Stowe gets valuable statistics on which items are clicked. This redirection mechanism is essential for gathering the data used in these statistics.

Nevertheless, FeedBurner is modifying the contents of a feed item. This is a classic slippery slope. What happens if a FeedBurner feed is consumed by another service, which also modifies the <link> URL, and this new feed is then consumed by a mashup of some type? Somewhere during this chain of transformations the redirection to an item’s original owner could be lost. At that point who is the “owner” of record? I understand that “information wants to be free,” but in this quest for freedom do we run the risk of breaking the chain of ownership?

Don’t get me wrong. I’m not being critical of FeedBurner. They offer bloggers excellent value in exchange for the chance to sell them services like advertising. Bloggers benefit and FeedBurner benefits. This is capitalism at its best, but it is interesting to watch the barriers around feed items being broken down.