Skip navigation

Monthly Archives: June 2009


While there has been much breathless speculation about the threat that Twitter poses to Google’s iron grip on the search market, I haven’t seen many specific ideas about how that would work. By now we all know that searching real-time content is an increasingly valuable resource for information, but its utility seems to be limited to certain kinds of information like breaking news events and consumer product reviews.  It certainly doesn’t appear to be as flexible a mechanism for locating the wide range of content that traditional search has been since the inception of the world wide web.

The most intriguing speculation that I’ve read about the future of search has been this post about the notion of ‘PageRank for People’. In essence, the idea states that the current algorithms that govern the ranking of search results are inadequate because they rely too heavily on the location of the content being listed. Since Google relies on the volume of inbound links to judge the value of content, it favors content posted in popular locations. The thing is, a piece of good content is just as valuable if it is posted to ‘Bob’s Blog’ or the ‘New York Times’. The solution that was proposed, was a system that factors the ‘reputation’ and ‘authority’ of the content’s author when ranking search results. Just how to calculate these numbers, though, is the tricky part. One answer might be Twitter.

There are two aspects of Twitter that need to be changed. Both the ‘Retweeting’ and ‘Hashtag’ behaviors need to be provided as official features of the service. That means the mechanism for these actions needs to be separated from the 140 character text string of each tweet. That is to say, we should lose the ‘RT’ syntax and make the identity of the original poster some form of metadata that exists outside the post itself. Likewise, tags like those currently labeled with hash-signs ‘#’ should be saved as metadata separate from the actual tweet. With those two changes, searching the web could become a lot more useful. Here’s how it would work:

When I publish a post to Twitter, I should have the ability to tag that post with semantically accessible identifying labels. Each post associated with a given subject (as described by the label) potentially contributes to my ‘authority’ about that subject. Now, when someone retweets my post, they are in essence endorsing what I have said and contributing to their own ‘authority’ about that subject. If there was a scoring mechanism that assigned, say, one point for each endorsement, then you’ve created a system that establishes quantitative values for ‘authority’. Imagine it this way:


For each retweet, additional points are cascaded up the tree. That way the original poster is always given the most credit for contributing the idea, but those that help propagate it are given credit as well.  Authority is defined by the community, not the individual. It should be pointed out that for each subsequent retweet, the poster will have the opportunity to revise the tagged metadata, either adding more detail, or removing labels they believe are not appropriate. That way, the system guards against abuse by so-called trend-squatting.

Now, once we start getting values assigned for Twitter users’ authority on specific topics, search engines can start factoring this in to their rankings. So, content authored by an individual with higher authority for the subject of that content are favored over others. Content authored by organizations might be scored, in part, by the collective authority of that organization’s members. It would create a tremendous upward pressure to contribute value to the community. For some industries, I imagine that one’s scores in this respect would become a factor in employment decisions or compensation levels.

Now,  what I’ve described would need to be only one part of the search ranking algorithm. As described by the original post, there are many other factors that should be considered. Additionally, the scoring mechanism described above is probably far too simplistic and vulnerable to abuse. For example, one complication that could make the system more reliable would be to consider the reputation of the endorsing party when assigning a value to the score that their retweet provides the original poster. That way, it is more valuable to get retweeted by individuals with more authority. Note that ‘authority’ as I’ve described it is an entirely separate metric from ‘popularity’, which is defined by the number of followers that a user has.

There would be many additional side-benefits of such a system. For example, much of what is posted on Twitter are links to content elsewhere on the web. A robust labeling function would turn Twitter into a tagging system for the entire semantic web.

Here is a related discussion.

The flood of activity surrounding the turmoil in Iran on Twitter the past few days has got me thinking about ways to make the platform more useful when big news events occur. Much has been said about how this episode is demonstrating the failure of traditional media, as real-time media has now become the main source of breaking information in Iran. There is no doubt that Twitter, Flickr, YouTube and the like are now a very important resource, but I think it’s important to recognize what we might lose if we completely throw the CNN model overboard. Important services that traditional TV news outlets provide include: intelligent filtering of the raw data stream, and a format that makes commentary accessible. TV news coverage does these things well, in that there are news directors who prioritize information, and a presentation format that weaves background information and editorial content together in a digestible package. These piece have been notably absent from the ‘coverage’ provided by #IranElection.


Here is an idea that occurred to me, for a specialized blogging platform that would allow anyone with sufficient passion or knowledge of an event to produce their own live news coverage of a story, broadcast it to the web, and take advantage of all these amazing real-time media tools. Imagine a website, let’s call it, that you could go to when a story breaks. If you wanted to provide real-time  news and analysis of the story, you could log in via Oauth with your Twitter credentials.  You would then be presented with an administrative interface where you could set variables like: The Headline of the story, a friendly looking URL and a hashtag for your news coverage.

You would then have the opportunity to tweet directly from this interface with commentary on the event. In addition to your regular Twitter feed, your tweets would show up on the public facing news page that you’ve generated (pictured below in section ‘A’). In this respect, you would be serving as a news anchor for the story. As anchor you would have the opportunity to populate your news page with real-time content from many different sources. For example, any relevant photos or videos could be embedded into section ‘E’. You could also display relevant news feeds from traditional news outlets there (if they are bothering to provide coverage at all). This section could also provide space to display a feed of viewer beedback, that could be populated by tweets from viewers who have tagged their posts with the hashtag for your coverage.


If you wanted to provide more compelling updates, there could be a section of the page that allows you to display a live video or audio stream of yourself (section ‘D’) providing commentary. Relevant hashtags could be displayed easily (section ‘B’), providing links to more conversations about the event.

Most importantly, there would be a section where you could provided a moderated feed of miro-blogging content (section ‘C’). The Anckr interface would allow you to follow individual tweeple who you believe are valuable sources. For example, you could have a feed for ‘witnesses’ of the event, people tweeting from the scene. Or, you could have a feed comprised of knowledgeable commentators.

Now, I could imagine traditional news organizations using a branded version of this sort of interface. Alternatively, anyone who chose to start a page, could become their own ‘reporter’. Since no one can provide 24 hour analysis by themselves, it would be cool if there was a mechanism that allowed you as the anchor, to hand off control to someone else of your choosing, kind of like how BNO news does now on Twitter.

All of the tools for creating such a website exist. I’ve listed some of the resources above, including: Twitter, FriendFeed, UStream, YouTube, Flickr, TweetCloud and Paratweet. Thesea re just some examples, but I’m sure there are many others.

The closest thing that I’ve found is, which is very cool, but more of an automated filtering service. If you are aware of anything like what I’ve described above, please let me know. If I had any free time, I might build this myself. Since I don’t, I hope someone gives it a try.