Seth Goldstein

An Algorithm is a set of instructions or procedures for solving a problem.

In the same way that computer scientists 50 years ago focused on the single problem of designing a general purpose computer, there is a similar focus in 2005 among leading Internet service architects:  creating a social media computer that leverages user generated content to automate the production of commercial content.  In so far as this represents the important problem that the best and brightest of us are looking to solve, then to an extent it is a race for the best algorithm. 

From PageRank to PeopleRank

Hovering over this endeavor is the shadow of the last great algorithm, namely Google search engine.  At its core, Google is PageRank (which nominally cites both one of its founders Larry Page and its subject of operation, Web pages):

"PageRank relies on the uniquely democratic nature of the web by using its vast link structure as an indicator of an individual page’s value. In essence, Google interprets a link from page A to page B as a vote, by page A, for page B. But, Google looks at more than the sheer volume of votes, or links a page receives;  it also analyzes the page that casts the vote. Votes cast by pages that are themselves "important" weigh more heavily and help to make other pages "important.”"

In this case, (1) the input for the algorithm is the population of web pages, (2) the instructions rank them in value based on their link structure, and (3) the output is the list of links that you see when you search for something.

Now transpose people for web pages, and you see how the race for the next great search algorithm has less to do with organizing static HTML content than with coordinating the constantly changing expressions of millions of distributed people.  For an interesting perspective, see my fellow entrepreneur Mark Pincus’s riff on the PeopleWeb.  Many Internet businesses have tried to direct user behavior into certain architectures of participation.  Services such as Friendster, Orkut, and even Pincus’s own Tribe, presume to address all of a person’s social communication needs in one place.  All of these services, however, are now rapidly trying to reinvent themselves to stay relevant to a community that refuses to be intermediated by somebody else’s system. 

The services that seem to do the best job at enabling users to communicate on their own terms are those that manage to find a middle ground between the DIY (do-it-yourself) ethos that is beginning to pervade the web and the need for structure to guide constructive interactions (ie the reason by Wikipedia succeeds and most other Wikis fail).  LinkedIn, with its two million profiles of professional affiliations, provides the tools for interesting social media production, even if the site itself limits one’s imagination (open up the API please).  The reason behind the annoying digerati blogfest on folksonomies (myself included) stems from the simple but mildly heretical notion that users, given decent primary (meta)data, might actually be able to create their own systems that scale.  Clay Shirky (lighting designer for the Wooster Group, CTO of SiteSpecific, advisor to Flickr, current leading pundit for the digerati at shirky.com) captures the anxiety perfectly in his title to last week’s panel at ETech:  "Folksonomy, or How I Learned to Stop Worrying and Love the Mess".  The question is, then, whether a PeopleRank algorithm that uses community driven tags as its input, could do to About.com, Gawker Media, and Weblogs what Google did to Alta Vista, namely deliver a superior end-user experience that requires only incremental server bandwidth to scale. 

Self Operating Machines

The most exciting new Internet companies are focused on lead generation, behavioral targeting, co-registration paths (aka coreg) and domain name brokerage.

I seem to stumble every day across some new firm propping itself up on the shoulders of Google, Yahoo! or others to take advantage of a current wrinkle in an otherwise perfectly efficient landscape.  The fact is, however, that these wrinkles never seem to disappear.  These advertising mechanisms emerged alongside of the pure media companies, starting with Doubleclick back in 1996, and followed by other advertising networks such as Flycast, Overture and Advertising.com, each firm iterating on the prior model to gain some head room from the imminent internal advertising service offerings of the pure media companies.

Although these advertising technologies have focused on targeting the behavior of consumers, they have also tended to ignore the role of consumers in the production process of the media that they consume.  This is why these networks tend to dynamically balloon in terms of sales growth.  They capitalize on a behavioral blindspot, where the supply of inventory versus the demands of advertising value are disjointed.  As consumers become smart about these artificial mechanisms (banners, keywords, freeipods) their effectiveness drops and they look to get acquired by larger media entities.

The elusive goal of internet media (and the advertising that drives its value) has been to keep up with changing consumer preferences as the technologies of communication continue to evolve.  The adoption of a new means of using the Internet (whether it be ecommerce, webmail, search engines, or shortly blog readers) creates enormous economic value for the donors incredibly quickly. 

The latest shift in online consumption has been the institutionalization of amateur publishing tools.  The unique attribute of RSS is how simply it enables individual publishers of data to connect with individual subscribers to such data.  RSS tools like typepad, flickr, adsense and del.icio.us have made it easy for individuals to syndicate their preferences, memories and desires.  RSS tracking tools like feedburner make it easy for individuals to track who is paying attention.  At the site you are on, I am streaming my Flickr photos, am recommending books as an Amazon associate and am promoting a Google ad-sense ad.

Not unlike dial-up authentication protocols (remember the classic AOL logon "hand-shake"),  the interaction between RSS feeds and their readers is a structured negotiation:  do you, John Q. Public, agree to take the whole feed, nothing but the feed, until you delete it?  I do. Click.   

When you aggregate all of these individual reading and writing agents, it looks more like a landscape of cellular automata than a tradition publishing model.  This would seem to be the essence of social media (props to my wife and guide Tina Sharkey for coining this years ago and registering the domain) and social computing, two memes that seem to be growing in influence.  When individual decisions such as applying certain tags to pages or photos achieve a broad social consensus, then it as if these tags begin to self replicate which is the essence of automatic behavior.

There is a good word to describe this, which comes out of physics, namely Excitable Media.  as per Wikipedia:

Cellular automata provide a simple model to aid the understanding of excitable media. Each cell of the automaton is made to represent some section of the medium (for example, a patch of trees in a forest, or stress in heart tissue). Each cell can be in one of the three following states:

Quiescent or excitable — the cell is unexcited, and can be excited. In the forest fire example, this corresponds to the trees being unburnt.

Excited — the cell is excited. The trees are on fire.

Refractory — the cell has recently been excited and has not yet been through the refractory period. A patch of land where the trees have burnt and the vegetation has yet to regrow.

The concept of cellular automata is useful as a metaphor for next generation Internet content, which is similarly dynamic, member-generated, and excitable.  In the next post, I will focus on algorithms, as they transform the automatic social media into business rules and procedures.