Seth Goldstein

Application Programming Interface

I never studied computers, formally at least.  My Dad was, and is, a software quality assurance engineer.  My first memory of computing was an LED display of a teddy bear on his screen in the mid-late 1970’s, when he used to work at Applicon, an early CAD/CAM technology company on Route 128 in Massachusetts.  My next computer memory was designing a maze video game for the TRS 80 with my Dad and brother Jonas.  In addition to placing the pixels for the shape and color of each alien enemy, we had to choose names of levels.  One was "Estra" after my grandmother whose name was Esther.  I thought we were being quite clever at the time, manipulating the interface of the game based on our personal preferences.  I never pursued technology with formal training, and instead went on to a life in the theatre, as a child actor, then drama student as a literature major at Columbia.  In 1993, working as the archivist for Robert Wilson’s Byrd Hoffman’s Foundation , I stumbled onto CD-Roms and later the Internet as a means of communicating complex multimedia stories.  The prospect of using the Web as a medium for advertisers was not that far afield.

In February 1995, I worked as an HTML programmer for Agency.com, specializing in creating nested tables that had just been released as a spec by Netscape.  I remember working on a catalogue for HBO Video, meticulously framing each image and description to make it easy for visitors to navigate through the video library.  At the time, there were few professional HTML coders which meant that I could command $75/hour to do what would become commodified by programs like FrontPage within months.  In any case, my role at that moment in time (which was coincident with the launch of akebono.stanford.edu aka Yahoo!) was to manually encode static information for individual visitors.  It was incredibly exciting despite how lame it sounds in retrospect.

APIs are the new HTML

But now, in tackling the concept of API, even as it relates to something familiar like Internet Advertising, I am intimated by the history of professional, enterprise computing.  It seems like the antithesis of my own amateur technology tinkering.  For me, APIs are IBM mainframes and Microsoft SDK’s, arcane languages of translation between hardware and software.  This was hard-core C coding, the kind you learned at MIT or CMU, not what you picked up on your own.  But there is another definition of APIs that is emerging in this current Web 2.0 landscape, one that has evolved into (regressed back to?) precisely the amateur, family-oriented activity that I am almost embarrassed to describe as technology. 

It is now 10 years later, and information has now auto-propogated itself to the point where a new kind of API is required, one that enables users to connect and communicate with eachother rather than with the information itself.  Web services are essentially apertures that refract user driven data towards a specific end.  This would seem to have little to do with the heritage of the original Microsoft Windows API  which enabled professional programmers to develop desktop applications on top of the Windows platform.  In late 1999 I sat down with a key figure and evangelist of the Windows platform and API, Brad Silverberg who had recently left Microsoft, and after taking some time off, started a new venture fund called Ignition with the belief that wireless networks would create new platform opportunities.  It would not be hyperbole to describe Silverberg as the chief developer of the most valuable technology platform in history, Microsoft Windows 95.  I asked Brad to describe what made a platform successful and he replied without pause:

- distribution:  first and foremost you need to have a critical mass of customers using the platform

- tools:  you need to have good tools for developers to build applications that can easily leverage the key features and functionality of the platform

- case studies:  you need to have examples of developers who are making good money selling applications leveraging the underlying platform

As of 2005, the Internet has replaced the desktop PC as the primary platform for APIs.  Unlike Microsoft and the desktop, however, nobody controls the web as a platform; although certain companies do oversee enormous pools of user data and have the opportunity to direct such traffic as they see fit.  The talk of Google and Yahoo! (and now IAC) as web platforms center around their ability to recycle users through complex interconnecting networks of search, email, dating, travel, shopping, local services and more.  This is the web version of the gated AOL community circa 1996.  Ironically, AOL is now desperately racing to open their proprietary (Rainman) environment to a public web site (AOL.com) before Yahoo! fully eclipses its relevancy.

Virtually all of the major Web 2.0 platforms (GOOG, YHOO, IACI, AMZN, EBAY) recognize how critical it is to engage their users in the act of media production, and therefore are (in different ways) releasing APIs that stream their consumers’ meta data.  Such data is not simply theirs for redistributing, but rather needs to be the byproduct of some other functionality such as Amazon wishlists, Flickr tags, or EBay auction trends.  Along these lines, the value of a Web Service API is tied to its ability to convert granular feeds of individual data into useful social media contexts.  It is not particularly helpful to think of APIs as simply conduits of data, since the way in which API’s package data are frequently as valuable as the data itself.  In order to access Flickr’s API, for example, you need to choose whether to organize the data by groups, contacts or favorites.  Central then to the evaluation of an API is to what extent it performs high level operations on low level data, and how interesting the ensuing abstractions are to a broad community of users.

A relevant example is my Flickr Zeitgeist stream to the right of this post.  A few weeks ago, I found this simple application that allows you to syndicate a photostream based on your own and/or other people’s photos.  Once you choose, it creates HTML, which you can simply copy and paste within your blogging system.  Curiously, the exposure of my images to a wider audience has pressured me to upload and tag more photos to my Flickr account than I would otherwise.  Call this the blog-narcissism complex:  although I am only sharing my Flickr portfolio with the handful of visitors like you that come to my blog, I feel responsible to provide you with the richest and most dynamic feed of pictures possible. 

For a more comprehensive list of platforms, APIS and applications, I have pulled together the following set of useful links:

Amazon
- Data source
- Example

Del.icio.us
- Data source 
- Example 

EBay
- Data source 
- Example 

Flickr
- Data source
- Example

Google
- Data source 
- Example:  None public as of now.

Yahoo!
- Data source
- Example:  None public as of now.

While most of these platforms (save perhaps for del.icio.us since Flickr now has Yahoo!’s reach) have achieved ample distribution and provide solid tools for application development, none of them have as of yet produced a profitable web service application to fulfill their platform requirements.  One could argue that EBay does this every day as part of their seller network, or that Google has enabled many profitable media enterprises (ie my pals Jason, Gordon and Shawn at Weblogs), but we have still yet to see fully formed businesses leverage these platform APIs and thrive as stand alone entities.  These will likely emerge in the next 3-6 months. 

In the meantime, the goal was simply to establish the API as a key component of media futures, specifically as the hinge between the algorithm that processes raw human meta data and the moment of alchemy that occurs when you discover something you didn’t even know you were looking for, courtesy of some people that you didn’t even know that you knew.

An Algorithm is a set of instructions or procedures for solving a problem.

In the same way that computer scientists 50 years ago focused on the single problem of designing a general purpose computer, there is a similar focus in 2005 among leading Internet service architects:  creating a social media computer that leverages user generated content to automate the production of commercial content.  In so far as this represents the important problem that the best and brightest of us are looking to solve, then to an extent it is a race for the best algorithm. 

From PageRank to PeopleRank

Hovering over this endeavor is the shadow of the last great algorithm, namely Google search engine.  At its core, Google is PageRank (which nominally cites both one of its founders Larry Page and its subject of operation, Web pages):

"PageRank relies on the uniquely democratic nature of the web by using its vast link structure as an indicator of an individual page’s value. In essence, Google interprets a link from page A to page B as a vote, by page A, for page B. But, Google looks at more than the sheer volume of votes, or links a page receives;  it also analyzes the page that casts the vote. Votes cast by pages that are themselves "important" weigh more heavily and help to make other pages "important.”"

In this case, (1) the input for the algorithm is the population of web pages, (2) the instructions rank them in value based on their link structure, and (3) the output is the list of links that you see when you search for something.

Now transpose people for web pages, and you see how the race for the next great search algorithm has less to do with organizing static HTML content than with coordinating the constantly changing expressions of millions of distributed people.  For an interesting perspective, see my fellow entrepreneur Mark Pincus’s riff on the PeopleWeb.  Many Internet businesses have tried to direct user behavior into certain architectures of participation.  Services such as Friendster, Orkut, and even Pincus’s own Tribe, presume to address all of a person’s social communication needs in one place.  All of these services, however, are now rapidly trying to reinvent themselves to stay relevant to a community that refuses to be intermediated by somebody else’s system. 

The services that seem to do the best job at enabling users to communicate on their own terms are those that manage to find a middle ground between the DIY (do-it-yourself) ethos that is beginning to pervade the web and the need for structure to guide constructive interactions (ie the reason by Wikipedia succeeds and most other Wikis fail).  LinkedIn, with its two million profiles of professional affiliations, provides the tools for interesting social media production, even if the site itself limits one’s imagination (open up the API please).  The reason behind the annoying digerati blogfest on folksonomies (myself included) stems from the simple but mildly heretical notion that users, given decent primary (meta)data, might actually be able to create their own systems that scale.  Clay Shirky (lighting designer for the Wooster Group, CTO of SiteSpecific, advisor to Flickr, current leading pundit for the digerati at shirky.com) captures the anxiety perfectly in his title to last week’s panel at ETech:  "Folksonomy, or How I Learned to Stop Worrying and Love the Mess".  The question is, then, whether a PeopleRank algorithm that uses community driven tags as its input, could do to About.com, Gawker Media, and Weblogs what Google did to Alta Vista, namely deliver a superior end-user experience that requires only incremental server bandwidth to scale. 

Self Operating Machines

The most exciting new Internet companies are focused on lead generation, behavioral targeting, co-registration paths (aka coreg) and domain name brokerage.

I seem to stumble every day across some new firm propping itself up on the shoulders of Google, Yahoo! or others to take advantage of a current wrinkle in an otherwise perfectly efficient landscape.  The fact is, however, that these wrinkles never seem to disappear.  These advertising mechanisms emerged alongside of the pure media companies, starting with Doubleclick back in 1996, and followed by other advertising networks such as Flycast, Overture and Advertising.com, each firm iterating on the prior model to gain some head room from the imminent internal advertising service offerings of the pure media companies.

Although these advertising technologies have focused on targeting the behavior of consumers, they have also tended to ignore the role of consumers in the production process of the media that they consume.  This is why these networks tend to dynamically balloon in terms of sales growth.  They capitalize on a behavioral blindspot, where the supply of inventory versus the demands of advertising value are disjointed.  As consumers become smart about these artificial mechanisms (banners, keywords, freeipods) their effectiveness drops and they look to get acquired by larger media entities.

The elusive goal of internet media (and the advertising that drives its value) has been to keep up with changing consumer preferences as the technologies of communication continue to evolve.  The adoption of a new means of using the Internet (whether it be ecommerce, webmail, search engines, or shortly blog readers) creates enormous economic value for the donors incredibly quickly. 

The latest shift in online consumption has been the institutionalization of amateur publishing tools.  The unique attribute of RSS is how simply it enables individual publishers of data to connect with individual subscribers to such data.  RSS tools like typepad, flickr, adsense and del.icio.us have made it easy for individuals to syndicate their preferences, memories and desires.  RSS tracking tools like feedburner make it easy for individuals to track who is paying attention.  At the site you are on, I am streaming my Flickr photos, am recommending books as an Amazon associate and am promoting a Google ad-sense ad.

Not unlike dial-up authentication protocols (remember the classic AOL logon "hand-shake"),  the interaction between RSS feeds and their readers is a structured negotiation:  do you, John Q. Public, agree to take the whole feed, nothing but the feed, until you delete it?  I do. Click.   

When you aggregate all of these individual reading and writing agents, it looks more like a landscape of cellular automata than a tradition publishing model.  This would seem to be the essence of social media (props to my wife and guide Tina Sharkey for coining this years ago and registering the domain) and social computing, two memes that seem to be growing in influence.  When individual decisions such as applying certain tags to pages or photos achieve a broad social consensus, then it as if these tags begin to self replicate which is the essence of automatic behavior.

There is a good word to describe this, which comes out of physics, namely Excitable Media.  as per Wikipedia:

Cellular automata provide a simple model to aid the understanding of excitable media. Each cell of the automaton is made to represent some section of the medium (for example, a patch of trees in a forest, or stress in heart tissue). Each cell can be in one of the three following states:

Quiescent or excitable — the cell is unexcited, and can be excited. In the forest fire example, this corresponds to the trees being unburnt.

Excited — the cell is excited. The trees are on fire.

Refractory — the cell has recently been excited and has not yet been through the refractory period. A patch of land where the trees have burnt and the vegetation has yet to regrow.

The concept of cellular automata is useful as a metaphor for next generation Internet content, which is similarly dynamic, member-generated, and excitable.  In the next post, I will focus on algorithms, as they transform the automatic social media into business rules and procedures.