Random thoughts on the evolving Web

Google Wave - A power play to change Content and Communication for the Web

3scale was lucky enough to have a sandbox slot at Google I/O in May - and an awesome event it was for us too (check out Martin talking technology here!). Things have been hellishly busy since - but now a lot of the hype died down, there's a chance to comment on the biggest announcement at the event: the launch of Google Wave (see the wave keynote here - watch if all if you haven't already).

There's been loads of Wave coverage - most of it just after announcement (Mashable, Techcrunch, ...) and Dion Hinchliffe already has some trademark great diagrams available. There are plenty of posts which commented on the boldness of the vision and a lot of others which focused on the technical breakthroughs.

While I think the technology is a true achievement, more than anything I'm still staggered by the strategic move which is being played out here. The Wave team has clearly done an awesome job - both in engineering but also in "re-imagining" the web (see the blog post which announces wave) - credit to Google's management as well for giving vent to this imagination. The keynote and the blog post very much present Wave as a kind of skunk works effort - which no doubt it is - but I'm pretty sure there is a very hard nosed, long term strategy backed in here.

Success for Wave would mean nothing less than rewriting the nuts and bolts of the Web as we know it - rewriting all the rules about how content is created, exchanged and managed. The demo is really a peek at the future which will inevitably come - and Google is making a strong play to be the organisation which tkes a defining role in that future. Why say that?:

• Wave would do away with the need for IMAP, SMTP, FTP and a whole bunch of other protocols.
• The cool demos which show a nice interplay between an online blog and a Wave users "inbox" are a key indicator of what's already happening - content anytime, anyplace - separated from design (and possibly context) - how often will you visit your favorite blog's homepage once it flows into your inbox? comments included? Ads not included?
• The content objects exchange will no doubt morph to include all sorts of generic structures - making spreadsheets, docs and other formats fodder for your communications inbox - again power to Google and Google docs (where is Office live?)
• Wave providers could potentially become massive repositories of structured data and communications information (is this is the ultimate cloud?) - all of it indexable - and Google being the biggest of all. 
• The play for wider adoption of the protocol could tempt large swathes of the largest content generators to get on board - watch Blogger "go Wave" at some point in the not too distant future - Will Sixapart and co have to follow?
• Google's push for HTML5, Android and Chrome all seem to gel perfectly - a new world of content with flexible browsers (with no more need for Flash?) and other apps to seemlessly consume this content.

Effectively Wave could do for Content and Communication what openID, openAuth etc. are doing for identity - it's that big.

Google's play to make the protocol and server open is also incredibly smart and beneficial to the web as a whole. There's no doubt that some of Google's competitors will see this as the ultimate poison chalice but it creates some very positive dynamics:

• If we (the people) could reinvent the way the web works - how would we do it? As a debae starter Google Wave is great.
• A powerful trend towards unification of content interchange standards (how long has it taken for IM networks to interoperate?)

The open strategy, if genuinely persued, also benefits Google hugely since it will be hard for others to fight it - even proposing another standard is joining the conversation. My guess is that the low key "it's a skunk works project" launch was very deliberate to get the ball rolling, build critical mass while Google's competitors most likely do nothing and then suddenly find themseleves facing a groundswell of developers adopting wave. Announcing an initiative which spanned the major players at this stage would likely simply have been impossible and killed the protocol stone dead (actually trying to coordinate such an annoucement would have killed it).

We've had plenty of fun in the team trying to work out who should fear Google Wave most amongst Google's competitors and who it's aimed at - Microsoft, Yahoo, Adobe and Facebook etc. Facebook created a lot of debate - consensus opinion was that Wave is a direct attack on Facebook's rise - my take is a little different: Facebook is showing an uncanny knack of holding a growing its user base - this is unlikely to change. At some point in the future if Wave really did made inroads it could simply become a Wave provider and its users would have even less reason to leave their facebook pages - all their content would be held by facebook yet, they'd be able to participate in any conversation. Potentially this even suits Google since it needs competition to avoid being seen as a monopolist.

Still, making wave stick is a major challenge - the opening pitch as a developer project is a perfect opening but there is a long road to travel. If Google doesn't waver however, they are perhaps the only company that can pull it off. The move creates so many unkowns (even for Google's own revenue engine - Adwords - what happens to Ad revenue if all content can be consumed in the inbox?) but if this is really a major strategy pilar they'll likely push through on it. If it eventually comes off, we'll probably see all the following stages:

• Disbelief that it will work and technical objections (lots of interesting debate already). 
• Denial (see Steve Balmer do this for the iPhone and Android - Ray Ozzie already did it for Wave).
• Grudging acceptance (but "it's just the small guys ...").
• Capitulation (Looks like we'll have to adopt this).
• Leverage (people working out smart things to do with it).

It's still a hard thing to pull off and plenty of things could go wrong - but Wave is the boldest vision of a new Web which we've seen in a long time, so it will be interesting to see how it plays out. The smartest companies might be the ones that work out how to leverage before going through some of the other steps (denial, acceptance, capitulation, ...).

The bottom line is that I think we saw a vision of the future Internet on the 28th May which you don't often see. It's one of those moments in which you see the world shift briefly before settling back to its orginal perspective - the comfort zone is to think "nothing happened here, move along..." - the opportunity zone is to hold the moment and figure out what can be done in this new world!

No doubt things wont work out quite as planned - but what's clear is that should Wave get traction, there'll be a huge range of things which get turned inside-out. Congrats to Google for taking the bold steps and throwing down the challenge to everybody to move the Web forward!

Opening the Web's Data - the motivation for the UNDATA API

A couple of weeks ago I had to chance to talk to Jon Udell about one of the projects we've done with 3scale tools to create new APIs -putting some of the great data available at UNDATA online as an API. You can find the API at http://www.undata-api.org/ the conversation on line here.

Since we're an infrastructure I guess it's fair to ask why we put an API like this together in the first place. While without doubt there's the positive benefit of showcasing what our tools can do (the whole site was put together in a couple of days using 3scale's Pro tools and Google's Java AppEngine), there's also a wider purpose in trying to show how easy and valuable it was to get this kind of data on line.

The UN's original site is great for human navigation - you can explore the data sets online and browse at will + search is well organised. Unfortunately though the data can't be downloaded except in zipped bundles of XML which are timestamped at download time - making it hard to grab the bits you really want in real time for any application. The new API certainly isn't perfect - we're still on just a few data sets and the query / return formats are clunky - but this will improve over time and it already allows an application to browse through the data in small chunks. We'll be adding more formats to make that easier and working it into some mashups to show what can be done.

In the long run, the plan is to keep improving the API and open it up for community support - source code for the app will be out shortly + some do's and don'ts for the Java App Engine. We also see it as a growing best practice model for other high value data which should really be out there in API form - and it allows us to make use of latest power tools and get them into people hands for free.

Cloud Computing Standards and the Changing Cloud

The concern and anger unleashed by this weeks leaking (or maybe the writing) of the 'Cloud Manifesto' has managed to drag a blog post out of me. James Urquhat's take (Cloud Manifestogate) seems pretty on the nail too me - the document is really a collection of high level principles (unless there is much more which isn't public), and far from a technical standard. The moves do unfortunately look rather a lot like the fun and games which revolved around SOA standards between OASIS and W3C at key times. The seeming chilling effect on activities like the CCIF could certainly be negative - I guess we'll see this week.

Taking a step back from politics however, it's a little hard to see where the main impact of such alliances (and potential standards might be). 'Cloud computing' means many things to many people, but at the risk of putting some noses out joint, most of the discussion right now is around infrastructure on demand clouds - CPU cycles, Storage, Bandwidth, OS variants, CDN capacity etc. then followed (but at some distance) by Data and Application Clouds. While I fully believe all three levels will see radical changes - many of them due to the emergence of a whole host of standards and common formats in various places - I doubt de jure standards will have much impact at least in the medium term.

First up, at the infrastructure level, clouds have already meant a radical shift in the convenience of using hosted services - Machine AMI (Amazon) and automated build scripts from all sorts of packages (e.g. Rightscale) make a huge difference to how easy it is to port services from one host to another. Standardizing APIs to access compute resources saves time and resources - great when it happens - however the number of these APIs seems unlikely to explode exponentially (leave that to data and apps). Given the similar hard/software profiles emerging it seems unlikely the APIs are going to be a major barrier to cloud-to-cloud mobility.

There is a great thread on cloud computing on Fred Wilson's Blog (Can Amazon run the Table...?) which makes a lot of points on the Pros and Cons of various clouds and the strategies of the players involved. From the infrastructure cloud players id's expect:

•  Them all the shoot for scale.
•  Them all the shoot for geographical presence (big apps want seamless movement of points of present around the globe).
•  Them all to shoot to support most standard hardware/software configurations.

Seems to me that players yet to emerge will come from existing hosting providers which might be able to use cloud software to turn themselves into alliances of clouds - all of which support identical services - and at the same time provide excellent global coverage. Companies like 3tera and Rightscale are in my opinion also dead on the money - given enough developer time you can work around most differences in the majority of infrastructure APIs and abstract them for your customers.

For application and data clouds things are much more complex - I'd expect to see an amazing amount of standardization in a lot of places as lead players open up APIs, Services and hosted solutions which set the standard pretty much everybody will start to follow in that area. It seems unlikely this will be driven by a standards body but evolve differently in each area (but quickly). Lets how patents, copyright and other disputes don't get in the way of data interoperability too much!

 -

Spain: A Land Untouched by Modern Dentistry

A sweeping generalization - but after today's experience I'm not looking forward to finding out if it's just one dentist's surgery:

 - D: So what seems to be be the problem?
 - S: bottom left tooth, right at the back is broken and open, has been since December...
 - D: okay.. let have a look - this hurt (pokes the enamel with the pick...)
 - S: a bit
 - D: and, lets see .. over here (sticks the pick straight down into the obviously open nerve) ...

 - ... patient struggles to pull himself back up off the floor...

 - S: ?!???#?!?
 - D: sorry about that - you appear to have the open nerve exposed!
 - S: you're not kidding - just like I told you when I walked in here.
 - D: oops that's serious, nurse..!!!
 - D: looks like you left it a bit of a long time
 - S: er... right, it happened in December and it's been impossible to get an appointment until Today!
 - D: nurse

 - ... about 40 seconds of activity and a pain later a lump of paste is shoved into the
    hole and sealed...

- D: we had to patch the hole but you'll have to come back next week to remove it...
- S: ok.. so you don't plan to even clean it?
- D: Can't - open nerve you see.
- S: right.... you mean the open nerve you just shoved a lump of dental paste into?...

 - Receptionist: how would April 2nd be for your next appointment?
 - R: oh it's urgent... hmm

Surreal - dentists are no fun anyway, but I suspect some basic training teaches you that sticking sharp objects into raw nerves probably isn't going to be fun for the patient.. Anyway, I guess I won't be back in a hurry - not to find another way ... any way ... to extract the tooth.. Slamming door + piece of string anyone?!

We'll it could always have been worse I guess.... but definitely some research on good dentists needed! - the really really bad thing about bad dentist experiences is ... you just know that the next appointment is already in the diary. O, a couple of ibuprofen more...

ouch...

Facebook Status, Twitter and the Realtime Web

There was a lot of blogsphere buzz about the Facebook's move to open an API for status updates and speculation that this might kill twitter (see more here, here and here). I doubt the direct impact on twitter is going to be that dramatic (for similar reasons as those given by Hypeup and Fred Wilson - the competition isn't direct) but what's more interesting is the massive trend it shows to what Fred Wilson labelled The "Realtime" Web (Google Latitude is another annoncement which plays into this space this week).

As pointed out Facebook relies on a "friend model" in which both parties need to be connected to one another, whereas twitter's feeds are by default public and followable by anybody. This makes facebook a more private/personal medium than twitter and less a form of public communication. What both have in common though is that updates are highly time sensitive, contextual updates. I.e. they are to blogging what IM was to email. API connections for these service look they'll massively increase the potential for near real time applications across the web.

It used to require huge engineering effort to put together an application which 1) communicated with users on multiple devices, 2) linked content from multiple communities and 3) worked in real time - now it almst functions out of the box. Add to that the availability of all sorts of other data and processing via APIs and a lot of things become possible.

Facebook status also throws down the gauntlet to other social networking sites to open status streams - Latitude and Fireeagle are doing the same for location, which will mean pretty soon we should be able to get this data from pretty much anywhere. Hmm... looks like we might need a kind of standard backbone to share it... better be one which is more or less neutral ... hmmm - maybe? twitter?!

Anyway, for now what makes, twitter a different animal to facebook status is the focus of the medium - facebook is 100% people centric, twitter on the other hand alredy has a whole host of other faces: from corporate streams to system uptime alerts and a whole bunch of other stuff. Essentially twitter provides an alert broadcast backbone for anything (not just anyone).

Looks like exciting times are ahead for new Apps!

Twitter's API Rate Limiting

There was a lot of good coverage (SocialToo, EquityKicker, TechCrunchIT) on Twitter's API rate limits this week (including some outrage). I think Nic Brisburne and TechcrunchIT are spot on when they point to the foundations of a business model. I'd actually go further and say that there are probably a whole bunch of business models here. Apart from the obvious possibility of licensing the whole pipe, there are quite a few things twitter could do.

• A variety of usage licenses (commercial, non-commercial but also potentially "reseller" licenses,
• Clustered by topics, tags, location and a whole lot of criteria,
• Or, even more interesting - clusters by social graph or even user profile characteristics. 

Some users may be turned off the service if it's too invasive, but a whole host of third party services are already deep scanning twitter content, and many twitter users want to be "heard" in any case.

As the volume of the whole feed grows (keep twittering!)it becomes less and less easy to handle the main stream and either twitter itself could offer pre-filtered content bundles or allow downstream resellers to do this. All of this could be done with different API queries, but prepackaged bundles would already go a long way.

How much could they sell the stream for? This is hard to say, but certainly there seems to be a case for keeping costs low for  considerable time. Twitter has a shot at become the global "alert message" infrastructure if it leverages its interconnection with different devices (input) and access for API partners well.

A great example of an API being a potential major key to revenue (forget ads...).

Off to post this on twitter I guess...

Predictions for Web APIs in 2009

As you may have noticed - a little less blogging over the last couple of months than I'd hoped. We've had an amazing year at 3scale (new offices, great new team members, techcrunch50, product launch, leweb and some great early customers - thanks most of all to them), but it's also been amazingly busy, so tough to get back to the blog.

Still, a bit of time to reflect over the holidays makes it easier to get back to posting and since the end of the year is the time for whacky, way off predictions about the year to follow (yep - of course they're likely to be way off) I thought it would be fun to do the same.

Probably the thing that strikes me most about what we've done at 3scale is the sheer number of companies we've talked to who have APIs on their roadmaps and the great things they plan to do with them. It really seems like we're seeing the tip of Iceberg right now and I think 2009 will be the year were we see a lot of great new stuff. Programmable Web's index has just been growing month on month and the APIs have been exposing more and more interesting functionality. On 3scale's platform we've also seen some very neat services from general utility like ISBNdb's Book information API to the super niche like Dilog's Dive Record API - check them out.

So from that - what will 2009 hold? I can't promise anything as good as ReadWriteWeb or Fast Company's predictions but here's a shot for the API world - be great to hear what other people think. In no particular order:

1) ProgrammableWeb's directory will break 2,500 APIs in total - up from a bit under 1100 at the end of 2008.

2) There'll be a significant increase in the number of services which come with commercial terms of service - not just "use at your own risk and for non-commercial purposes", but serious business APIs.

3) High quality data providers who already provide high quality API based services to big ticket clients will increasingly set them free and release versions at lower ticket prices for a broader range of customers.

4) At least one of the TC50 2009 companies will be a pure play API company - a company which effectively has no usable functionality provided via it's human accessible web pages, but only a powerful API.

5) It will continue to get easier and easier to publish APIs - REST and SOAP are already supported in most major web languages, but we'll see more and more modules for packages like Drupal as well. The vast majority of Web APIs will still be lightweight HTTP/XML, REST, Javascript etc. - rather than heavier SOAP variants.

6) oAuth will continue to make inroads and become the defacto way of establishing remote data access (here's hoping anyway!).

7) openID will grow too - but big players like Google, Yahoo etc. will all still resist welcoming incoming openIDs from other services for their own systems (but they may crack in 2010).

8) Web APIs will make it onto the Gartner hype cycle for emerging technologies in the the technology trigger phase.

9) SOA technologies more common in Intranets will start to pop up in Infrastructure for Web APIs with SOAP standards like WS-Addressing, WS-Security, governance tools and other bits and bobs showing up in lightweight versions.

and, finally

10) (the most obvious) - Amazon will continue to be king of the hill in the Web Services infrastructure space. They've done an amazing Job in 2008 and it's going to be hard to match them.

Hard to say how they'll play out but in any case it will be interesting to see - look forward to following events this year. If you've got the prediction bug there are plenty of people smarter and more entertaining than me making them - check out Dion Hinchcliffe's list here.

No doubt everybody predicting will have to eat some humble pie come this time 2010 - and if you're up on your Black Swan theory you'll know that the big stuff wont be on anybody's list!

Flickr - Obama on Election night

Saw this on Silicon Alley Insider - the posting of photos the Obama family on election night on Flickr. Impressive how "old media" is being bypassed by this. Even more so because it's after victory - hopefully a good sign of future openness.

The poor guy looks drained - but no doubt everybody is greatful that he pushed for every last second to make sure!

OS-X v's Windows 7

Hat tip to Venture Beat for pointing out the Onions timely advice :) -> OSX v's Windows 7.

1000 APIs & Web 3.0

Time flies when you're having fun, so much so I didn't get a chance to comment on the 1000 API landmark which was crossed at the end of October on programmable web - great news. Back in August, when it crossed 900 I'd actually predicted a date of mid October so I was out by a couple of weeks but it's still very quick growth. Looking at the rate its now pretty much 100 new APIs every month.  If the growth rate didn't  change this would mean crossing 2000 in August next year, but given that its speeding up I'd put my money on well before the summer.

The last few weeks also saw a lot of posts in the blogsphere about Web 3.0, what it might be and how we might get there - bubbled up by the Web 3.0 event in Santa Clara. While I'm not a big fan of buzzwords there is an interesting question as to what the next major step for the Web is. Much of the opinion seems to point to one of several directions: the semantic web, the mobile web or the personalised web (see ReadWriteWeb's recent summary for example). But, while I think all of these things will play a part and technolgies such as semantic markup and search will eventually create their own paradigm shift, I believe that the next next tectonic shift will be something else.

Given what 3scale does I guess its easy to take a stab at what I think a big part of the answer is - APIs and Web Services. Why is that? The primary reason is that I see the other technologies as bringing incremental gains to what we already have in the short and medium term. APIs and Web Services on the other hand are already beginning to radically restructure the very nature of the Web - directly changing the business relationships between large numbers of web companies. The explosive growth of Facebook, Plaxo, Twitter and other companies which launched APIs is now well known but this kind of change is propagating further. There are already companies which operate solely or primarily from their API - Zemanta and Disqus are great examples - I think we'll see a lot more.

In posting this I'm not trying to belitle the other trends - I'm a strong believer in the power of semantic web technologies and the impact of mobile for exampe - but once the API trend takes real root it will actually enable some of the things and it seems highly likely to me that APIs will end up being the next far reaching major step.

In the long run it's mobile semantic web services all the way!

Lastly it's sad to see that despite the Obama win (impressive and - sorry guys, a huge relief for the rest of the world!), Proposition 8 was defeated in California. Still can't quite understand why human society feels it wants to regulate that part of some people's lives.