Inopia

My Data, Your Service - The thin red line of User Data Access APIs (LinkedIn shuts out BranchOut and Others)

Today's news that Linked in has shut off API access to Branch Out, BeKnown, Visible.Me and a number of others once again illustrates the tension between openning data and controlling monetization (in a similar way to Twitter's service changes recently). LinkedIn's reason for shutting down the access is primarily that these companies may be planning premium services which draw on LinkedIn's data (and potentially be a threat to LinkedIn's own monetisation plans).

 

It would be easy to say that this is a mistake on LinkedIn's part (because of the chilling effect it will almost certainly have on usage of the API) or that it's a no-brainer since other companies are getting (free) access to LinkedIn's prized asset. However, there is another factor to take into account which is neatly put in BranchOut's response to LinkedIn:

"we believe user data should be owned by the user, and that people should be allowed to share their data with the new services and contexts that provide the most utility."

While this may appear a little self serving, LinkedIn profiles are curated by the owners - LinkedIn is effectively the channel for these profiles to reach an audience. Although LinkedIn's terms of service may have all manner of restrictions within them and LinkedIn adds a large amount of value this still raises the question - is LinkedIn an effective channel for distribution of my profile information if it chooses to exclude a broad range of uses of it's API?

In other words, as a LinkedIn user one of my greatest utilities is that I can point people to my profile, have them find me and connect to keep in touch. This is not disimilar to Tweets on Twitter - the fact that Twitter pushes tweets to others by many channels is one of it's major value adds. Shuting off API access actually reduces that value and might make me seek alternatives. 

So how can this circle be squared? 

I'd argue that rather than shutting off access, LinkedIn should instead;

• Put a price on this access above a certain volume.
• Potentially even as a %age of revenue that value added services earn.
• Set rules such as "no caching" beyond a strict time limit to ensure data cannot be entirely replicated.
• See these services as a value add that will actually drive more users to add profiles to LinkedIn because distribution is wider.
• Set clear rules of engagement and not change them over time.

Might some of the services conflict with LinkedIn's own plans? Possibly, but as long as the price is right, this may even be a net gain. 

Facebook seems to be ahead of both LinkedIn and Twitter in this respect - it's 30/70% revenue split with Zynga and other games already ensures that there is always cashback for innovation that draws on it's platform.

BranchOut's statement that the loss of API Access is not a big deal ring a little hollow, however LinkedIn's actions certainly mean that people may begin to seek other places to host their professional profiles that allow more distribution.

 

 

 

 

 

Twitter client restrictions - bad for everybody (including Twitter)

It's a shame to see Twitter putting restrictions on client development via their API (more on GigaOm here). Obviously there's speculation as to what the business reasons might be - and the effect on the ecosystem - the move seems more than likely to be about controlling advertising real estate. The original post is here. 

The post makes some valid points about privacy and no doubt Twitters transition to the consumer mainstream has promted some of this, but it still seems to be unfortunate step.

Quote from Ryan Carver (via techcrunch): 

“We need to move to a less fragmented world, where every user can experience Twitter in a consistent way.  This is already happening organically – the number and market share of consumer client apps that are not owned or operated by Twitter has been shrinking,” he writes.

While that might make some sense - mostly likely because it gives Twitter closer control over the Ad real estate associated with the last milimetre to the customer - the twitter client is (almost) the new browser. No doubt this is a major driver - Facebooks monetisation prospects w.r.t. Twitter are simply so much greater simply because most people experience Facebook on almost entirely Facebook controlled real estate. 

Promoted tweets may be one way of advertising but they are annoying for users and interupt the application itself - whereas "out of band" ads on the side of a device attract less ire and can be omni-present. 

The question is - does this make good business sense? and does it signal bad news for "open API" strategies?

The twitter team are smart and no doubt a lot of thought has gone into this change and the perceived economic benefits must have outweighed the risks/negatives (developer anger, loss of innovation and goodwill). However firstly it seems unlikely that this means bad news for the open strategy (that's what got twitter here in the first place and this restricts one dimension of the APIs usage), further for a couple of reasons I think it's a bad move for twitter itself: 

• The loss of developer trust and confidence is not to be underestimated - and although this would seem not to affect "niche" clients for specialist devices/environments (which Twitter will never likely cover), people will be much less free with their innovation.
• The move defacto forces existing twitter clients to integrate other services in addition to twitter - when many might have been content to stay twitter only. 
• Twitter is betting on it's clients being one of the "goto" apps on any device going forward (alongside facebook, the email client, google search etc.) but this is by no means a given - particularly if the number of twitter client options drops. Over time there is a significant risk the facebook and others squeeze twitter into the background (particularly if they offer a channel to twitter).  

There seems to me to be no good reason why Twitter could not have continued to innovate heavily on it's own clients while giving everybody else free reign - if they were better, they'd get more audience. In fact it's precisely the biggest reach clients like TweetDeck which Twitter hasn't restricted. 

So given there is money to be made from ads and the last milimeter to the user, what should Twitter do other than what it has? In my view the right strategy would be to permit and even encourage client innovation but to add revenue share restrictions and requirements on openning up ad real-estate where possible. E.g. 

1. Allow development of twitter clients and use of the API.
2. Provide additional APIs for advertising feeds (potentially provided by 3rd party ad networks).
3. Require clients seeing any kind of volume to either carry ad inventory or pay pay volume fees. 

This is obviously harder to enforce than a blanket "ban", however Twitter's current approach is likely to lead to a "smaller" pie with more for twitter rather than a "larger pie" with more creativity in terms of business models. 

Hopefully in time the decision will get reversed and API innovation will flow once more!

 

 

What is an API? Your guide to the Internet Business (R)evolution

There's a new whitepaper up on the 3scale website - What is an API? Your guide to the Internet Business (R)evolution - definitely worth checking out. It's always tough to find the right metaphors to explain the value of APIs, so we've given it a shot. 

Dion Hinchcliffe summarizes it best:

“We are nearing the time when opening our supply chains across the Web isn’t just a good idea, it will be essential for competitive survival"

A nice timely reminder since just a couple of days ago programmableweb crossed 3000 APIs in their directory - impressive and more to come no doubt!

Services, Apps and the API Powered Web

Last week I had the opportunity to do a series of talks on fun subjects for the open informatics program at CTU in Prague. It was a good opportunity to reflect on some of the trends different people in the API, Web Services, Apps world are picking up, so for one of the talks I tried to pull together some thoughts on APIs, Apps and the Web (embedded below).

There are some great presentation on this subject already out there (check out Sam Ramji's Darwin's Finches for example), but as more APIs emerge it's also becoming clearer just how fast this innovation is happening. 

I've also included some of the API business model material we use with our customers are 3scale - hopefully that provides an insight on what people are doing with APIs.

Services, Apps and the API Powered Web

Many thanks again to the team at CTU for the invitation and the chance to spend some time deep diving into these topics!

API First, Mobile Second, Web Third

Fred Wilson wrote a post about how companies build their mobile presence first, and their web presence second, showing a shift in how people use the internet.

I would probably modify this idea, with the following:

API First, Mobile Second, Web Third

The most important consumer of your product in this age is not necessarily a person, but a machine.

via www.switzer.org

Great post by Scott Switzer and wholeheartedly agree - an API is becoming genuinely essential to launching a online company - and for companies starting from scratch, the API should be first on the drawing board. 

What Foursquare should do to compete with Facebook - "Go Public"

I'm sure the foursquare team has their strategy well in hand after the announcement of Facebook places and maybe this is way off base anyway, but I was suprised to see little real press analysis on what paths they might take (instead is was mainly doom and gloom - (TNW, CNN)). But it seems to me there are clear paths and one of the them could be to yank the steering wheel to one side and "Go Public" (not in the IPO sense - in the social network model sense!). 

To explain this, one of the things that always struck me as the *key* difference between twitter and facebook is: twitter is public broadcast, facebook is group narrowcast. I.e. twitter has asymmetric relations, facebooks are symmetric.

So facebook (although there is a public mode) requires both parties to friend.

This makes twitter and facebook fundamentally different - and I think means that twitter can and will continue to grow very strongly. Because of the type of data shared on facebook it's simply difficult for them to push asymetric relationships.

On location I could see the same thing happening. The "normal" way of thinking about location is "I share with my friends" - symmetric relationships. This also makes sense for security in some sense.

So it seems to me that if foursquare is able to push it's asymetric relationships - i.e. anybody can follow anyone for location information:

• it would make a big change to the service 
• but it would give it a much different utility to facebook.

broadcasting location publically has the chance to create chance meetings, to interact with locations / brands in a way narrowcasting to friends doesn't. Also facebook is unlikely to open the privacy settings much (or make the defaults public) because it is already embroilled in various privacy issues.

An example would be, people checking into a bar in Barcelona - they might post on facebook so their friends know, but they might also post on foursquare so "everybody" knows (basically this is equivalent to posting "I'm at Harry's Bar in Barcelona" on Twitter). Facebook places may allow you to do this also but the focus is different and theres a big risk of confusion between public and private settings. 

Basically a foursquare which focused on a twitter like public broadcast of location could provide a much different platform to facebook places and become the basis of different types of location aware services. You may post there "less" but you'll post there when there is some real utility to doing so like meeting new people in a bar, collecting coupons & points, getting timely information etc. 

P ≠ NP-- Greg and Kat’s blog

P ≠ NP

August 7th, 2010, 8:21 pm UTC by Greg

An email I was recently forwarded (a couple of steps removed) from Vinay Deolalikar from HP Labs:

Dear Fellow Researchers,

I am pleased to announce a proof that P is not equal to NP, which is attached in 10pt and 12pt fonts.

The proof required the piecing together of principles from multiple areas within mathematics. The major effort in constructing this proof was uncovering a chain of conceptual links between various fields and viewing them through a common lens. Second to this were the technical hurdles faced at each stage in the proof.

This work builds upon fundamental contributions many esteemed researchers have made to their fields. In the presentation of this paper, it was my intention to provide the reader with an understanding of the global framework for this proof. Technical and computational details within chapters were minimized as much as possible.

This work was pursued independently of my duties as a HP Labs researcher, and without the knowledge of others. I made several unsuccessful attempts these past two years trying other combinations of ideas before I began this work.

Comments and suggestions for improvements to the paper are highly welcomed.

The paper is about 100 pages, and looks serious (but being a decade away from last thinking about complexity, I am unable to give any more useful evaluation than that). I’ll refrain from posting the paper itself.

Deciding P ≠ NP is a Millennium Prize Problem and I don’t think I’d get much argument to say it is the biggest open problem in computing science.

Update: I see someone else has uploaded the paper. I should point out that in the email thread I got, Stephen Cook said “This appears to be a relatively serious claim to have solved P vs NP.

via gregbaker.ca

Would be stunning if the proof is validated. Although it wont radically alter out day to day as much as a P=NP result would have (Q mass panic since every encryption system out there would be breakable - and vast amounts of computational problems would become soluble!), it's still a big deal.

The synopsis of the proof sounds intuitive though I have no idea on many of the details. I like the idea of interconnectedness at great distance in the solution space causing the breakdown in linear algorithms. Seems like the right kind of idea that might also help people figure out if quantum computing can ever bridge this gap and how it might do so.

Why locking your content to one format or device is a bad idea (Even for Amazon)

Wylie's agreement this week with with Amazon to allow Amazon exclusive rights to publish eBook versions of it's classics on the kindle (techcrunch) has a lot of publishers up in arms because of it's potential to fragment the eBook market. On the one hand it's arguably just another sign of the disruption electronic publishing is likely to wreak on the publishing industry but on the other it also seems like a spectacularly bad idea in the medium and long term. Although it's not the fist such deal, the trend seems to be changing from one off's like Stephen King's UR to larger catalogues.

It's also logical that super-star owners will seek to make deals directly with distributers as far up the value chain as possible.

The exclusivity deal itself though, firstly smacks somewhat of impotence in the face of Apple's iPad - trying to leverage exclusive content to defend the Kindle's market position. It signals that this market fragmentation is "the best option" now - which is a shame.

However before authors and publishers go down this route they should think about where the real clearing houses for electronic books may end up. and owners of Book stores about which audiences they can serve:

APIs now allow seamless and highly flexible delivery of content in raw form - a perfect opportunity to restructure how user receive content, how they consume it (a multitude of readers apps could now emerge) and how they pay - it should be irrelevant where you buy a book (the content and presentation) from - the experience should be the same. 

Even worse if content is tied to a specific device you'll need many devices to read what you want to read and (worse) innovation in the reader device market will stall almost before it started, because the biggest platforms will be the only ones with noteworthy content.

• This is bad for publishers since their patch to an audience is fragmented
• This is bad for readers since they can't consume content in the form they want despite technology allowing that for the first time.
• It is also bad for Amazon and Apple in the long run since they will fail to win all deals and they wont create a sufficient device ecosystem around their sales channel.

Publishers would be well advised to stay away from single channel sales until those channels genuinely support access to the majority of the world reader audience. It seems to me that:

• Amazon's best strategy would be to open it's eBook distribution store to as many platforms as possible (iPad is already covered) but Nook and others - to make it unattractive for anybody to do exclusive deals which exclude them.
• Further, amazon doing exclusives now when they address few 3rd party readers risks killing an ecosystem which they later may need.
• Apple's best strategy would be to do the same with iBooks - open to as many platforms as possible, to also make it unattractive to do exclusive deals with Amazon.

In fact - how about this: an open DRM standard for eBooks which works across all reader devices which all eBook stores (let's root for ePUB) support and APIs for those stores. Apple would hate it (see iTunes / ACC), Amazon would in theory loose an advantage - but in practice very likely gain heavily from it (at the expense of API and not having to develop it's own hardware exclusively).

With the world heading for open ecosystems of content and the chance to decouple data, algorithms and viewing experiences it would be a shame if publishers ended up locking book content into closed ecosystems. If Amazon precipitates this they could loose more than they gain by crippling the ecosystem.

Applying Software Patterns to the Cloud and the Internet Operating System

The MVC for the Cloud meme got me thinking about other principles that might apply to programming the Web as a whole. A lot of the principle's we routinely apply to web apps to great effect seem to have analogies when taken beyond individual apps. While MVC seems a clear fit, here are some others:

• DRY - Don't Repeat Yourself!
• Dynamic Languages
• Convention over Configuration
• Data and Control Abstraction
• Continuous Integration

Taken together with MVC these also make a lot of sense applied to the cloud. On the first one, I think it's fair to say we're already getting there:

• DRY (or DIE): the last 20 years have seen an amazing ecosystem of open source software become available and most programming frameworks have a truly awesome ecosystem of plugins and code available to make the core frameworks more powerful. Now this is starting to be replicated at the service level - there are very few reasons now to develop your own credit card processing, email sending, exchange rate checking etc. systems - these are all freely available via external services which can be integrated by API. The trend seems to be accelerating with even user authentication management (Facebook Connect), file storage (Amazon S3), and a large volume of external data sources (event databases, weather, finance etc.) all starting to become externally available. It's becoming the rule that before you code anything or gather anything it makes sense to say "Is there an API for that?".

This isn't an argument that there should be only ONE API for it - diversity is crucial, but one perhaps the may just be ONE unified common interface. For the next, if we look hard enough I'd say the Web is already the best example of a runtime computing system based on dynamic languages: 

• There is no fixed type system: While SOAP Web Services have taken hold within the enterprise, on the Internet REST is King and there are few formal specifications used for API Interfaces (WSDL, WADL and others do see use - but it's still sparse) - although this will change over time and we encourage our customers at 3scale to provide good specs for their APIs, we're a long way from a formal type system since there are no shared object definitions which are commonly agreed. (Semantic Web efforts are helping change this is also - but I suspect they'll take a long time to break through).
• Even if there were a fixed type system there would still be no notion of "compilation" in the cloud for applications which span multiple services because different services are operated by different companies and individuals and could change at any time.
• End services can also be written in almost any web language from C++ or Ada to Django or Java - even if everything could be frozen and "compiled" there would be no way to do it in the traditional sense. 

"It's about the Interface Stupid" might apply well here because amazingly things still work - today's API providers are realizing the importance of taking care of their interfaces over time. In the long run, at 3scale we see this as evolving into a Contract First environment where we don't just see languages like WSDL & WADL define what an interface does functionally, but what the precise commitments are from the provider to the users of their services (see some great work on this in the European Commission funded IST-CONTRACT project btw).

Seeing the Cloud as a big Operating System based on dynamic languages means the following two memes are all the more important, but are only just emerging:

• Convention over Configuration: Frameworks such as Ruby on Rails have a huge advantage over other web development frameworks by using the idea that if a set of "normal ways of doing things" was established then in many cases there would be no need to maintain complex configuration files everywhere (JSPs - we're looking at you..), what are often a few logical lines of code in a rails app required multiple lines in other configuration files in a JSP driven App. In the cloud it's fair to say this idea is slowly emerging - c.f. the adoption of Twitter's API structure by a range of other services. REST itself is also something which follows a Convention pattern with it's 7 core operations applied to RESTfull data objects. We're a still a long way from being able to integrate 3rd party services almost without checking the documentation. 
• Data and Control Abstraction: within modern programming frameworks ORM (Object Relational Mappings) provide a powerful layer of abstraction over database entries that allow developers to think in terms of objects rather than lower level structures. Again in the cloud REST helps us generate some of this abstraction - new NoSql database even make some mappings even clearer (check out this amazing app - Jquery + CouchDB - look no middle layer.). Web APIs are by their very nature an abstraction of data and functionality - without this layer we would be writing raw SQL queries into linkedIn's database rather than using their API. What hasn't yet emerged is a standard model for representing many of the objects which are being shared - again something progress of the Semantic Web may change.

Programmable Web now lists nearly 5000 mashups of 2000+ Public APIs - this is a huge number of integrations - and is testament to the hardworking developers than made this happen (each one of those links took a lot of debugging) - conventions and abstractions can come fast enough to make this all easier. Those 5000 mashups also highlight something which relates to the last of the 5 memes I picked on - continuous integration:

• Continuous Integration: applying continuous integration to software development is a no brainer for almost any complex system - the technique relies on the creation of batteries of software tests for all aspects of the software application which can be executed in an automated fashion (often the tests are written even before the application code) and runs these tests on any new build of the system when it becomes available. This means that if new code breaks the development build there's immediate awareness of the problem and it can be fixed. Continuos Integration allows a dev team to keep track of how the code matches up to how it is supposed to behave at all time (we keep 3scale's Hudson Installs pretty busy :-) ). In the cloud context, we're still only taking baby steps towards an equivalent of automated testing and continuous integration. Most tests are still before deployment only and after that we hand of to far less detailed monitoring systems - despite the fact that many apps depend on external services. Continuous Integration will likely eventually need to be really continuous and operate at runtime against live deployed systems - not just pre-deploy time.

All in all, we're likely to want to replicate all the power and flexibility we have available for building individual applications to Applications which depend on Multiple cloud services. The scope of the problem changes - but the metaphors still seem to be highly valid. 

I especially like the dynamic languages and DRY analogies - it's great to see the heterogeneity of available Web APIs and amazing to see that stable applications still emerge - Web API interfaces provide a critical glue to make this work and it's interesting to see that they are emerging "bottom up", rather than via top down de jure standards. 

MVC for the Cloud

While planning for a talk at Cloud Expo Europe a few weeks ago I was thinking about appropriate metaphors for the way APIs are changing the web. Although the title was APIs as glue for the Cloud, I think the core metaphor behind it deserves some explanation: MVC for the Cloud. I thought I'd add some notes here as to what this might mean.

MVC or Model View Controller is an architectural pattern for software that seperates out three import things - Models (or Data), Views (visualisation of that data) and Controllers (operations on the data). Since it's invention at Xerox Parc in the late 70's, MVC has a had a huge impact on software engineering and nowhere more so that on Web Applications - there are MVC frameworks for almost every Web programming language and framework out there. 

What I'd call "1st generation" MVC was applied to desktop applications and gave great new flexibility at design and build time but separating these three elements out and allowing independent work on all three pieces. When the software shipped everything got baked together and the end user had no knowledge how the underlying system was stitched together.

Moving to Web Applications there is a "2nd generation" of MVC which takes this even further - M, V and C can even stay separate at deploy time for a web app - this means developers of SAAS apps can independently work on and update their data models, views and controllers without affecting the other components - again all without the user needing to know. We can even allow users to switch interfaces "views" according to their needs - and keep exactly the same underlying logic and data models. Advanced testing frameworks and continuous deployment make it increasingly feasible to decouple almost everything. 

Cloudscale MVC

However, arguably we're seeing the birth of a "3rd generation" (or is that a third wave?) of MVC which has the potential to have an every broader and deeper impact - Cloud (or Web) scale MVC - all enabled by APIs. To explain this: MVC models have to date been though of as applying to single software applications (desktop or web), but the metaphor extends much further. Namely:

• Models (Data): can mean any data anywhere, held by any company.
• Views (Interfaces): can mean any view onto some data or function - i.e. an iPhone app, a mashup with other data, produced by the owners of that data / function or by somebody else.
• Controllers (Logic): can mean any piece of business logic applied to data, again by the owners of the data or by somebody else. 

The implications of this are enormous - instead of a world in which individual applications/services are silos which have M+V+C locked inside of them the MVC connections a can perfectly well exist between applications and between companies. Better stated a company with great data assets can enable access to this data to aggressively grow an ecosystem of partners who provide exiting views (often coupled to audiences) and transformations of this data.