Why coding is decidedly not a commodity

I’ve lost count of the number of times I’ve heard that supposedly “coding is a commodity” in large corporates. In such organisations this has been the mantra over the past few years, supporting a shift of such “commodity” skills to low-cost offshore locations.

It’s a deeply depressing mantra, because it dismisses an incredible skill as of little value and in quite a condescending manner. It also creates a massive skills pipeline issue, because how are we supposed to grow management and IT architectural skills, if those entering the industry don’t have a chance to cut their teeth on coding? With this approach we were in danger of breeding future management types who had no experience of what software-engineering is. I’ve always believed you can’t manage what you don’t understand, so this was never going to work.

My experience is that the “coding is a commodity” inevitably results in large and bloated teams of average-skilled people (or worse). Because man-day rates are so low, there’s little incentive to not add more people to your project. But of course adding more people rapidly results in a coordination nightmare. And when the focus is on cheap people, its hard to get really good people. This is an unpleasant place to be.

Thankfully this “coding is a commodity” mantra is a bankrupt one. Even if it was true at one point (which I do not believe), things are changing – and changing rapidly. I’m happy to report that, IMHO, we now have ample evidence to put to bed forever this argument that coding skills are a commodity.

The first piece of evidence we have is the incredible success of Silicon Valley. It’s an over-used phrase, but software really is eating the world. Major corporations like Facebook and Google have forged new and very lucrative business models through incredible software innovations.
This software isn’t written by low-cost commodity skills, its forged by some of America’s brightest minds. Top universities like Stanford are hunting grounds for the internet companies. Google is well known for only employing the very brightest of employees. Heck, competition for software skills is so intense that many companies erroneously entered into illegal ‘no poaching’ agreements to try to calm down the overheated silicon valley software jobs market. Top-class coding skills are a career again.

But its not just the Internet companies where this is true. Some of our most cherished traditional industries are being reinvented by software. As Mary Meeker so eloquently puts it, we’re seeing the “re-imagination of nearly everything”.

• Encylopedias are going out of print
• Printed photos are being replaced by on-screen digital equivalents
• Newpapers are being replaced by internet browsing
• Diaries are being replaced by social media
• Scrapbooks are replaced by Pinterest
• Magazines are being digitised by the likes of Flipboard
• Books are being revolutionised by Kindle
• Music and Videos are now more commonly downloaded than bought in an HMV store
• Maps are replaced by sat-nav, which in turn is replaced by mapping apps on smartphones

Everywhere we look, industries are being turned on their head by technology - much of it software. This doesn’t feel like a commodity to me. And its not only happening in wacky West-Coast American technology companies; every industry is affected, or will be shortly.

The second piece of evidence I’d like to call upon is my own personal experience. It seems everyone wants to talk to “the guy that wrote the code”. Jaded corporate types are tired of talking to spin-doctors; they want the real techies. Real techies are generally assumed to be coders. If those coders can spin up a few charts to explain their points, then all the better. My experience is that many of them can, and often do a much more credible job than the professional powerpoint jockeys.

Now, more than ever, it seems that people want to listen to the doers, not the professional talkers.

The third piece of evidence I’d like to discuss is the emergence of a new approach to software development. In the “coding is a commodity” world, projects were often large and sprawling. Large and sprawling implies lots of people and lots of people implies lots of cost. In such environments it’s perhaps not surprising that man-day-rate was king, because the pennies (or cents) soon add up.

However, we are seeing an increasing proliferation of a very different software architecture. This is an architecture that supports small, multi-disciplinary teams working together in an agile fashion. Typically they deliver new function in 2–3 months, rather than 2–3 years. Small teams sitting close with those who understand the business requirements, a multi-disciplinary approach rather than handoffs to large faceless specialist centres. These kinds of approaches are gaining ground and pricking the attention of even those who led the charge of coding commodity.

However, the complexity of the large core systems that runs many big companies remains. This new approach only becomes viable when that complexity can be hidden and development against it made ‘self service’. The moment we need a meeting with a specialist core systems team is the moment this new model starts to break. Hiding complexity and enabling self-service development are therefore critical in this new world.

There’s an increasing consensus around how we make this happen. Typically this requires a combination of

• A set of well-structured Application Programming Interfaces(APIs) that hide the complexity of the legacy function. New teams code to these APIs and don’t need to understand anything of the internal structure of the underlying application code.
• A self-service development platform that enables development teams to immediately gain access to the infrastructures and tools they need to build new software systems. This is sometimes referred to “platform as a service (PAAS)” and for mobile developers the phrase “Back-end as a service(BAAS)” is also gaining ground.

Application Programming Interfaces(APIs)

APIs are clever because they present a programming interface that hides what happens underneath them. Historically we’ve tried to do this, but it has often been tricky to gain the value promised.

Sometimes this failure was because of a lack of standardisation on the technology, or because the technology was overly complex and difficult to use. Other times it was because the “self service” aspects of the modern API movement were missing. In all cases it was because it was just too damned hard.

The modern API movement resolves these issues. It brings an industry consensus on a very simple-to-use technology set that supports modern and simple APIs. By moving to a standard technology set that everyone understands, and one that is easy to pick up rapidly for those that don’t, we avoid the complexity that previous attempts suffered from. And the addition of self-service capabilities like discussion forums, code samples, API explorers, etc mean its easy for anyone to work out how to use an API without needing to pick up the phone or attend a meeting.

APIs aren’t just about the API itself - there’s a wider ecosystem that is required in order to get the ‘self service’ development benefit. This only comes when three components are integrated, namely:

1. APIs that:
   • Adopt industry consensus on API technology, namely REST (architectural style), OAuth (security), JSON (data structure).
   • Are clear, logical and simply structured - aimed at easy adoption by developers, rather than intellectual cleverness
2. An API Gateway that provides the run-time infrastructure to support the APIs including:
   • Applies security policies
   • Enforces volume limits, protecting the underlying service
   • Provides data and analytics to help manage and plan the API service
3. A Developer Portal that includes:
   • Clear self-service documentation
   • Code samples in popular programming languages to illustrate static documentation
   • An ‘API Explorer’ that allows developers to interactively play with the API in real time, inputting calls and receiving returns in a way that allows them rapidly validate their understanding of the API.
   • Developer forums that allow questions to be asked and answers provided, all recorded for posterity and for the next developer to learn from.

Its common to find APIs implemented that focus on (1). Without (2) and (3) its not surprising that the authors are disappointed at their failure to encourage a new agile development culture. Only by addressing all of the aspects of a true API, in the way that popular internet companies like Facebook have, can this new world be fostered in. However, when done well, it becomes possible to exploit small, local, multi-disciplinary teams that rapidly deliver new capability.

Platform as a Service(PAAS)

I’m not the biggest fan of acronyms like PAAS, which often seem designed to obscure rather than inform. However, PAAS has gained some industry momentum, so I use it despite my reservations. PAAS describes an important capability that massively simplifies the life of developers, which is the self-service provision of the software/hardware infrastructure necessary to support their application code.

As a software developer I need access to things such as an application server that executes my code, a database to store/retrieve data, a user registration service that provides userid/password capabilities, a messaging service, an analytics services, etc. Building such IT infrastructures can be an unusually complex and lengthly process. I’ve seen big companies take upto 6 months to do this, during which time the development team is on-hold, waiting for the capabilities it needs to start work.

PAAS services provide a website where a developer can choose the infrastructure services her new application requires. This isn’t about low-level geekery. I don’t choose my operating system or versions of software - I only get to choose the type of service I require and all that low-level stuff is hidden. The provision of those services is then automated, with a live development environment being provided within a matter of seconds/minutes. Capabilities like IBM’s BlueMix typify this radical new approach to supporting developers. The ability to eliminate all the typical corporate meetings and delays and get working immediately can be a massive boon to productivity.

APIS + PAAS = Freedom

When we put a well-engineed API together with a responsive PAAS environment, development teams are set free. They become self-sufficient end empowered to do what they do best; build software. In many big companies the delays, complexities and endless meetings are what made development so slow and costly. By removing that cost and delay we make small, local teams an economic reality. Small teams that are close to those who understand the business are nearly always more responsive than large and remote teams.

Software is redefining what is possible in business and the software-engineers that make that possible are far from being a commodity. When an API+PAAS strategy is put in place, it empowers a new generation of developers that, sitting close to the business, are best placed to build solutions that redefine what that business is capable of. And once again the IT industry has a natural career structure that enables those at the top to start their careers by doing stuff, rather than just talking about it.

Evidence

So how do we know this all works? We have the evidence in front of our eyes. This combination of APIs and PAAS is exactly what companies like Facebook have used to popularise third-party services and apps that build on their capabilities. Because Facebook has has such enormous volume, it cannot afford to have meetings with every app developer. Its API is successful precisely because it has been built to be self-service.

When Facebook updates its platform it doesn’t hold a meeting with the tens of thousands of developers that interface with it. That's clearly impossible, so it has engineered a solution that makes those meetings unnecessary. Which is a pretty powerful concept - thousands of apps interlinked without any coordination, discussions or meetings. If Facebook can do this, why cannot a Bank, an Insurance company or a Retailer?

 

Posted with Blogsy