The Unintended Consequenses of Too Many Choices

You Too May Be A Victim Of Developaralysis | TechCrunch.

Great article, told from a coder’s viewpoint. So many languages, libraries and frameworks is a real problem. What it missed is the effect on operations.

Here’s a common scenario for large, nimble organizations. Let’s say there are 300 developers, in 50 teams, in 5 divisions that “own” parts of the system. Now let’s say the company leadership decides that being first and fastest is the top priority and sets these divisions up in an internal competition for bragging rights and are free to go in any technical direction they want. Now let’s say this company is located where coders regularly roll in and out of various employers.

I have observed this happen first hand: Common Architecture will be shot down at the executive level almost immediately, allowing “complete freedom”. Divisions will ignore Architectural guidelines and select languages, libraries and frameworks that they think will get them to the market fast; often because they are most familiar to the new coders that hire in. More than likely the divisions will use different languages, libraries, and frameworks, over time growing the number of distinctly different base architectures in production in the same datacenters and cloud environments.

So what? At first, no biggie, but if the company does multiple releases then in just 2 to 4 years the unintended consequences start showing up. Surprisingly the biggest consequences are not in the code maintenance, but in Operations. In the first year of no-architecture they will have 5 times the work, training, and knowledge needs, than if the company had common standards. It’s more expensive, but it can be handled.

Where there is a continual churn of developers coming and going, each generation will bring in their own environmental preferences and since the only goal is to get to market ASAP, more new environments will go into production in addition to the old environments that are still “good enough”. Each reflecting the preferences of the moment.

Over 2-4 years that infrastructure group will have to support all of these environments. I’ve seen groups add an average of 1 significantly new environment every 2 years. In 4 years the infrastructure group will be supporting 10x – 15x more environments than they did at the beginning.

Sure, added freedom lets divisions make some short term gains, and their execs will be rewarded, but it’s common that the infrastructure group will not get a budget that reflects this quickly growing diversity. They will be constantly under the gun, routinely blindsided and constantly needing to retrain, hire new expertise, and keep a growing number of legacy experts.

Too much choice without some architectural oversight results in complexity that increases multiplicatively and not additively. ALL need to be supported. Systems in operation are not viewed as “new”, so top management that values only new business, will view the operational side of their company as a money loser which often leads to even tighter budgeting. This hurts not just the infrastructure group but also the company as a whole, even though it won’t be recognized by its leadership.

Architectural coherence is not just an aesthetic thing. It has real consequences on operating costs and up-time. It also has real impact on how the infrastructure operations groups are viewed and funded. In situations like I’ve described, infrastructure will always be underfunded and overworked. Technical debt will grow to the point where the overall technical systems will impede the company’s ability to keep up with its business, rendering it less competitive. That kind of company will usually become uncompetitive in less than 10 years, and their technical agility will not keep up with the needs of their business.

I don’t need to show examples of this. You can see them all around you.