DevOps Includes Performance Testing

A quick post about this today. More posts over time…

DevOps. Cute word and a nice concept. Embed your operations into your Software Development LifeCycle, and that SDLC is almost certainly some kind of Continuous Deployment job flow. So what does performance testing have to do with it?

Many people think that performance testing is some dinosaur from the 1990’s. Slow, expensive, an obstruction to any hope of velocity. Well, that’s certainly the case when it’s done wrong, and (write this down) performance testing is usually done WRONG.

Right now I just want to line up and set out what you should be able to expect with a set of right and proper performance testing practices (surprises first, then the obvious benefits):

  • For each component in your application, you will find the 10 app metrics that will tell you all you’ll need to know about that component’s run-time health. If you’re really clever some of those metrics will also tell you way ahead of time whether the component is heading to trouble or recovering.
  • You will routinely embed true performance tests into your Continuous Integration job flows, and they will tell you more than just relative performance build over build.
  • You will catch builds that overload the services your components consume.
  • You will know why a build fails your performance criteria. Almost always it will be right away.
  • Your Operations staff will confidently know what your environment changes will do to the overall environment, If they are doing their own job right then they will be able to predict the impact within 20% of what actually happens when put into production.
  • You will become close friends with your marketing department, since you will be able to tell them the likely impact of new users from new market segments on your site(s).
  • You’ll be able to verify and quantify your components’ scaling rules (up, out or step-wise? Finite? Etc.)
  • Obviously you’ll know your components’ saturation traffic levels. Not only that but if everyone does their homework you’ll be able to come up with an equation that will reasonably predict the total end-to-end saturation levels of each of your common customers’ browsing and conversion paths.

In subsequent posts I’ll tell you how you can accomplish these things. I’ll also get into all the ways it can be done wrong. There are very common and repeated ways that people do it wrong.