The Importance of Metrics in Delivery
The story goes that you get what you measure. This seems intuitive. You count the number of widgets produced, and you can understand many things from that number. Cost of goods sold, gross sales potential, any number of useful tidbits. The corollary of course is that when people are measured, they are very good at maximizing any measurement that you can come up with.
There was a (probably apocryphal) story told in the early days of data warehousing of an insurance claims processing company that had an unusually high number of cases of broken arms. After some analysis, it was discovered that “broken arm” was the default value in the form that claims processors used to input the claim data. Processors were paid for quickly entering claims, not for their accuracy. Thus, while the processors were efficient, it wasn’t really what the company was looking for.
The same is true for IT. For as long as we have been building systems, people have been looking for ways to understand and measure the “productivity” of that creative enterprise. Unfortunately, (even with AI) the act of taking an idea out of someone’s head and then building a system to produce that result is not a simple process, and even harder to measure. As a profession, we have been struggling for decades to find something that we can hang our hat on.
I once had a conversation with a CTO who told me that he was going to measure Lines of Code. He understood that LoC was a widely debunked metric, but he needed something to help him understand why his IT was “expensive and always took too long”. Needless to say, we did not become fast friends. But as I tried to convince him of the error of his ways, I was struck by my inability to come up with something better than “well, you just know”. “If the customer is happy, you don’t care how long it took”. “If you just sit with the teams, you can tell the productive ones”. These were not satisfying retorts to someone dealing with 1,500 systems (and 500 developers, but that is a story for another day).
With the advent of agile and the concept of stories, many organizations latched on to using points and “velocity” to try to measure the output of a team or even worse to compare teams. No amount of shouting about the inappropriateness of points or velocity as a measure of anything outside of the context of a single team will dissuade them. Not even calling them Gummi Bears.
Technical Debt became a way for developers to talk about the fragility of a system under change that was popularized as part of the refactoring movement. It is not hard to find places where this concept has been co-opted as a real and measurable thing. While there is some value with areas of the business understanding the concept, it has become a negotiating tactic between demand generators and teams in order to create time and space so that the team can “go back and fix things”. Hell, there are even tools now that will examine a code base and generate a technical debt number (in monetary terms no less).
But no more, I say! We are out of the desert. We have a way to talk about development productivity. The great minds that brought us the book Accelerate have done the research and have proven that which we all sort of suspected, but were unable to convincingly articulate. There are 4 critical metrics that will tell you the health of a system and the productivity of the team developing it (Deployment Frequency, Lead Time (or Cycle Time), Mean Time to Repair, and Change Failure Percentage). Not only will these metrics give you insight into the health and productivity of teams and systems, they are not magical, mythical, or all that hard to measure.
We at Idea Harbor are here to help you instrument your estate so that you can visualize this critical information and then begin the task of improving each of those metrics. Give us a call and let’s talk about how your organization can improve with proof.
