Measurement has always mattered to me, both in my personal life and at work. In the Lindy Effect, which is something like a rule of thumb or a heuristic, it is said that some things do not inherently have a fixed lifespan – such as books, works, ideas, protocols, cultures, religions, and so on. The fact that they have survived in the past can be an indication of the probability that they will survive in the future.
This was exactly the thing Nassim Nicholas Taleb picked up in Antifragile and just wouldn’t let go of. But the issue with this theory is that just because something has lasted 100 years, you cannot conclude it will definitely last another 100 years. Abstract and demonstrable examples include religion or the poetry of Shakespeare and Ferdowsi. But survival is a probability, not a certainty. Goodhart’s Law is cooler.
Goodhart’s Law
Goodhart’s Law says that when a measure becomes a target, the measure itself gets corrupted. In other words, in that situation, once you start measuring the metric as a target, it is no longer a good metric. For example, a service desk manager may decide to reward employees using a measurement based on the number of closed tickets or requests. So, what happens?
To increase that number, employees start choosing easy requests. They close tasks quickly but incompletely, and they split one simple issue into several separate requests. As a result, the number goes up, while quality drops.
So instead of measuring performance by the number of closed requests, it becomes the target itself. That’s where people start gaming the metric. Because they want both themselves and their manager to reach the target.
Or, for example, in school, if success means getting high student grades, then a school, to become one of the top schools in the country, may simply teach test-taking techniques. The scores go up, but deep learning does not.
The same story is true on social media. When follower count, likes, and views become the target instead of measures, valuable content stops being produced. Everyone just wants to avoid falling behind the algorithm. That’s it.
The point is not that we should have no metrics. The point is that the metric must not become the target, because then the whole thing gets screwed up.
The Observer Effect
In physics, this idea is more closely related to the Observer Effect, and in quantum mechanics, to the measurement problem. In other words, the measurement instrument may interact with the system and change its state.
What is even more interesting is that in information technology, we have a near-direct equivalent called the Probe Effect.
Meaning, if you want to understand how fast a system is, you put monitoring tools, logging, tracing, and profilers on that system. But those same tools consume CPU, I/O, memory, and introduce latency.
Meaning that to understand the behavior of a system, you add something to the system that changes the behavior of the system! There is even a funny term in software called a Heisenbug: a bug that, when you try to debug it, changes its behavior or completely disappears.
Measurement Changes Reality
Numbers were supposed to describe reality, but the trouble begins when reality is forced to adapt itself to the numbers. Because measurement is not merely reporting reality; sometimes it changes reality itself. Overall, I think measurement itself is a problem!
Measurement, by its very nature, is a kind of reduction of reality. Put differently, every measurement is an intervention, sometimes physical, sometimes behavioral, and almost always conceptual.
But the interesting paradox is that without measurement, we can hardly manage anything either. Yet the more we hand management over to numbers, the more likely we are to lose the very thing we were trying to understand.
Measurement – The Solution
The answer is not to abandon measurement. But we should lower the position we give it in our system of valuation. A number should be a tool for thought, not a tool for decision-making, and certainly not the truth itself.
To escape this paradox, I have five rules:
- Never make a decision based on a single metric; look at several independent signals together.
- Don’t connect a metric directly to reward and punishment, because it will quickly be gamed.
- Always keep a qualitative element beside the number: observation, sampling, conversation, case-by-case review, etc.
- Regularly ask: What kind of behavior has this metric created? Not just: What is its number?
- Wherever the cost and effect of measurement itself become greater than its benefit, remove the measurement.
I think the general principle might be this: Measure, but do not believe in what you have measured.
Because numbers are for seeing not reality itself and not something to worship.
Measurement FAQs
Once a measure becomes a target, it stops being a good measure. People start optimizing for the number itself rather than for whatever the number was originally meant to represent.
If a manager rewards employees for the number of closed tickets, employees start picking easier requests, closing things quickly but incompletely, and splitting one issue into several separate tickets. The count goes up while quality drops.
It’s IT’s near-direct equivalent of the Observer Effect from physics. Monitoring tools, logging, tracing, and profilers consume CPU, memory, and I/O and introduce latency, so putting instruments on a system to measure its performance changes that system’s behavior. A related term is the Heisenbug: a bug that changes or disappears when you try to debug it.
No. The argument isn’t to abandon metrics; it’s to stop letting a metric become a target and to give numbers less authority relative to judgment and qualitative review.
Never decide based on a single metric; look at several independent signals together. Don’t tie a metric directly to reward and punishment. Keep a qualitative element beside every number. Regularly ask what behavior a metric has created, not just what its value is. And drop a measurement once its cost outweighs its benefit.
It’s the idea that things without a fixed lifespan, like books, ideas, or protocols, are more likely to keep surviving the longer they’ve already survived, though that’s a probability rather than a guarantee. The article uses it to set up the broader point that measurement can describe a trend without ever promising a particular outcome.
Further Reading
- https://itsm.tools/what-is-itsm/
- https://itsm.tools/what-is-itil/
- https://itsm.tools/itil-4-explained/
- https://itsm.tools/enterprise-service-management-explained/
- https://itsm.tools/itsm-tools-list/
- https://itsm.tools/itsm-pros-cmdb/
- https://itsm.tools/how-to-define-measure-and-report-service-availability/
Hadi Ahmadi (Soroush)
Soroush is a ITSM consultant and helping organizations to start, implement and develop effective services strategies. He is a leader, ITIL lecturer, and consultant with +15 years of activity to business strategy. Also, Soroush is a storyteller, poet and novelist and wrote and published a few books about specialized fields: ITSM,ITOM,ITAM, ESM and Digital transformation in Persian language.
