You get what you measure
This is a short post on how measuring performance can backfire. So you are trying to foster a culture and you decide to measure how close (or far) your organization is from that culture. Although your intentions are good, your efforts can backfire. You can end up with a culture that is the opposite from what you intended in the first place.
We will walk through two ideas which, put together, lead to the concept of “you get what you measure.” We then close with what this could mean for your organization.
Measurement influences behavior
Think about the difference between measuring the length of a house versus measuring people’s kindness. Unlike measuring inanimate objects, measuring adaptive systems is difficult. An adaptive system is a system that reacts to being measured. So, the act of measuring influences the outcome.
Different from measuring length of inanimate objects, if you measure people’s kindness, people will tend to act kinder. You can never get to the actual truth–there will always a bias in the measurement. That can be good. By measuring kindness, kinder people. Win!
A measurement is a proxy, not the actual attribute
The second idea is the idea that a measurement is a proxy, and a proxy is not the actual attribute.
Going back to kindness, how would you measure it?
Kindness is an abstract concept. You can’t measure it directly, but you can try to quantify it by measuring a proxy for kindness.
For example, maybe you measure whether a person makes charitable contributions, whether they sent you a message on your birthday, or whether they tend to hold the door for the next person coming in line.
This Zen buddhist saying “don’t mistake the moon by the finger pointing at the moon” can help us remember the distinction between an attribute and it’s proxy.
You get what you measure
So, the combination of (1) adaptive system and (2) measurement-by-proxy has a catchphrase: you get what you measure. Here is an exaggerated example:
You have a customer support center where people look into tickets created when a customer calls with a complaint. You want to measure and improve the quality of this organization.
If you measure quality by how fast tickets are closed, analysts will find the easiest answers to the complains and close the ticket with “xyz is likely the culprit” without doing due diligence. So customers will have to call back with the same complaint, and new tickets will be opened for the same issue. On paper, you are closing tickets faster than ever! In practice, your customers are getting pretty frustrated because their problems are not being addressed.
Say you decide to measure quality by how fast tickets are responded to (as opposed to how fast they are closed). Then someone will create a bot that fills the new tickets with messages like “Looking into it..” All tickets get an initial response, never mind the fact that the response is useless.
Once you have put a system in place to measure (1) an adaptive system that (2) can only the quantified via a proxy, then you get what you measure. And what you measure is not necessarily what you want to foster. No matter how complex you make the metrics, there will come a point where optimizing for the metric will clash with simply “doing the right thing.”
Over time
You are probably thinking: “That’s a pretty cynical view on people!” or “The people I work with are not like you describe”. You are right! Your employees or co-workers are not like that.. at least not now. But they may become more selfish once you put a performance measurement system into practice. The reason is that performance measurement systems put people on the spot. They have to solve a dilema: do I improve the metric or do I “do the right thing”?
May people will be altruistic. They want to do the right thing. Plus, if the company isn’t doing well, no one will do well. So it’s logical to look after the health of the company. But one bad apple is all it takes to shift behavior.
Although you may not notice when one person in your organization puts benefiting themselves over “the right thing”, other closer by will. People have beliefs and values (call it a compass), and they will not change their behavior because of one or two more self-centered individuals. Many of your employees will keep doing “the right thing”, but a few of them may decide to “do the right thing” somewhere else. Over time, the frustration of seeing a bad apple benefitting themselves will crumble your employees’ motivation . They will get tired of being put on the spot and forced to choose between “the right thing” and the metric. You can’t blame them. It’s not a fair situation to be in in the first place.
So, over time, the system will select for the less altruistic of us, and eventually the company will look the opposite of what the performance-measurement system intended to foster.
What then
Okay.. but if we shouldn’t measure performance, what can we do instead?
If the goal is to foster behavior, then create a culture that allows for this behavior to flourish. The shift is from measuring people to enabling them. Embrace the fact that none of this is a hard science, and stop trying to put a number on something that can’t be properly quantified. Encourage discussions on culture instead.