Number Games – Making Sure Metrics Drive the Right Behavior
In this third paper in a series of four, Frank explores how bad measurement practices can adversely effect an individual, department or organization's performance and focuses on how to align your measurements with your organization’s goals. Learn how to understand the importance of feedback, balance the metrics and create a process of healthy accountability.
In this series of papers, author and industry expert Frank Buytendijk examines the people side of business performance and business intelligence. They include:
- Fact-based decision-making, or is it? Take a close look at the process of decision making to ensure a rational, appropriate process is applied.
- What is the best methodology? Getting to double-loop learning. Understanding the right measurement methodology is essential to achieving your requirements.
- Number games: making sure metrics drive the right behavior. Why it’s important to use measurements that incent actions that benefit the organization as a whole.
- It’s the culture, stupid! Unless measurements are linked to an organization’s culture, the effect will be lost.
We've also pulled out the first several pages of the whitepaper for you to read. Download the PDF on the right to read the rest.
Metrics can tell you anything you want. For every event, metrics can be found that present different, even opposing, conclusions. Managers have an interest in finding and presenting the metrics that make their performance look good. It is hardly debated that many companies suffer from “gaming the numbers” and “cheating the system.” But it would be too easy to blame middle management or divide-and-conquer and other forms of opportunistic behavior.
The problem is not bad people. The real problem is bad measurement practices that makes people behave in opportunistic ways. All these dysfunctional behaviors tend to be hidden. So to understand them, and be able to predict and perhaps prevent the unintended consequences, we need to know where to look for these behaviors.
We can distinguish two basic types of consequences:
- People impacting measurement. People trying to play the numbers so that they don’t have to alter their actions.
- Measurement impacting people. The metrics put in place drive dysfunctional behavior.
There are various ways that managers play the numbers. Perhaps the most well-known example of dysfunctional behavior is measure fixation. It happens when running the numbers becomes more important for managers than running a successful business. An example of this is the railway organization that saw its accuracy deteriorate. Accuracy here is defined as the percentage of trains that leave the station on time and arrive at their destination on time. Confronted with the performance problems on this metric, the operations manager decides to widen the margin of the definition. Previously, “on time” was defined with a margin of two minutes, one minute before the listed time until one minute after. Now the metric is redefined and trains are considered to ride on time within a margin of four minutes.
When measure fixation grows out of control, it can lead to cheating the system. For instance, a group in a back office asks one member to work late, but to punch the time clock for all. Many of the recent bookkeeping scandals have been caused by pure misrepresentation. For instance, consider a multinational owning offbalance sheet entities that buy products and services at the end of the quarter so that the main entity makes the numbers it forecasted to the shareholders and financial analysts. Another example plays on a more operational level. A salesperson promised the customer an additional discount if he would order more products than were needed. Secretly the salesperson advised the customer to ship back the surplus of goods the day after receiving the order. Returns were not part of the compensation plans of the salesperson, and returns were not correlated to the revenues.
But measurements also affect the way people react and behave. Gaming is the opposite of measure fixation and misrepresentation. It means manipulating the business to make the numbers look good (instead of manipulation of the numbers to make the business look good). Both transgressions are equally serious. Gaming occurs when managers start to underachieve once the target has been reached. The next period’s targets therefore are forced not to be higher any more than absolutely necessary. Another variant is overspending at the end of the year to make sure all the budget is used and thus secure an equally high or higher cost budget for the next round.
Due to compliance and shareholder pressures, there is an increased focus on short-term objectives instead of the longer-term strategy. Myopia is the result. For instance, a professional services firm found out it had a problem with its days-sales-outstanding (DSO). All the account managers were urged to work with their clients to have the invoices paid sooner. One particular account manager was extremely successful—all the outstanding invoices were paid immediately. Unfortunately, these were the last invoices the firm could send. The account manager had pushed his client to such an extent that the client paid the bills and terminated the relationship. The actual performance indicator was considered more important than the overall customer relationship.
Another common behavior can be observed when managers focus not on the important targets and the key performance indicators, but on the targets and indicators that are easy to measure. This phenomenon is called tunnel vision. A widespread example is the use of “revenue” as a target for salespeople. This often leads to high discounting by the salesperson who needs to reach his or her target at the end of the quarter or year. At the same time, the CFO will complain about margin pressures. This sales behavior is a logical consequence of measurement on revenue because it is the easiest metric to track. Measuring salespeople’s performance based on contribution margin is more difficult, but also much more worthwhile.
A common best practice states that performance indicators should have a single owner and that management should provide this person with all the means to make the target on the performance indicator. Although this sounds good as a plan, in practice it can lead to suboptimization. Managers look at maximizing their own targets, even at the expense of the overall strategic objectives. Many small suboptimal results then lead to one big negative result.
