We use data to act as a proxy for some results of the system. Often people forget that the desired end result is not for the number to be improved but for the situation to be improved. We hope, if the measure improves the situation will have improved. But there are many reasons this may not be the case (one number improving at the expense of other parts of the system, the failure of the number to accurately serve as a proxy, distorting numbers, etc.).
I find something I learned from Brian Joiner an excellent summary – which I remember as:
Data (measuring a system) can be improved by
- distorting the system
- distorting the data or
- improving the system (which tends to be more difficult though likely what is desired)
Brian Joiner’s book, 4th Generation Management is a great book for managers.
continue reading: Dangers of Forgetting the Proxy Nature of Data