The importance of ‘looking behind the numbers’ in performance management systems May 30, 2010
Posted by Paul Duignan in : Communicating outcomes models, DoView, Easy Outcomes, Evaluation debates, Measurement, Performance management, Reporting systems, Using the approach , trackbackA colleague Stan Capela recently highlighted the importance of ‘looking behind the numbers’ in performance management systems. Pointing out that, if this is not done, false conclusions can be drawn from such systems. I think that most people would agree with this sentiment. The key issue for me is what is the most effective way of us ‘looking behind the numbers’ when measuring people’s, project’s or organization’s performance.
At the moment all that most people have to offer for ‘looking behind the numbers’ is the suggestion we verbally talk about what is behind the numbers and write text-based documents about what’s behind them. These can help improve ‘the numbers’ (i.e. the particular performance indicators being used in a particular case). However, what about instances where there are permanent problems with the numbers? i.e. important outcomes are not currently measurable with our current measurement technology and (equally importantly) how much we have to spend on measuring indicators? The numbers have a way of constantly reasserting themselves (reflected in the saying ‘what gets measured is what gets done’). This is because they are the only things on the page (on the PowerPoint, listed within the table in a printed document or in a traditional performance ‘dashboard’) when people sit down to make the important decisions.
My focus is on forcing a different discipline right into the heart of all discussions about performance. A discipline which provides a permanent reminder at the very moment when ‘the numbers’ are being considered that there is something ‘behind the numbers’. In my view, making sure that this discipline is followed at the moment when the numbers are being considered is critical.
It needs to be a robust discipline rather than just a lot of talk and background papers which all burn off when you are in a room with busy decision-makers and they have five minutes to make a decision based on whatever you give them. In my experience, general discussion and background papers about what is behind the numbers are largely irrelevant at such moments and just the numbers on their own come to dominate.
My suggestion is that all performance indicator systems should always consist of the numbers (the ‘indicators’) visually mapped back onto the underlying outcomes model of what is being attempted. It should be required that the visual model (including the measurable and the currently unmeasurable) is what is always placed in front of decision-makers at the moment they make strategic decisions. Using this discipline has the advantage that it both does justice to the numbers themselves (this is essential), but it also forces a transparent consideration of ‘what’s behind the numbers’ at exactly the same time.
This approach may initially be unsettling for some – those who like tidy lists of numbers with absolutely no hint that there may be important, but currently unmeasurable, outcomes which are being sought. But generally I find that those who start working with this new visual approach are relieved that finally there is some robust and disciplined way for them to bring the issue of the currently unmeasurable, but important, back into the strategic conversation.
Here is a paper on how this approach can be done easily:
Duignan, P. (2009). Mapping indicators onto a visual outcomes model. Outcomes Theory Knowledge Base Article No. 235. (http://knol.google.com/k/paul-duignan-phd/indicators-why-they-should-be-mapped/2m7zd68aaz774/72).
Here is a short YouTube video on how to do it:
Duignan, p. (2009). Painless Performance Indicators: A Visual Approach (YouTube Video) http://www.outcomescentral.org/performanceindicators5.html
Paul Duignan, PhD. (Follow me on my Outcomes Blog; Twitter; or via my E-Newsletter).
Comments»
[...] framework for performance measurement systems that … market research, surveys and trends The importance of ‘looking behind the numbers’ in performance … A colleague Stan Capela recently highlighted the importance of ‘looking behind the [...]