Indicators, targets, benchmarks - sorting out the terminology November 27, 2007
Posted by Paul Duignan in : Outcomes systems architecture, Outcomes theory, Indicators, Measurement, Using the approach , add a commentMany different terms are used in the outcomes and performance management area for measurement and indicators. Often there is considerable confusion about these terms. The short definitions I use from outcomes theory are:
- Outcomes - causes or effects in the real world. Whether or not such causes can be measured is a separate issue (see previous blog for features of outcomes)
- Steps - lower level causes which lead to higher-level outcomes. Because causal processes reside in causal hierarchies, outcomes at one level can be steps for achieving even higher-level outcomes, therefore to refer to causes and effects at any level the general term ‘outcomes and steps’ is used.
- Measurements - measure whether or not an outcome or step has occurred (or how much of it has occurred).
- Indicator - a measurement of an outcome or step.
- Target - a level on an indicator.
- Benchmark - levels on indicators already achieved by other players, or by the same player at an earlier time or in another setting.
- Priorities - an outcome or step which is thought to be the most important for a player to focus their efforts on changing.
Indicator Problems stressing out US Patent Office examiners October 22, 2007
Posted by Paul Duignan in : Outcomes theory, Indicators , add a comment
In an earlier blog posting on the banking system, I commented on the problems associated with indicators which can be distorted by employees. You can either use such indicators to get an accurate measure of an outcome or use them for incentivizing employees, but not both. Of course, if they’re being distorted by employess to maximize their bonuses, then the wrong types of behavior are probably being incentivized. A different problem can arise in those situations where it much harder to distort indicators. These situations in which it’s relatively easy to independently verify indicators are described in the social sciences as being situations where such indicators are reliable. However, indicators need to not only be reliable, they also need to be valid - they need to actually measure what they claim to measure. The US Patent and Trademark Office is having indicator problems of this second type with their production quotas. According to a Washington Post article their production quotas have not been adjusted since 1976 and modern patents are more compex and therefore take more time to process. As a result 67% of staff see production quotas as among the top reasons they would consider leaving and the office has a turnover crisis according to the General Accounting Office. (more…)
Get out of the dark - cut outcomes confusion with the five features of outcomes October 14, 2007
Posted by Paul Duignan in : Outcomes theory, Using the approach, Outcomes models , 3comments
Many terms are used in outcomes systems to describe the elements which go into such systems representing the steps in the causal processes leading from low level activities through to final high-level outcomes. Terms used include: outcomes, intermediate outcomes, outputs, activities, key drivers, key priorities etc. These terms tend to appear in outcomes models (program logics, strategy maps, ends-means diagrams, results chains etc.). In models which are drawn vertically, they normally appear down the left-hand side. There’s often confusion as to the exact distinctions between these terms and many a happy hour is spent by those trying to implement outcomes systems struggling to work out what a particular step should be called and exactly where it should go in an outcomes model.
Outcomes theory can help clarify this issue by using the five major features of outcomes identified in the theory. Outcomes or the steps which lead to them can be: influencable, controllable, measurable, attributable and accountable. Thinking about it in this way can save you a lot of time and confusion when working with outcomes systems. (more…)
Outcomes police, do they exist? October 7, 2007
Posted by Paul Duignan in : Outcomes theory, Standards , add a comment
Several days ago I came across an article by John Day asking the question - Accounting Police, do they exist? In it he talked about the role of the Accounting Standards Board (FASB) in the U.S. Members of this board, ‘Go through a lengthy process of analyzing and reviewing problems in the accounting field that are brought to them. After much thought, they will make a pronouncement as to what they think the new or revised way of approaching the treatment of an accounting issue should be.’
There’s no such body for the outcomes field, I think that there should be one. The absence of people doing this hard (it’s not necessarily the most exciting work in the universe!) but necessary work in the outcomes area means that we don’t have standards and conventions - people simply make it up as they go along - some times they get it right, sometimes they don’t. It would make life a lot simpler for everyone who has to work with outcomes systems if we had a set of well thought through rules for building and using them. (more…)