Indicators, targets, benchmarks - sorting out the terminology November 27, 2007
Posted by Paul Duignan in : Outcomes systems architecture, Outcomes theory, Indicators, Measurement, Using the approach , trackbackMany different terms are used in the outcomes and performance management area for measurement and indicators. Often there is considerable confusion about these terms. The short definitions I use from outcomes theory are:
- Outcomes - causes or effects in the real world. Whether or not such causes can be measured is a separate issue (see previous blog for features of outcomes)
- Steps - lower level causes which lead to higher-level outcomes. Because causal processes reside in causal hierarchies, outcomes at one level can be steps for achieving even higher-level outcomes, therefore to refer to causes and effects at any level the general term ‘outcomes and steps’ is used.
- Measurements - measure whether or not an outcome or step has occurred (or how much of it has occurred).
- Indicator - a measurement of an outcome or step.
- Target - a level on an indicator.
- Benchmark - levels on indicators already achieved by other players, or by the same player at an earlier time or in another setting.
- Priorities - an outcome or step which is thought to be the most important for a player to focus their efforts on changing.
Having these clear distinctions from outcomes theory helps avoid some common mistakes which arise when dealing with outcomes, strategy development and performance management. Some of these are:
- Demanding that a set of targets be provided when there are no appropriate indicators for some priority outcomes or steps. If control organizations insist that no outcome or step be identified in such target setting processes which does not have an indicator then there is a problem. Note that such insistence can be implicit rather than explicit and can result simply from the structure of the forms and documentation being used in the target setting process. The two problems which arise are: 1) priority outcomes or steps are left out because there is no indicator on which a target can be set, or 2) a target is developed based on an indicator which is actually an indicator of a lower level step rather than of the higher-level priority outcome. This means that achieving the target, which will start to be the focus of implementation organizations once the targets are released, will direct activity towards achieving the one lower level step from which the indicator has come. If there are other important lower-level steps needed to achieve the higher-level outcome, they may be neglected as a result. A more transparent way to proceed is to: 1) identify priority outcomes or steps which it is thought should be focused on; 2) identify which of these have indicators; 3) identify targets on these indicators; 4) launch indicator projects to develop indicators for those priority outcomes or steps which do not yet have them; 5) realize that it may take a long time to develop an indicator and target for one or more priority outcomes.
- Demanding that every outcome must have an indicator. This error is often disguised by confusing the definition of an outcome with that of an indicator. In these cases something is only allowed to be an outcome if it is measurable. The use of the traditional SMART acronym (Specific, Measurable, Achievable, Realistic and Time Bound)* for objective setting helps feed into this problem. Measurement is important, but lack an ability to currently measure an outcome or step should not prevent it being considered as an outcome or step within an outcomes model. As with the error above, what happens in such cases is either: 1) outcomes and steps without indicators are simply left out; or 2) an indicator from a lower level step is used as an indicator for a higher-level outcome or step, providing only a partial measure of that higher-level outcome or step.
- Making up targets out of thin air. Sometimes because there is a lack of clarity that what is being sought is priorities rather than targets, there is an insistence that a set of targets must be produced regardless of the feasibility of producing meaningful targets. At some stage I will blog on what a meaningful target is, it is easy enough to pluck targets out of thin air but what sort of criteria or process should we use to work out whether targets are sensible or not?
* There are various versions of what the SMART acronym means, this is just one.
Paul Duignan (outcomesblog.org)
Comments»
no comments yet - be the first?