Indicator Problems stressing out US Patent Office examiners October 22, 2007
Posted by Paul Duignan in : Outcomes theory, Indicators , add a comment
In an earlier blog posting on the banking system, I commented on the problems associated with indicators which can be distorted by employees. You can either use such indicators to get an accurate measure of an outcome or use them for incentivizing employees, but not both. Of course, if they’re being distorted by employess to maximize their bonuses, then the wrong types of behavior are probably being incentivized. A different problem can arise in those situations where it much harder to distort indicators. These situations in which it’s relatively easy to independently verify indicators are described in the social sciences as being situations where such indicators are reliable. However, indicators need to not only be reliable, they also need to be valid - they need to actually measure what they claim to measure. The US Patent and Trademark Office is having indicator problems of this second type with their production quotas. According to a Washington Post article their production quotas have not been adjusted since 1976 and modern patents are more compex and therefore take more time to process. As a result 67% of staff see production quotas as among the top reasons they would consider leaving and the office has a turnover crisis according to the General Accounting Office. (more…)
New outcomes models (program logics) put up on outcomesmodels.org October 17, 2007
Posted by Paul Duignan in : Using the approach, Outcomes models, DoView , add a comment
We’ve just put a number of new outcomes models (program logics) up on the outcomesmodels.org site. We’re planning to continue adding to the outcomes model collection at outcomesmodels.org for two reasons. The first is to give examples of how models can be drawn in DoView. The more examples people see, the easier they find it becomes to build DoView models. The second is to allow people to grab either the whole of one of these models or just some of the slices from one or more models to use as a start in building their own models. DoView’s unique modular approach using multiple diagram slices makes it very easy to do this. (more…)
Beware lumpers, splitters and slice globblers when you’re building outcomes models October 15, 2007
Posted by Paul Duignan in : Using the approach, Standards, Outcomes models, DoView , add a comment
When you’re drawing outcomes models (program logics, strategy maps, means-ends diagrams, results chains etc), using DoView or other software there’re a few things which will come up on a regular basis. The first of these is the personality difference between lumpers and splitters who are present in the room. Lumpers, obviously want to lump and splitters, well, they just want to split. So if you’re working on an outcomes model which at a high level includes, say, social and economic outcomes, a lumper will want a single outcome which goes ‘complete social and economic whatever’. A splitter, on the other hand, will want to have two separate outcomes - a ’social whatever’ and an ‘economic whatever’ outcome. Neither of them is right or wrong, although in the first instance I often let the splitter have their way because outcomes or steps can always be combined at a later stage if needed. (more…)
Get out of the dark - cut outcomes confusion with the five features of outcomes October 14, 2007
Posted by Paul Duignan in : Outcomes theory, Using the approach, Outcomes models , 3comments
Many terms are used in outcomes systems to describe the elements which go into such systems representing the steps in the causal processes leading from low level activities through to final high-level outcomes. Terms used include: outcomes, intermediate outcomes, outputs, activities, key drivers, key priorities etc. These terms tend to appear in outcomes models (program logics, strategy maps, ends-means diagrams, results chains etc.). In models which are drawn vertically, they normally appear down the left-hand side. There’s often confusion as to the exact distinctions between these terms and many a happy hour is spent by those trying to implement outcomes systems struggling to work out what a particular step should be called and exactly where it should go in an outcomes model.
Outcomes theory can help clarify this issue by using the five major features of outcomes identified in the theory. Outcomes or the steps which lead to them can be: influencable, controllable, measurable, attributable and accountable. Thinking about it in this way can save you a lot of time and confusion when working with outcomes systems. (more…)
Cutting out parallel processes - the CEO’s responsibility October 11, 2007
Posted by Paul Duignan in : Outcomes systems architecture, Outcomes models, Easy Outcomes , add a comment
Outcomes systems are a CEO and other managers’ way of aligning what happens in an organization with the outcomes it’s trying to achieve. It’s rather like the Captain on the bridge of a large aircraft carrier needing a well functioning communications system which flows all the way throughout the vessel. If they don’t have it, they’re not going to get the carrier to go where they want it to. The outcomes systems in many organizations are like an aircraft carrier which has a confusing mess of communications systems - all sorts of flags flying from the bridge, a telephone system, several radio systems, people shouting, small notes passed to individual sailors etc. All these have been introduced at different times, some are badly designed, some are now not functioning in the way they were originally designed to. Meanwhile, the captain is running around the bridge yelling into phones, scribbling on scraps of paper and waving flags while the carrier meanders around the ocean turning in larger and larger circles. (more…)
Outcomes police, do they exist? October 7, 2007
Posted by Paul Duignan in : Outcomes theory, Standards , add a comment
Several days ago I came across an article by John Day asking the question - Accounting Police, do they exist? In it he talked about the role of the Accounting Standards Board (FASB) in the U.S. Members of this board, ‘Go through a lengthy process of analyzing and reviewing problems in the accounting field that are brought to them. After much thought, they will make a pronouncement as to what they think the new or revised way of approaching the treatment of an accounting issue should be.’
There’s no such body for the outcomes field, I think that there should be one. The absence of people doing this hard (it’s not necessarily the most exciting work in the universe!) but necessary work in the outcomes area means that we don’t have standards and conventions - people simply make it up as they go along - some times they get it right, sometimes they don’t. It would make life a lot simpler for everyone who has to work with outcomes systems if we had a set of well thought through rules for building and using them. (more…)
Trading-off indicator accuracy for incentivization - forecast calls for pain October 3, 2007
Posted by Paul Duignan in : Indicators, Outcomes theory & the news , 2comments
An article in the London Times today (4 October 2007) - Why a bulging bonus is part of the problem, argues that the current credit crisis following the melt-down of the US mortgage markets has not yet run its full course. In particular, the problem is that in the global banking system, ‘the losers do not know that they are losers, even now’. It will only be when banks start selling off their investment assets that the real price of these assets will be determined and at that time there’ll be lots more pain. In the meantime, it’s likely that a falsely positive picture will be painted because of the bonus systems for employees in the banking industry. This system means that ‘there is tremendous incentive for everyone from the chief executive down to believe the best when millions in personal rewards are at stake’. As a result they’ll overvalue banks assets in the short term. In the run up to the crisis, the first half year results for banks were the best they’ve ever achieved. In order for employees to get an annual bonus they’ll need to also show good results in the second half of the year or else they’ll lose the potential bonus they have on the books from first half of the year.
This is a classic example of the outcomes theory principle of the trade-off between indicator accuracy and using indicators for incentivization where indicators are easy to falsify. And ‘it is the very nature of the bonus system that may end up prolonging the agony’. Perhaps we could go even further than the article and wonder about the roll of the bonus system not only in prolonging the agony, but as a contributing factor in the creation of the problem in the first place. (more…)
Increased airplane safety by IGNORING final outcomes October 1, 2007
Posted by Paul Duignan in : Accountability, Outcomes theory & the news, Outcomes models , 1 comment so far
The New York Times carries a story today, ‘Fatal Airplane Crashes Drop 65%‘. It reports that fatal airplane crashes in the US have dropped by 65%. The death rate in 1997 was one in 2 million whereas the death rate now is about one in 4.5 million. I think the level of airplane safety is one of the great administrative, regulatory and engineering achievements of our time. It shows what can be done when people are serious about managing negative externalities - bad things that happen in the course of selling goods or services. This success story illustrates an important, but seemingly counterintuitive, principle of outcomes theory - deciding whether or not a decision was a correct one often does not depend on the final outcomes from that decision. In other words, you can improve airline safety by ignoring final outcomes! (more…)