Trading-off indicator accuracy for incentivization - forecast calls for pain October 3, 2007
Posted by Paul Duignan in : Indicators, Outcomes theory & the news , 1 comment so far
An article in the London Times today (4 October 2007) - Why a bulging bonus is part of the problem, argues that the current credit crisis following the melt-down of the US mortgage markets has not yet run its full course. In particular, the problem is that in the global banking system, ‘the losers do not know that they are losers, even now’. It will only be when banks start selling off their investment assets that the real price of these assets will be determined and at that time there’ll be lots more pain. In the meantime, it’s likely that a falsely positive picture will be painted because of the bonus systems for employees in the banking industry. This system means that ‘there is tremendous incentive for everyone from the chief executive down to believe the best when millions in personal rewards are at stake’. As a result they’ll overvalue banks assets in the short term. In the run up to the crisis, the first half year results for banks were the best they’ve ever achieved. In order for employees to get an annual bonus they’ll need to also show good results in the second half of the year or else they’ll lose the potential bonus they have on the books from first half of the year.
This is a classic example of the outcomes theory principle of the trade-off between indicator accuracy and using indicators for incentivization where indicators are easy to falsify. And ‘it is the very nature of the bonus system that may end up prolonging the agony’. Perhaps we could go even further than the article and wonder about the roll of the bonus system not only in prolonging the agony, but as a contributing factor in the creation of the problem in the first place. (more…)
Increased airplane safety by IGNORING final outcomes October 1, 2007
Posted by Paul Duignan in : Accountability, Outcomes theory & the news, Outcomes models , add a comment
The New York Times carries a story today, ‘Fatal Airplane Crashes Drop 65%‘. It reports that fatal airplane crashes in the US have dropped by 65%. The death rate in 1997 was one in 2 million whereas the death rate now is about one in 4.5 million. I think the level of airplane safety is one of the great administrative, regulatory and engineering achievements of our time. It shows what can be done when people are serious about managing negative externalities - bad things that happen in the course of selling goods or services. This success story illustrates an important, but seemingly counterintuitive, principle of outcomes theory - deciding whether or not a decision was a correct one often does not depend on the final outcomes from that decision. In other words, you can improve airline safety by ignoring final outcomes! (more…)
Clarifying program outcomes to reduce ‘talking past each other’ about program effectiveness September 27, 2007
Posted by Paul Duignan in : Outcomes theory & the news, Outcomes models, DoView , 1 comment so far
This morning when casually looking through old Washington Post articles which make reference to the General Accountability Office (GAO). (Hey: people have all sorts of hobbies and no one challenges them about it - beer tab collecting etc.). I came across an article about DARE - the Drug Abuse Resistance Education program. This is a program which puts law enforcement personnel into schools to talk to school children about drugs. The wisdom of funding DARE and other drug education in schools is the source of endless debate in evaluation and public health circles as important summaries of the evidence (such as the 2003 GAO report) referred to in the article fail to find evidence of the program having an effect on illicit drug use. (more…)
Typical media story about outcomes - crime stats increase September 24, 2007
Posted by Paul Duignan in : Outcomes theory & the news , add a commentThe issues dealt with by outcomes theory - which looks at the ways we think, discuss and argue about outcomes - are central to many of the political debates which we read about daily in the media. For instance today (24 September 2007) the Washington Post carries a story with the headline FBI Report: Violent Crime on the Rise. It’s fairly typical of the type of story we see. From an outcomes perspective what are the important elements of these types of stories? Firstly, there’s a claim by the media that there’s some ‘news’ in what they’re reporting - that something is changing. The media need this, basically what they sell is news of change - if nothing changes there’s nothing to tell us about, there’s no ‘news’. In this case the subtitle is: ‘First significant increase in homicides and robberies since 1993 continues.’ Outcome measures can go up and down due to statistical variation, is this change actually significant from a statistical point of view? There’s no way to tell from the story, I presume they’re not referring to statistical significance in the title. If they were they would need to make this clear in the article if we are to be able to have any basis for deciding whether we should regard this as an actual change or not. (more…)