New outcomes modeling standards released November 1, 2006
Posted by Paul Duignan in : Outcomes models, Standards , trackbackThese days many people are involved in drawing outcomes models for organizations or projects. They go by various names – intervention logics, program logics, causal chains, results chains, program theories, program theories of action. I get to look at a lot of them and, to be frank, many of the ones I see are problematic. They’re either all over the place, or in many cases they are so limited (by limiting them to measurable and attributable outcomes – see later in this post) that they’re almost useless for most purposes. Don’t get me wrong, there’re also people out there drawing good models, but its often in spite of the received wisdom about drawing such models, rather than being because of it.
The problem with the bad outcomes models is that many of them are drawn using arbitrary rules which severely limit their usefulness. For instance in some cases, only three levels are allowed within such models (outputs, intermediate outcomes, final outcomes). While perhaps some simple programs can be modeled within this constraint (I suspect that there are few programs which can), increasingly I’ve come to think that it’s often crazy to limit people in this way when they are drawing outcomes models. I’ll be saying why in detail in later blog postings.
Anyway, a set of standards for drawing outcomes models which form the basis for outcomes modeling within Systematic Outcomes Analysis has just been released. I also think that they’re good standards for using in many other approaches to outcomes and performance measurement. They attempt to ensure that outcomes models are built as cascading hierarchies of causes in the real world rather than just being limited to outcomes that are attributable to a particular player. Of course attribution and accountability are important and need to be properly addressed (as I’ll talk about in later postings); but they shouldn’t be allowed to distort an outcomes model right from the very start.
If they do, they often make the outcomes model useless for discussion with strategic partners (you’ll all end up arguing about who’s allowed to have which particular attributable outcomes in their models) – I’ve actually seen this at meetings!. Secondly they won’t be any use for true strategic thinking when you are doing strategic planning (you’ll only have the measurable and attributable which means you often leave out the important but difficult to measure – sometimes exactly the place where you may be able to achieve some strategic advantage). Thirdly, if you use such models for indicator development you won’t be able to track down where you have an outcome but don’t yet have an indicator. Lastly if you use them for evaluation planning you’ll find you won’t have a place on the outcomes model for your more interesting evaluation questions such as those that require a lot of effort to evaluate for attribution.
Anyway, enough on that for now, check out the new standards and let us know what you think.
[Note added to this post later: If you want a quick summary of the standards look at the Easy Outcomes site and if you want a Powerpoint presentation on the guidelines look at the Easy Outcomes Resources page.]
Paul Duignan [outcomesblog.org]
Comments»
no comments yet - be the first?