Beware lumpers, splitters and slice globblers when you’re building outcomes models October 15, 2007
Posted by Paul Duignan in : Using the approach, Standards, Outcomes models, DoView , add a comment
When you’re drawing outcomes models (program logics, strategy maps, means-ends diagrams, results chains etc), using DoView or other software there’re a few things which will come up on a regular basis. The first of these is the personality difference between lumpers and splitters who are present in the room. Lumpers, obviously want to lump and splitters, well, they just want to split. So if you’re working on an outcomes model which at a high level includes, say, social and economic outcomes, a lumper will want a single outcome which goes ‘complete social and economic whatever’. A splitter, on the other hand, will want to have two separate outcomes - a ’social whatever’ and an ‘economic whatever’ outcome. Neither of them is right or wrong, although in the first instance I often let the splitter have their way because outcomes or steps can always be combined at a later stage if needed. (more…)
Outcomes police, do they exist? October 7, 2007
Posted by Paul Duignan in : Outcomes theory, Standards , add a comment
Several days ago I came across an article by John Day asking the question - Accounting Police, do they exist? In it he talked about the role of the Accounting Standards Board (FASB) in the U.S. Members of this board, ‘Go through a lengthy process of analyzing and reviewing problems in the accounting field that are brought to them. After much thought, they will make a pronouncement as to what they think the new or revised way of approaching the treatment of an accounting issue should be.’
There’s no such body for the outcomes field, I think that there should be one. The absence of people doing this hard (it’s not necessarily the most exciting work in the universe!) but necessary work in the outcomes area means that we don’t have standards and conventions - people simply make it up as they go along - some times they get it right, sometimes they don’t. It would make life a lot simpler for everyone who has to work with outcomes systems if we had a set of well thought through rules for building and using them. (more…)
Avoid being an outcomes model ‘Go-Between’ September 26, 2007
Posted by Paul Duignan in : Doing evaluation more efficiently, Using the approach, Standards, Outcomes models, DoView , add a comment
A while ago a colleague recounted to me how they’d ended up pulling out their hair because they found themselves in a ‘Go-Between’ role when drawing an outcomes model (also called program logics, results chains, strategy maps, ends-means diagrams). You need to try to avoid this at all costs, although when dealing with high level stakeholders it’s often not easy to do so. I found myself in this role on a major project a while ago and I certainly didn’t enjoy it.
(more…)
Outcomes: Keep it simple - but not simplistic! September 14, 2007
Posted by Paul Duignan in : Strategic planning, Communicating outcomes models, Standards, Outcomes models , 1 comment so farI was in on an interesting conversation a while ago between a strategic planner and a communications specialist in a larger organization. The issue they were discussing was how to communicate to staff the outcomes model which had been developed by the management team for strategic planning purposes. The communications person was saying that at first sight the outcomes model seemed too complex to communicate to staff and that they thought that it should be simplified down to four or so points which they could get through to staff. In fact, as far as outcomes models go, I didn’t think it was a very complex outcomes model at all. I don’t know what happened in the end in that organization - Comms may have worked out a clever and clear way of communicating all of the important elements in the full outcomes model to staff. However, the conversation led me to reflect on the issue of keeping things simple in outcomes modeling. (more…)
Can outcomes, monitoring and evaluation planning be standardized? July 22, 2007
Posted by Paul Duignan in : Systematic Outcomes Analysis, Standards, Evaluation planning , 1 comment so farI was involved in an interesting discussion recently with a group of evalutors about whether outcomes, monitoring and evaluation planning can be standardized. In my experience, much evaluation planning starts from a blank slate with evaluators and project staff sitting around wondering about how they’re going to evaluate a specific program. Or, in other cases, for budgetary or other reasons, people who are not trained in evaluation have to work their way through basic texts about evaluation trying to work out how to do an evaluation. This all takes a great deal of time. Does it have to be like this? I’m not sure that it does.
Every time an organization sets up an accounting system, you don’t get the feeling that accountants have to build the entire accounting system from scratch. They simply put in place a number of basic building blocks of such systems and tailor them to the requirements of the particular organization. Why should monitoring and evaluation be any different? What I’ve been trying to do over a number of years in developing Systematic Outcomes Analysis is to develop such a standardized approach. (more…)
New outcomes modeling standards released November 1, 2006
Posted by Paul Duignan in : Standards, Outcomes models , add a commentThese days many people are involved in drawing outcomes models for organizations or projects. They go by various names - intervention logics, program logics, causal chains, results chains, program theories, program theories of action. I get to look at a lot of them and, to be frank, many of the ones I see are problematic. They’re either all over the place, or in many cases they are so limited (by limiting them to measurable and attributable outcomes - see later in this post) that they’re almost useless for most purposes. Don’t get me wrong, there’re also people out there drawing good models, but its often in spite of the received wisdom about drawing such models, rather than being because of it.
The problem with the bad outcomes models is that many of them are drawn using arbitrary rules which severely limit their usefulness. For instance in some cases, only three levels are allowed within such models (outputs, intermediate outcomes, final outcomes). While perhaps some simple programs can be modeled within this constraint (I suspect that there are few programs which can), increasingly I’ve come to think that it’s often crazy to limit people in this way when they are drawing outcomes models. I’ll be saying why in detail in later blog postings. (more…)