Theory of Change Versus Theory of Action January 15, 2010
Posted by Paul Duignan in : Attribution, Doing evaluation more efficiently, Outcomes theory, Standards, Using the approach , 2commentsWhat’s the difference between a Theory of Change and a Theory of Action? I’m just clarifying my thoughts on this issue and how it relates to my work thinking about how we conceptualize outcomes models (logic models) within outcomes theory. In summary, at the moment – apart from a Theory of Action just being an outcomes model drawn at a lower level – I can’t see a major difference. However I’m happy to be contradicted on this and will change my view if there are convincing arguments for making the distinction. My current thinking is as set out below. (more…)
The evolution of the logic model January 13, 2010
Posted by Paul Duignan in : Communicating outcomes models, Doing evaluation more efficiently, DoView, Outcomes models, Standards, Using the approach , 4commentsI’ve just posted an article on the evolution of the logic model within evaluation. Over the last couple of decades, increasing numbers of evaluators have started using logic models. For those not familiar with what logic models are – they are simply tabular or visual representations of all of the lower-level steps needed to achieve high-level outcomes for a program, organization or other intervention. They go by different names, for instance: program logics, intervention logics, results maps, theories of change, program theories, results hierarchies, strategy maps, end-means diagrams etc.). A traditional way of drawing logic models has evolved (known as the inputs, outputs, intermediate outcomes, final outcomes structured logic model) which often attempts to restrict logic models to a single page. However, many evaluators are now breaking away from the constraints of this traditional format and exploring various alternative ways of representing logic models. (more…)
Getting outcomes creds and saving time! September 25, 2009
Posted by Paul Duignan in : Accountability, Doing evaluation more efficiently, DoView, Easy Outcomes, Outcomes models, Outcomes systems architecture, Philanthropy, Standards, Strategic planning, Using the approach , add a commentPublic sector organizations these days have two important imperatives: establishing that they are truly ‘results and outcomes-focused’ while also becoming more efficient in their internal organizational activity. The really good news in the outcomes area is that using a central tool of outcomes work – outcomes models (a particular type of visual model of all of the high-level outcomes the organization is seeking to achieve and the steps it is taking to do so) is that organizations and programs can do both at the same time. (more…)
Intense analysis of the U.N. Results-Based Management System May 5, 2009
Posted by Paul Duignan in : Accountability, Attribution, Doing evaluation more efficiently, Indicators, Measurement, Outcomes models, Outcomes systems architecture, Outcomes theory, Reporting systems, Standards, Using the approach , add a commentI have just put up an Outcomes Theory Knowledge Base article which is an intense analysis of the United Nation Results-Based Management System. (Its obscure work, but someone has to do it!). The exciting part is that it has let me road-test my new Outcomes Systems Checklist. This now provides a common framework for analyzing any outcomes system – outcomes systems being any system which attempts to identify, measure, attribute or hold parties to account for outcomes or the steps which it is thought lead to them. A 2008 report from the U.N. itself on its Results-Based Management System said that the system was: ‘an administrative chore of little value to accountability and decision-making”.
Why just about every indicator system in the world needs to be fixed! April 5, 2009
Posted by Paul Duignan in : Accountability, Communicating outcomes models, Doing evaluation more efficiently, Easy Outcomes, Evaluation planning, Indicators, Measurement, Outcomes models, Outcomes systems architecture, Reporting systems, Standards, Using the approach , 1 comment so farI’ve just posted a new article in the Outcomes Theory Knowledge Base on why it is essential to map indicators onto an underlying visual outcomes model. I blogged a little while ago about why we should be wary of too-tidy indicator sets and in the article I explain why. The vast majority of indicator systems in the world suffer from the problem set out in the article – they are just a straight list of indicators set out in tabular format. They give the user no idea as to whether a number of important steps and outcomes are not being measured. Those using such systems remain blissfully unaware of this. In my view, all these straight indicator sets need to be fixed. It’s not particularly difficult, it just requires some work. How to draw the underlying outcomes models is set out in the outcomes model standards and how to then use such models for indicator mapping and many other things is described in detail in the applied version of outcomes theory – Easy Outcomes. (more…)
Beware lumpers, splitters and slice globblers when you’re building outcomes models October 15, 2007
Posted by Paul Duignan in : DoView, Outcomes models, Standards, Using the approach , add a comment
When you’re drawing outcomes models (program logics, strategy maps, means-ends diagrams, results chains etc), using DoView or other software there’re a few things which will come up on a regular basis. The first of these is the personality difference between lumpers and splitters who are present in the room. Lumpers, obviously want to lump and splitters, well, they just want to split. So if you’re working on an outcomes model which at a high level includes, say, social and economic outcomes, a lumper will want a single outcome which goes ‘complete social and economic whatever’. A splitter, on the other hand, will want to have two separate outcomes – a ‘social whatever’ and an ‘economic whatever’ outcome. Neither of them is right or wrong, although in the first instance I often let the splitter have their way because outcomes or steps can always be combined at a later stage if needed. (more…)
Outcomes police, do they exist? October 7, 2007
Posted by Paul Duignan in : Outcomes theory, Standards , add a comment
Several days ago I came across an article by John Day asking the question – Accounting Police, do they exist? In it he talked about the role of the Accounting Standards Board (FASB) in the U.S. Members of this board, ‘Go through a lengthy process of analyzing and reviewing problems in the accounting field that are brought to them. After much thought, they will make a pronouncement as to what they think the new or revised way of approaching the treatment of an accounting issue should be.’
There’s no such body for the outcomes field, I think that there should be one. The absence of people doing this hard (it’s not necessarily the most exciting work in the universe!) but necessary work in the outcomes area means that we don’t have standards and conventions – people simply make it up as they go along – some times they get it right, sometimes they don’t. It would make life a lot simpler for everyone who has to work with outcomes systems if we had a set of well thought through rules for building and using them. (more…)
Avoid being an outcomes model ‘Go-Between’ September 26, 2007
Posted by Paul Duignan in : Doing evaluation more efficiently, DoView, Outcomes models, Standards, Using the approach , add a comment
A while ago a colleague recounted to me how they’d ended up pulling out their hair because they found themselves in a ‘Go-Between’ role when drawing an outcomes model (also called program logics, results chains, strategy maps, ends-means diagrams). You need to try to avoid this at all costs, although when dealing with high level stakeholders it’s often not easy to do so. I found myself in this role on a major project a while ago and I certainly didn’t enjoy it.
(more…)
Outcomes: Keep it simple – but not simplistic! September 14, 2007
Posted by Paul Duignan in : Communicating outcomes models, Outcomes models, Standards, Strategic planning , 1 comment so farI was in on an interesting conversation a while ago between a strategic planner and a communications specialist in a larger organization. The issue they were discussing was how to communicate to staff the outcomes model which had been developed by the management team for strategic planning purposes. The communications person was saying that at first sight the outcomes model seemed too complex to communicate to staff and that they thought that it should be simplified down to four or so points which they could get through to staff. In fact, as far as outcomes models go, I didn’t think it was a very complex outcomes model at all. I don’t know what happened in the end in that organization - Comms may have worked out a clever and clear way of communicating all of the important elements in the full outcomes model to staff. However, the conversation led me to reflect on the issue of keeping things simple in outcomes modeling. (more…)
Can outcomes, monitoring and evaluation planning be standardized? July 22, 2007
Posted by Paul Duignan in : Evaluation planning, Standards, Systematic Outcomes Analysis , 1 comment so farI was involved in an interesting discussion recently with a group of evalutors about whether outcomes, monitoring and evaluation planning can be standardized. In my experience, much evaluation planning starts from a blank slate with evaluators and project staff sitting around wondering about how they’re going to evaluate a specific program. Or, in other cases, for budgetary or other reasons, people who are not trained in evaluation have to work their way through basic texts about evaluation trying to work out how to do an evaluation. This all takes a great deal of time. Does it have to be like this? I’m not sure that it does.
Every time an organization sets up an accounting system, you don’t get the feeling that accountants have to build the entire accounting system from scratch. They simply put in place a number of basic building blocks of such systems and tailor them to the requirements of the particular organization. Why should monitoring and evaluation be any different? What I’ve been trying to do over a number of years in developing Systematic Outcomes Analysis is to develop such a standardized approach. (more…)