The evolution of the logic model January 13, 2010
Posted by Paul Duignan in : Communicating outcomes models, Doing evaluation more efficiently, Using the approach, Standards, Outcomes models, DoView , 3commentsI’ve just posted an article on the evolution of the logic model within evaluation. Over the last couple of decades, increasing numbers of evaluators have started using logic models. For those not familiar with what logic models are - they are simply tabular or visual representations of all of the lower-level steps needed to achieve high-level outcomes for a program, organization or other intervention. They go by different names, for instance: program logics, intervention logics, results maps, theories of change, program theories, results hierarchies, strategy maps, end-means diagrams etc.). A traditional way of drawing logic models has evolved (known as the inputs, outputs, intermediate outcomes, final outcomes structured logic model) which often attempts to restrict logic models to a single page. However, many evaluators are now breaking away from the constraints of this traditional format and exploring various alternative ways of representing logic models. (more…)
How many evaluators does it take to change a light bulb? January 9, 2010
Posted by Paul Duignan in : Communicating outcomes models, Using the approach, Outcomes models , add a commentIn response to a series of ‘How many evaluators does it take to change a light bulb?’ jokes on the evaluators list EVALTALK, I whipped up an outcomes model (logic model) for a Changing Light Bulbs Project (some days one does wonder if this is what evaluators do for fun, it must be some sort of illness!).
Anyway here it is http://www.outcomesmodels.org/models/lightbulb62.html
Paul Duignan, PhD. (Follow me on my Outcomes Blog; Twitter; or via my E-Newsletter).
Flow of causality in outcomes models and feedback loops August 14, 2009
Posted by Paul Duignan in : Measurement, Outcomes theory, Communicating outcomes models, Using the approach, Outcomes models, DoView , 4commentsA quick technical blog here. Fellow evaluator Rick Davies pointed out in a post on one of my outcomes theory articles (on how to best represent causal models), that strictly visualizing causality as flowing in one direction within an outcomes model (logic model, results map, logframe, theory of change etc.) could be seen as preventing the representation of feedback loops. This is because if you are, as I usually do, representing causality as flowing from bottom to top within a model (others do it left to right) then when you want to represent a feedback loop it will, of necessity, have to flow back down the logic model against the direction in which causality is being represented. (more…)
The good old ‘one pager’ contraint again July 9, 2009
Posted by Paul Duignan in : Doing evaluation more efficiently, Outcomes systems architecture, Communicating outcomes models, Using the approach, Easy Outcomes, Outcomes models, DoView , add a commentI’ve been up against the good old ‘one pager’ constraint again in a couple of contexts recently. This is where there is the demand that the outcomes for an organization or program be ‘put on one page’. It comes in various forms and is often a demand from senior management or a perceived demand from them to ‘keep things simple’. Now, there’s nothing wrong with the idea of summarizing things and paper overviews play a role in that. But such things should be seen as one of a range of different types of summaries and products which are produced by an underlying outcomes system, not the beginning and end of an outcomes effort. (more…)
Separating analysis from writing in philanthropic grant applications July 8, 2009
Posted by Paul Duignan in : Grant writing, Philanthropy, Strategic planning, Communicating outcomes models, Easy Outcomes , add a commentThe other night I was doing some pro bono work with a small non-profit organization which is attempting to restore a significant landmark building and promote its use for educational and community purposes. I was building a DoView results map (outcomes model) for them in real-time in the course of one of their monthly meetings. They’re using the visual model as their strategic planning approach (instead of the traditionally long narrative text-based plans many people in such small organizations spend many hours sweating over).
Building a DoView model is a very quick way for a community organization to build a simple but effective strategic plan. Once built they can then show others the model and quickly communicate that they’ve thought through what they’re planning to do. (more…)
Visual model of what I’m trying to do with my outcomes work April 28, 2009
Posted by Paul Duignan in : Communicating outcomes models, Outcomes theory, Using the approach, Outcomes models, DoView, Easy Outcomes, Blog info , 2commentsThought that I would apply a taste of my own medicine to my own work, so I drew a visual outcomes model of what it is that I’m trying to do with my work in the outcomes area. It is here. At the top is my high level outcome ‘Make working with outcomes, monitoring, evaluation etc. easier’ and below that is all of the lower-level steps I am using to get to this high-level outcome. I have included hyperlinks out to the various web sites where I am attempting to do the things listed in the lower-level steps.
Paul Duignan, PhD
Outcomes and Evaluation Blog (OutcomesBlog.org)
Making outcomes theory more concrete - checklist for assessing outcomes systems April 21, 2009
Posted by Paul Duignan in : Outcomes theory, Outcomes systems architecture, Attribution, Reporting systems, Accountability, Measurement, Using the approach, Communicating outcomes models, Strategic planning, Doing evaluation more efficiently, Outcomes models , 1 comment so farMost normal people would think that it’s very very obscure, but I’ve just put up a Checklist for Analyzing Outcomes Systems in the Outcomes Theory Knowledge Base and it’s a very exciting development. Up until now the Outcomes Theory Knowledge Base has consisted of a set of articles which outline various aspects of outcomes theory. Outcomes theory is a general theory which covers all types of outcomes systems. Outcomes systems are any type of performance management system, results-base system, monitoring system, evaluation system, outcomes-focused contracting system, or strategic planning system (the term even includes evidence-based practice systems). Such systems have, in the past, been seen as somewhat different types of things without a common theory existing to analyze them. Outcomes theory is based on the insight that we can theorize them as a common type of system and then use the theory to work out how such systems should be best structured. This approach becomes powerful at the moment that we can start applying it to actual real-world outcomes systems. This is the role of the checklist I’ve just developed. (more…)
What we are all on about - representing causal models April 19, 2009
Posted by Paul Duignan in : Outcomes systems architecture, Use of terms, Doing evaluation more efficiently, Strategic planning, Using the approach, Communicating outcomes models, Outcomes models , add a commentWhether we know it or not, a lot of us in evaluation, monitoring, social programs, philanthropy etc. spend a lot of time working with ‘causal models’. We call them all sorts of things - program justifications, rationales, program activities and objectives, logic models, logframes, intervention logics, strategy maps etc. - and most people who work with them don’t think of them as causal models. But that’ s what they are if we see causal models as just being an attempt to set out ‘what it is believed causes what in the world’. In the case of a program, the model is going to be a model of the steps which you think a program needs to take in order to cause high-level outcomes to occur. We really should get our heads around the best way to represent such models because at the moment I think that there is a great deal of wasted talk and effort about all of this. And it distracts us from getting on with the job of implementing good programs as fast as possible. Every dollar or every hour spent on struggling with an inefficient way of representing our program is a dollar or hour wasted. (more…)
Why just about every indicator system in the world needs to be fixed! April 5, 2009
Posted by Paul Duignan in : Accountability, Measurement, Indicators, Outcomes systems architecture, Reporting systems, Doing evaluation more efficiently, Communicating outcomes models, Evaluation planning, Outcomes models, Standards, Using the approach, Easy Outcomes , 1 comment so farI’ve just posted a new article in the Outcomes Theory Knowledge Base on why it is essential to map indicators onto an underlying visual outcomes model. I blogged a little while ago about why we should be wary of too-tidy indicator sets and in the article I explain why. The vast majority of indicator systems in the world suffer from the problem set out in the article - they are just a straight list of indicators set out in tabular format. They give the user no idea as to whether a number of important steps and outcomes are not being measured. Those using such systems remain blissfully unaware of this. In my view, all these straight indicator sets need to be fixed. It’s not particularly difficult, it just requires some work. How to draw the underlying outcomes models is set out in the outcomes model standards and how to then use such models for indicator mapping and many other things is described in detail in the applied version of outcomes theory - Easy Outcomes. (more…)
Using visual outcomes models to describe and communicate best practice April 3, 2009
Posted by Paul Duignan in : Outcomes theory, Doing evaluation more efficiently, Outcomes systems architecture, Reporting systems, Research influening policy, Strategic planning, Communicating outcomes models, Easy Outcomes, Evaluation planning, Outcomes models, Using the approach, DoView , 1 comment so farYesterday I blogged about what is meant by the term ‘best practice’. As I said then, I think that there is some conceptual confusion out there about it, and I am not sure that I have yet tidied it up my own thinking about it in a satisfactory way. However, the great thing is that regardless of how we define it, the idea of identifying the types of things that people currently call best practice and communicating these between programs is a great idea. The most difficult thing in many cases is to get best practice actually applied to on the ground after we have identified it. I have put up an Outcomes Theory Knowledge Base article (link to the article is at the bottom of this posting) on the issue suggesting that an efficient way of describing and communicating best practice may be to use visual outcomes models (a type of logic model). (more…)