How many evaluators does it take to change a light bulb? January 9, 2010
Posted by Paul Duignan in : Communicating outcomes models, Outcomes models, Using the approach , 2commentsIn response to a series of ‘How many evaluators does it take to change a light bulb?’ jokes on the evaluators list EVALTALK, I whipped up an outcomes model (logic model) for a Changing Light Bulbs Project (some days one does wonder if this is what evaluators do for fun, it must be some sort of illness!).
Anyway here it is http://www.outcomesmodels.org/models/lightbulb62.html
Paul Duignan, PhD. (Follow me on my Outcomes Blog; Twitter; or via my E-Newsletter).
Developing an M&E plan using a visual approach January 8, 2010
Posted by Paul Duignan in : Doing evaluation more efficiently, DoView, Easy Outcomes, Evaluation planning, Using the approach , add a commentOn various lists I am on I often see requests by people wanting to develop what is called an M&E plan. This terminology is often used in the international development area. It refers to a Monitoring and Evaluation Plan. The way these requests are made makes me think that the way you should monitor and evaluate different projects varies a great deal. (more…)
Christmas is over – back to blogging January 7, 2010
Posted by Paul Duignan in : DoView, Outcomes models , 1 comment so farI’ve been quiet for some time on the blogging front as I’ve been developing a number of resources on outcomes modeling (logic modeling), evaluation and related topics. Now Christmas is over and I’m back to work I’ll be blogging in the next few weeks about these resources. I’ll also be blogging about DoView 2.0 which has just been released. It’s full of new features suggested by users and I’ve been building a range of really cool outcomes models with clients using the beta version and using that for input into its development. (more…)
Getting outcomes creds and saving time! September 25, 2009
Posted by Paul Duignan in : Accountability, Doing evaluation more efficiently, DoView, Easy Outcomes, Outcomes models, Outcomes systems architecture, Philanthropy, Standards, Strategic planning, Using the approach , add a commentPublic sector organizations these days have two important imperatives: establishing that they are truly ‘results and outcomes-focused’ while also becoming more efficient in their internal organizational activity. The really good news in the outcomes area is that using a central tool of outcomes work – outcomes models (a particular type of visual model of all of the high-level outcomes the organization is seeking to achieve and the steps it is taking to do so) is that organizations and programs can do both at the same time. (more…)
Using an outcomes modeling approach to action research September 9, 2009
Posted by Paul Duignan in : Doing evaluation more efficiently, Easy Outcomes, Outcomes models, Using the approach , add a commentWill get back to blogging on the Australasian Evaluation Society Conference when I get a moment (may not be for a few days). In the meantime had to prepare an article about using outcomes modeling as a basic tool within an action research approach. Because outcomes modeling – developing visual outcomes models (like a type of logic model, or theory of change model) – according to the outcomes theory set of standards for building such models is a generic process, such models can be used for a wide range of purposes. They can, for instance, be used within an action research approach. Action research is an approach which attempts to work in cycles of research/action/research. It has the great virtue of ensuring that research is connected to action and action is connected to research.
(more…)
The Taxi Driver and ‘why don’t you just measure outcomes’ – on the way to AES conference September 1, 2009
Posted by Paul Duignan in : Doing evaluation more efficiently, Easy Outcomes, Measurement, Outcomes models, Outcomes systems architecture, Reporting systems, Using the approach , add a commentOn my way to the Australasian Evaluation Society Conference in Canberra my taxi driver in from the airport asked me what I do. When I explained that I ‘measure whether programs, often government programs, work or not so the taxpayer gets value for money’, he was right into the concept. Although I think he thought that I was over complicating things a little. He said: ‘shouldn’t it just be a matter of using statistics to measure if things are getting better or not about a program.’ What he was talking about was one aspect of monitoring and evaluation – an important piece – but just one of the Five Building Blocks I see lying behind all monitoring and evaluation systems (outcomes systems). (more…)
Can an exhaustive list of impact evaluation designs be developed, or is my mission on this futile? August 27, 2009
Posted by Paul Duignan in : Attribution, Doing evaluation more efficiently, Easy Outcomes, Evaluation debates, Evaluation planning, Impact evaluation, Outcomes systems architecture, Outcomes theory, Using the approach , add a commentI have set out on a mission as a part of outcomes theory to attempt to develop an exhaustive list of impact/outcome evaluation designs – evaluation designs which make a claim that changes in high-level outcomes can be attributed to a particular intervention. If we could pull off developing such a list that most people are happy with, it would be very powerful. First it could be used in evaluation planning to work out if all of the possible impact evaluation designs had been assessed for their appropriateness, feasibility and/or affordability. At the moment I think that almost every evaluation planner walks around wondering if there is some sort of impact evaluation design they have not considered.
(more…)
Untangling evaluation terms – discussing evaluation ‘types’ with clients often more useful than evaluation ‘approaches’ August 24, 2009
Posted by Paul Duignan in : Doing evaluation more efficiently, Evaluation debates, Evaluation planning, Use of terms, Using the approach , add a commentI have just put up a outcomes theory article based on a book chapter I wrote some time ago dividing the terminology used in evaluation into five groups of terms about five different ‘aspects’ of evaluation. These aspects are: evaluation approaches; evaluation types (based on the purpose of the evaluation); evaluation methods; evaluation information analysis techniques; and evaluation designs. Approaches tend to combine a range of different elements including general approaches to evaluation, philosophy of science views and for instance, quasi-political perspectives on the relationship between empowered and disempowered groups. Evaluation approaches are often not mutually exclusive from each other from a conceptual point of view. Evaluation approaches include such things as Scriven’s Goal Free Evaluation, Patton’s Utilization Focused Evaluation and Fetterman’s Empowerment Evaluation. While I find these very interesting from the point of view of stimulating my thinking about evaluation, I often (but not always) do not find them very useful when talking to a client about a specific evaluation.
(more…)
Formative evaluation versus impact/outcome evaluation August 20, 2009
Posted by Paul Duignan in : Doing evaluation more efficiently, Easy Outcomes, Evaluation debates, Evaluation planning, Outcomes theory, Use of terms, Using the approach , add a commentIn response to a posting on one of my outcomes theory articles by Marcus Pilgrim who ran the recent YEN Evaluation Clinic in Damascus, I have worked up an article on the difference between formative, process and impact/outcome evaluation. As Marcus points out in his posting, the term formative (or developmental) evaluation is not one which is widely known in all sectors. Formative evaluation is directed at optimizing program implementation. Process evaluation attempts to describe the course and context of a program. Impact/outcome evaluation looks at the intended and unintended, positive and negative outcomes of a program and whether they can be attributed to the program. (more…)
Mapping indicators onto a logic model is obvious – but why haven’t we always done it? August 18, 2009
Posted by Paul Duignan in : Doing evaluation more efficiently, DoView, Easy Outcomes, Indicators, Measurement, Outcomes models, Outcomes theory, Reporting systems, Using the approach , add a commentI was running a workshop today teaching policy analysts the basics of my approach to program evaluation (Easy Outcomes). One of the participants, when I talked about the importance of always mapping indicators back onto a visual model, commented that when you do it, it’s so obviously the right approach that you can’t understand why we’ve not been doing it for years.
The idea behind this approach is that the way we almost always approach indicator work is to eye-ball a list or table of indicators and ask the question of a group of busy people sitting around a table – ‘does this list of indicators look any good?’
(more…)