Is it the role of an evaluator to always ‘value’ what they are evaluating? January 21, 2010
Posted by Paul Duignan in : Evaluation debates, Use of terms , 8commentsI’ve had occasion recently to need to think about whether or not the notion of ‘valuing’ something is always an essential part of evaluation. To question this may seem a heresy to some evaluators who see this as the defining aspect of evaluation (for instance as opposed to ‘research’ where they don’t see such valuing as needing to take place). I’m not definite in my thoughts on this issue and below just want to float the argument which has been rattling around in my head for a while and which I have not had a chance to get down in writing to see if it can be shot down – in which case I will change my mind. (more…)
Untangling evaluation terms – discussing evaluation ‘types’ with clients often more useful than evaluation ‘approaches’ August 24, 2009
Posted by Paul Duignan in : Doing evaluation more efficiently, Evaluation debates, Evaluation planning, Use of terms, Using the approach , add a commentI have just put up a outcomes theory article based on a book chapter I wrote some time ago dividing the terminology used in evaluation into five groups of terms about five different ‘aspects’ of evaluation. These aspects are: evaluation approaches; evaluation types (based on the purpose of the evaluation); evaluation methods; evaluation information analysis techniques; and evaluation designs. Approaches tend to combine a range of different elements including general approaches to evaluation, philosophy of science views and for instance, quasi-political perspectives on the relationship between empowered and disempowered groups. Evaluation approaches are often not mutually exclusive from each other from a conceptual point of view. Evaluation approaches include such things as Scriven’s Goal Free Evaluation, Patton’s Utilization Focused Evaluation and Fetterman’s Empowerment Evaluation. While I find these very interesting from the point of view of stimulating my thinking about evaluation, I often (but not always) do not find them very useful when talking to a client about a specific evaluation.
(more…)
Formative evaluation versus impact/outcome evaluation August 20, 2009
Posted by Paul Duignan in : Doing evaluation more efficiently, Easy Outcomes, Evaluation debates, Evaluation planning, Outcomes theory, Use of terms, Using the approach , add a commentIn response to a posting on one of my outcomes theory articles by Marcus Pilgrim who ran the recent YEN Evaluation Clinic in Damascus, I have worked up an article on the difference between formative, process and impact/outcome evaluation. As Marcus points out in his posting, the term formative (or developmental) evaluation is not one which is widely known in all sectors. Formative evaluation is directed at optimizing program implementation. Process evaluation attempts to describe the course and context of a program. Impact/outcome evaluation looks at the intended and unintended, positive and negative outcomes of a program and whether they can be attributed to the program. (more…)
What we are all on about – representing causal models April 19, 2009
Posted by Paul Duignan in : Communicating outcomes models, Doing evaluation more efficiently, Outcomes models, Outcomes systems architecture, Strategic planning, Use of terms, Using the approach , add a commentWhether we know it or not, a lot of us in evaluation, monitoring, social programs, philanthropy etc. spend a lot of time working with ‘causal models’. We call them all sorts of things – program justifications, rationales, program activities and objectives, logic models, logframes, intervention logics, strategy maps etc. – and most people who work with them don’t think of them as causal models. But that’ s what they are if we see causal models as just being an attempt to set out ‘what it is believed causes what in the world’. In the case of a program, the model is going to be a model of the steps which you think a program needs to take in order to cause high-level outcomes to occur. We really should get our heads around the best way to represent such models because at the moment I think that there is a great deal of wasted talk and effort about all of this. And it distracts us from getting on with the job of implementing good programs as fast as possible. Every dollar or every hour spent on struggling with an inefficient way of representing our program is a dollar or hour wasted. (more…)
What exactly is ‘best practice’? April 2, 2009
Posted by Paul Duignan in : Evaluation debates, Impact evaluation, Outcomes theory, Philosophy of science, Use of terms, Using the approach , add a commentIdentifying and communicating best practice is widely recommended in many sectors and disciplines. But I’ve sometimes wondered as I’ve sagely recommended in a serious voice, ‘I think that we should use an approach based on identifying and implementing best practice here’ exactly what best practice is? I think that doing it is often a good idea and I can work out how to identify it and share it, and I will blog about that tomorrow, but what I’m not clear on is exactly how we define ‘best’ in the term ‘best practice’. It’s not clear whether best practice consists of: 1) claims that practitioners, from their own experience, believe the practices concerned to be feasible and ‘useful’ to implement; or 2) practices which have been proven to improve high-level outcomes (through making a strong outcome/impact evaluation claim of some sort such as is made using some of the types of designs listed here). (more…)