Damascus – YEN Evaluation Clinic August 12, 2009
Posted by Paul Duignan in : Doing evaluation more efficiently, Evaluation planning, Impact evaluation, Outcomes systems architecture, Using the approach , trackbackApologies for not blogging for a while, I’ve been involved in considerable travel and lots of other work – but that’s really no excuse. Maybe I just got all blogged out. What with Knolling, Blogging here and Twittering, maybe it all just got too much. Anyway, I’m back in the saddle now as they say! Last month I was fortunate to be an evaluation expert at the YEN Evaluation Clinic in Damascus. YEN is the Youth Employment Network – an International Labour Organization, World Bank, United Nations collaboration. A site for the evaluation clinic has been set up at yenclinic.groupsite.co.
The Evaluation Clinic took two examples of youth employment programs and worked through designing an impact evaluation for them. It was a fascinating experience. I’ll blog about what it was like being the sole psychologist evaluator working with five economists evaluation specialists (from the ILO and the World Bank) another day!
One of the first things which came of out the clinic for me was the realization of how we need to develop better ways of communicating to busy program staff the technical aspects of designing impact evaluations. It is complicated stuff and we need to work out how to communicate it quickly and effectively using examples that program staff can understand. Of course, building technical knowledge regarding how to do impact evaluations is only one part of impact evaluation capability building (the other part is increasing people sophistication regarding when impact evaluation should, and should not, be done – but I’ll also blog about that later).
Anyway, being involved in the YEN Evaluation Clinic inspired me to develop what I’m calling an Impact/outcome evaluation decision-making table and checklist approach to impact evaluation planning. What I realized is that as evaluators we often talk to program staff about evaluation approaches and designs. However, the thing that program staff know most about is their program. So maybe the best way to approach teaching them about impact evaluation designs is to start from the point of view of the constraints which exist in regard to their program (e.g. not being able to withhold the intervention from a control group) and work back from this to help them determine the most appropriate evaluation design for their particular program.
This is the approach I’ve taken in the impact evaluation decision-making table and checklist approach. The user sees a table which along the top sets out constraints regarding the program and then they can look down the table under any constraint which applies to their program to work out which of a set of seven possible impact/outcome evaluation designs is likely to be the most appropriate.
This whole thing is very much a draft so I’d really appreciate any comments people have on how to improve it. Put them at the bottom of the Knol article. It is at:
Duignan, P. (2009). Selecting impact/outcome evaluation designs: a decision-making table and checklist approach. Outcomes Theory Knowledge Base Article No. 256. (http://knol.google.com/k/paul-duignan-phd/selecting-impactoutcome-evaluation/2m7zd68aaz774/115).
Paul Duignan, PhD
(Outcomes and evaluation blog OutcomesBlog.org)
Comments»
Paul:
While you’re on the subject check out our calls for proposals related to “Doing Impact Evaluation” (http://3ieimpact.ning.com/)
Regard
Ron Bose, PhD
3ie
Hi Ron, I had a quick look for what you are referring to and could not locate something exactly by that name. Will check again.
Thanks Paul.