Don’t assume that impact evaluation should always be done April 26, 2009
Posted by Paul Duignan in : Impact evaluation, Attribution, Outcomes systems architecture, Doing evaluation more efficiently, Evaluation planning, Easy Outcomes , trackbackImpact evaluation - evaluation which looks at whether changes in high-level outcomes can be attributed to a particular program, organization or other intervention - is a particularly useful type of evaluation when done properly. It clearly tells us what works, and what doesn’t work, and this information can be used in decision-making about which programs should, and should not, be funded in the future. However, particularly at the present time, with all of the enthusiasm for evidence-based practice, many people mistakenly assume that impact evaluation should always be attempted in regard to any program, organization or other intervention. Assuming this is a serious mistake. I’ve just put up an article in the Outcomes Theory Knowledge Base which sets out in detail why it is and the way we should approach assessing when impact evaluation should be done.
There are three steps in assessing whether impact evaluation (known by the formal name high-level outcome/impact attribution evaluation within outcomes theory). The first of these is to change perspective from the traditional ‘program-centric’ view of the world to a more ’sector-centric’ view. In a sector-centric view, evaluation is seen as being about collecting useful information for sector decision-making. This means that evaluation priorities are not determined by the individual program (a program-centric view) but by what needs to be found out in a whole sector. The second step is to work out whether you want to do an impact evaluation on what is called the ‘full program roll-out’ or if you just want to do it on a pilot and then just ensure that best practice from that pilot is applied in the full roll-out of a program. The third step, if your analysis from 1 and 2 above is that you still think that you want to do impact evaluation, is to carefully assess the appropriateness, feasibility and affordability of the types of impact evaluation design identified in outcomes theory. This means that you can reach a defensible decision as to whether or not it is a priority attempt to go ahead with an impact evaluation.
Check out the article and leave any comments you have at the bottom of it. It is:
Duignan, P. (2009). Impact evaluation - when it should, and should not, be done. Outcomes Theory Knowledge Base Article No. 242. (http://knol.google.com/k/paul-duignan-phd/-/2m7zd68aaz774/86)
Paul Duignan, PhD
Outcomes and Evaluation Blog (OutcomesBlog.org)
Comments»
no comments yet - be the first?