Using large visualized outcomes models in policy development December 18, 2008
Posted by Paul Duignan in : Outcomes systems architecture, Strategic planning, Communicating outcomes models, Outcomes models, DoView , 1 comment so far
I recently presented a paper to the European Evaluation Society Conference in Lisbon on the use of large visualized outcomes models in policy development. The conference theme was on ways that evaluation could contribute to policy development. I see visualized outcomes models as one way that evaluators can contribute. Check out the full paper here. Evaluators may see themselves as ideally suited to contribute to high level policy making, however, regardless of evaluators’ views, the powerful players already involved in policy development (stakeholders, politicians, senior public servants, policy analysts, economists and others) need to be convinced that the evaluation discipline can add real value to the policy making process. What actual value can evaluation add? (more…)
Evaluation Advisory Committees - getting it right at the start December 16, 2008
Posted by Paul Duignan in : Doing evaluation more efficiently, Using the approach, Evaluation planning , 1 comment so far
Recently I attended the first meeting of an Evaluation Advisory Committee and I was reminded once again of the importance of setting up evaluations properly at the start. I have set out below the issues which need to be dealt with when such committees are set up. This list is drawn from my experience on a number of these types of committees. The issues you need to deal with include: (more…)
Getting evidence implemented in practice - easier said than done - the Rich Dialog Process December 14, 2008
Posted by Paul Duignan in : Uncategorized , add a comment
Getting evidence implemented in practice is a lot more difficult than just waving around the odd evaluation report and pointing someone at an evidence-based web site like the Cochrane or Campbell collaboration sites. I was involved in a process earlier this year which was an attempt to work with a sector to get evidence-based practice implemented in practice.It used the Rich Dialog Process (RDP) - a simple deliberation and dialog process to get groups of stakeholders who may have different views on an issue to talk those differences through. The focus was on getting evidence-based practice really happening in the school drug and alcohol education area. An area in which there are a lot of different types of programs running. (more…)
More on politicians - accountability (praise and blame) and having ‘blown it’ December 10, 2008
Posted by Paul Duignan in : Evaluation debates, Reporting systems, Outcomes systems architecture, Outcomes theory, Accountability, Indicators, Outcomes theory & the news , add a comment
Further to my last posting on whether or not the jump in President Bush’s approval rating was attributable to him (i.e. provable that he caused most of it, rather than it just being the fact that he happened to find himself being President on September 11th), I came across a letter to Newsweek which relates to another aspect of the features of steps and outcomes in outcomes models as related to politicians. (The letter is the second letter listed here). This time it is the issue of accountability. (more…)
Attibution and President Bush’s approval rating December 7, 2008
Posted by Paul Duignan in : Impact evaluation, Attribution, Reporting systems, Outcomes systems architecture, Indicators, Outcomes theory & the news, Accountability, Measurement , add a comment
I’ve been doing a lot of thinking about indicators and attribution these days. I think that a lot of problems (particularly misinterpretations of how well or badly parties are doing within outcomes systems of all types arise because of confusion between attributable and not-necessarily attributable indicators. The underying Five Building Blocks Diagram which is used in outcomes theory makes a distiction between attributable and not-necessarily attributable indicators.The Washington Post today had an interactive showing President Bush’s Approval Rating over time. At various points, the graph had labels showing what was happening at the time, presumably to help the reader work out why his rating went up and down at the points it did. The biggest jump was around September 11 where his approval took a massive leap from something like 56% to 92% (reading off the graph). Below is a section from the interactive. (more…)
Sharing the pain - crazy indicators, targets and funder reporting systems December 4, 2008
Posted by Paul Duignan in : Indicators, Outcomes systems architecture, Reporting systems, Accountability, Measurement, Easy Outcomes, Using the approach, Doing evaluation more efficiently, DoView , add a comment
This blog post is a follow-up to an earlier posting on my blog. In a comment on that posting, M&Egirl outlined problems she was having with a funder demanding targets that are meaningless and asked for more comment on this obsession with targets on the part of funders.
The program she is working on is a Gender Related Violence program, which for the sake of this discussion I’ll presume involves women who have been subject to gender related violence receiving some sort of intervention to help them be safe and move beyond the trauma they have suffered. Because the issue of indicators and targets is such an important issue I thought that I’d do a further posting on it. If you have a moment, have a quick read of the original posting and M&Egirl’s comment before reading this blog posting. (more…)
Impact evaluation’s day in the sun (Part III) December 3, 2008
Posted by Paul Duignan in : Attribution, Evaluation debates, Impact evaluation, Outcomes systems architecture, Evaluation planning , 1 comment so far
[Please Read Part I and Part II first]. Now that I have got to Part III of this posting on the impact evaluation debate, the time has come for me to front up and tell you how I think things should be done if I were in charge of the world. First, the concept of there being a shoot-out between impact evaluation and other types of evaluation is about as sensible as trying to have a debate as to whether a fork or a knife is the better kitchen utensil. It obviously depends on what you want to do with it. This does not, of course mean that it might not be reasonable to argue that there are not enough knives in the kitchen at a particular point in time and that we should go out and encourage knife collecting rather than just getting more and more forks. (more…)
Impact evaluation’s day in the sun (Part II) December 1, 2008
Posted by Paul Duignan in : Attribution, Impact evaluation, Measurement, Uncategorized , 1 comment so far
(See the previous posting Part I first). Progressing the debate around impact evaluation means unpacking some of the different issues which are involved in the debate. In doing this, it should not be assumed that those on various sides in the debate will always side in the same way with each of the sub-issues. In my doctorate (Duignan, 1997) I did this type of analysis for the positivist versus constructivist debate which was hot for a while within evaluation circles. Drawing on that analysis, we can now to look at the sub-issues which underly the impact evaluation debate because there are a number of similarities with the earlier debate. The sub-issues which can underly the impact evaluation debate and two contrasting positions that can be taken on them include the following: (more…)