jump to navigation

Tracking jobs created under the U.S. Recovery Act - when should the attempt at measurement be abandoned? June 16, 2009

Posted by Paul Duignan in : Impact evaluation, Outcomes systems architecture, Attribution, Reporting systems, Outcomes theory & politics, Indicators, Accountability, Using the approach, Doing evaluation more efficiently, Measurement, Outcomes theory & the news, Evaluation planning , add a comment

The default expectation in at least some sections of the U.S. public sector seems to be that it should always be feasible and affordable to both measure and attribute the results of interventions. This is using the term attribution to mean being able to actually demonstrate that a change in an outcome has been caused by a particular intervention rather than being the result of other factors (see here for more on attribution). The recent U.S. Recovery Act is a case in point.  While it’s reasonable to start from the position that you should routinely assess the possibility of measuring and attributing changes in outcomes of particular interventions, you can’t start by just assuming that it will always be feasible or affordable to do this. Clinging to such an assumption, where it is untrue, can result in you either measuring an outcome when the data you are collecting is not accurate, or acting as though what you are measuring (even if it is an accurate measurement of a change in an outcome) is demonstrably attributable to a particular program, when in fact it may not be.  (more…)

Social Innovation, evaluation and outcomes April 6, 2009

Posted by Paul Duignan in : Impact evaluation, Outcomes theory & politics, Research influening policy, Outcomes theory, Using the approach, Easy Outcomes, Outcomes models, DoView , add a comment

I attended a launch of the New Zealand national Center for Social Innovation last night. Geoff Mulgan from the Young Foundation (a similar center in the U.K.) talked about social innovation. The social innovation movement is about getting stakeholders and sectors together to do things differently to achieve better social outcomes. Already a dynamic movement, it has recently received a shot in the arm from the global economic melt-down - traditional ways of doing things are increasingly being questioned and people are looking for new solutions. A number of points made by Geoff and in the subsequent discussion are particularly relevant to outcomes and evaluation were: (more…)

Extraordinary circumstances and Dick Cheney’s ’stuff happens’ March 29, 2009

Posted by Paul Duignan in : Attribution, Outcomes theory & politics, Evaluation debates, Outcomes theory, Accountability, Outcomes theory & the news , add a comment

In my last blog posting I commented on Jon Stewart’s critique of ex-Vice President Dick Cheney’s claim that the Bush administration should not be held accountable for the U.S. economic melt-down because a number of things happened during their term which affected the economy. The Vice President summarized this by saying that ’stuff happens’ and this ’stuff’ unexpectedly blew their budget. The ’stuff’ included the wars in Afghanistan and Iraq, and Hurricane Katrina. In technical outcomes theory terms, the Vice-President was mounting an ‘extraordinary factors’ argument to reduce his administration’s accountability for the economic melt-down. (more…)

To attribute or not to attribute - Jon Stewart vs Dick Cheney March 27, 2009

Posted by Paul Duignan in : Attribution, Outcomes theory & politics, Evaluation debates, Outcomes theory, Accountability, Outcomes theory & the news , 3comments

In a recent episode of Jon Stewart’s Daily Show he deconstructs a high-profile interview with Dick Cheney, the previous Vice President of the United States, undertaken by another interviewer. While this is a comedy show, being an overly analytical sort of person, I can’t watch it without analyzing what is going on it in from the point of view of outcomes theory! At a technical level, the key issue Stewart is focusing on in creating his laughs is what in outcomes theory is called - demonstrable attribution. Demonstrable attribution is being able to demonstrate that an improvement which occurs following an intervention has been caused by a particular intervention (see here for more outcomes theory information on this). In summary, Stewart claims that Dick Cheney is applying a double standard around demonstrable attribution. (more…)

If all knowledge is subjective, why listen to evaluators? March 24, 2009

Posted by Paul Duignan in : Philosophy of science, Outcomes theory & politics , 1 comment so far

This week I’m blogging on a Rich Dialog Process (RDP) I facilitated recently on the interface between research (and evaluation) and policy. Check out my last couple of blog postings for more information on the actual process. One of the most interesting moments for me was a relatively short interchange around the issue of research ‘agendas’ and subjectivity and objectivity in research. I decided that given the range of issues we had to cover in the RDP there was not enough time to get right into this topic, important though it is, and the participants did not seem to want to get  into it at that moment, so we moved on. However, this is an issue which I’ve been interested in for a long time. (more…)

Time for an Attribution Commission? March 17, 2009

Posted by Paul Duignan in : Outcomes theory & politics, Attribution, Accountability, Economic analysis, Outcomes theory & the news , 1 comment so far

A blog posting by Dean Baker (he is one of the few economists who accurately predicted the housing collapse) looks at the ‘comparative responsibility’ of Bush and Obama for the 2009 U.S. deficit in the face of some Republican claims that somehow it is Obama’s fault. He argues that most of it is the legacy of Bush. Whenever an administration changes you get endless debate as to who is responsible for what. In fact much of political discourse is about attribution and accountability, where attribution is about attempting to demonstrate that an agent caused something to happen and accountability is whether they should be punished or rewarded for what has happened. (See the article here for more on attribution and accountability within outcomes theory). (more…)

Bonuses role in the financial melt-down March 16, 2009

Posted by Paul Duignan in : Outcomes theory, Reporting systems, Outcomes theory & politics, Indicators, Accountability, Measurement, Outcomes theory & the news, Outcomes models , add a comment

President Obama has amplified the attack on bonuses being paid to staff in companies which have been bailed out by the U.S. government (CNN, 16 March 2009). What does outcomes theory have to say about the role of the bonus system in the current financial meltdown? I blogged in 2007 about the problem of bonuses in the financial system and how it was possible the full extent of the melt-down would take time to be revealled. Thinking in terms of an outcomes model, what has happened is that financial institutions should have been aiming (as they do in healthy times) at the overall outcome of Sustainable long-run profitability. (more…)

14 years for revealing an indicator! February 8, 2009

Posted by Paul Duignan in : Outcomes theory & politics, Indicators, Outcomes theory & the news , 1 comment so far

One of the principles of outcomes theory is that if you want groups of stakeholders to not have a complete understanding of what is happening as an outcomes model plays out, you try to suppress information about indicators of outcomes which are heading in the wrong direction. There is currently a story in the media about a UK officer being arrested in Afghanistan for allegedly supplying civilian casualty figures to a human rights campaigner. The CNN story is here.

(more…)