Op Ed on junk economic estimates August 22, 2011
Posted by Paul Duignan in : Uncategorized , add a commentI’m back blogging now. As an outcomes specialist, I’m always irritated when I see people making what I think are junk estimates. There’s a lot of this going on in economics from the Credit Rating agencies underestimating the risk of junk bonds through to various estimates plugged into all sorts of cost-benefit and other economic analyses that I see from time to time.
One I got my teeth into recently in an Op-Ed I wrote is an estimate regarding the cost of one of the many after-shocks which struck Christchurch NZ in its recent sequence of earthquakes. In this case, an estimate was made of one set of aftershocks which did not seem to take into account the fact that many of the buildings damaged in the after-shock had already been damaged in the earlier earthquakes. As one commentator said, ‘there are only so many times you can write off a building’!
This estimate was shot down by many people when it arose but the problem with junk economic estimates is that they tend to keep popping up like zombies like this one did in at a later date, but without the original criticism which put it in perspective. Anyway, check out the Op-Ed at http://www.outcomescentral.org/files/duignan299quake.pdf.
Paul Duignan, PhD. (Follow me on my OutcomesBlog.org; Twitter; or via my E-Newsletter; many resources at OutcomesCentral.org
DoView use for visual strategic planning growing April 30, 2010
Posted by Paul Duignan in : Uncategorized , add a commentMy best intentions for blogging on a regular basis have fallen by the wayside as I have been swamped with work from all directions. Just last week I spent the evenings webinaring into Cairo and into Europe as the demand for people wanting to know about advanced ways of using DoView outcomes and evaluation software are mounting. Interestingly, my day consulting work at the moment is focusing in on a number of assignments where DoView is being used for strategic planning, rather than evaluation work. I will be putting up some case studies of this work soon as some of the organizations I’m working for are graciously letting me share the work I’m doing with them. More in later blog postings on these case studies. (more…)
Developing a comprehensive sector visual outcomes model January 20, 2009
Posted by Paul Duignan in : Communicating outcomes models, DoView, Easy Outcomes, Indicators, Outcomes models, Reporting systems, Uncategorized , 2comments
In earlier blog postings I’ve talked about the use of large scale outcomes models for various purposes such as overall policy development, evidence-based practice and monitoring and evaluation. The next version of DoView outcomes and evaluation software [[Disclosure: I am involved in the development of DoView] is going to allow images to be included in DoView files and in the web page models which can be created within DoView and then put up on the web. A mock-up of what parts of a visual sector plan using the upcoming version of DoView could look like is available. This will be a no cost update to DoView which is planned for release in February 2009.
What can be claimed about whether a program works or not from a logic model? January 13, 2009
Posted by Paul Duignan in : Attribution, Communicating outcomes models, Impact evaluation, Outcomes models, Uncategorized , 1 comment so far
We sometimes hear things like: “a logic model was used to show that the program works”. I’m interested in tidying up such talk so that we are very clear about exactly what it is that is being claimed in regard to showing whether or not a program works. I’ve put up a new article in Knol which deals with the question of the types of claims we can make in regard to logic models (I call them outcomes models) and the types of arguments we can mount about whether or not programs work based on these claims. The article is here.The article is rather intense, but I think that it’s important that, as evaluators, we get on top of this sort of thing. In the article, I set out three claims that can be made in regard to logic models (or sub-parts of logic models). These are: (more…)
Drawing logic models (outcomes models) January 5, 2009
Posted by Paul Duignan in : Communicating outcomes models, DoView, Outcomes models, Outcomes theory, Uncategorized , 1 comment so far
I have just finished a substantial article on what I call outcomes models. They go by names such as: logic models, program logics, program theories, theories of change, ends-means diagram, strategy maps etc. and are used widely through evaluation.
In the article I attempt to go beyond the traditional received conventions about logic models which I think are usually unexamined in ‘how to’ books and much of the discussion of logic models.
The essence of my argument is that:
- There are six possible purposes for outcomes models – the first being to provide a picture of ‘what it is believed causes what to happen’ – what I call a ‘full’ outcomes model.
- Because people attempt to also use models to indicate measurability and demonstrability (attribution of changes in high-level steps and outcomes to a program) they end up truncating their models so much (limiting them to the measurable or the demonstrable) that they fail to achieve the first purpose of creating a picture of a ‘full’ model. Measurement and demonstrability (attribution) should be done after the model has been built.
- Most ‘technologies of representation’ – tables, single page diagrams etc – are hopeless at representing anything like a ‘full’ model of the causality behind a program. Hence they attract attacks as inadequate representations of programs.
- Models should be drawn in logic model software such as DoView so as to be able to represent full models (as large as they need to be, anything linked to anything else) and then measurement and demonstrability (attribution) can be mapped back onto them. This means they can achieve the purposes of the traditional conventional approaches to logic models without having their limitations.
If you have a moment, check out the article, any feedback appreciated. The article is here.
Paul Duignan PhD
Getting evidence implemented in practice – easier said than done – the Rich Dialog Process December 14, 2008
Posted by Paul Duignan in : Uncategorized , add a comment
Getting evidence implemented in practice is a lot more difficult than just waving around the odd evaluation report and pointing someone at an evidence-based web site like the Cochrane or Campbell collaboration sites. I was involved in a process earlier this year which was an attempt to work with a sector to get evidence-based practice implemented in practice.It used the Rich Dialog Process (RDP) – a simple deliberation and dialog process to get groups of stakeholders who may have different views on an issue to talk those differences through. The focus was on getting evidence-based practice really happening in the school drug and alcohol education area. An area in which there are a lot of different types of programs running. (more…)
Impact evaluation’s day in the sun (Part II) December 1, 2008
Posted by Paul Duignan in : Attribution, Impact evaluation, Measurement, Uncategorized , 1 comment so far
(See the previous posting Part I first). Progressing the debate around impact evaluation means unpacking some of the different issues which are involved in the debate. In doing this, it should not be assumed that those on various sides in the debate will always side in the same way with each of the sub-issues. In my doctorate (Duignan, 1997) I did this type of analysis for the positivist versus constructivist debate which was hot for a while within evaluation circles. Drawing on that analysis, we can now to look at the sub-issues which underly the impact evaluation debate because there are a number of similarities with the earlier debate. The sub-issues which can underly the impact evaluation debate and two contrasting positions that can be taken on them include the following: (more…)
Standardized visual evaluation plans – quick and effective October 9, 2008
Posted by Paul Duignan in : Communicating outcomes models, Doing evaluation more efficiently, DoView, Easy Outcomes, Evaluation planning, Outcomes models, Systematic Outcomes Analysis, Uncategorized, Using the approach , add a comment
I’ve not had much time to blog recently due to building a number of large outcomes models for public sector organizations; having input into the further development of DoView; and presenting at international evaluation conferences on Easy Outcomes, DoView and related evaluation and outcomes topics. A lot has been happening though, from version 1.14, DoView is now able to create web page versions of its visual outcomes models. I’ll do several postings showing how this new feature can be used. The first is that now, once an outcomes model has been built in DoView, the user can quickly create a web page version of the same model and then have it put up on an intranet or the internet. You can see (and use) a number of examples at OutcomesModels.org. The second great thing is that you can now produce visual evaluation plans will save you a great deal of time. I delivered a paper on this at the recent European Evaluation Society Biennial Conference in Lisbon. (more…)
Avoiding the question: Defining quality assurance versus program evaluation February 12, 2008
Posted by Paul Duignan in : Doing evaluation more efficiently, DoView, Easy Outcomes, Outcomes models, Outcomes systems architecture, Systematic Outcomes Analysis, Uncategorized, Using the approach , 4comments
Sometimes it’s more useful to avoid initially answering a question that’s posed in a particular way because there’s a better way of addressing the concern that lies behind the question. Such is the case if you’re ever asked to define the difference between quality assurance (or monitoring) and program evaluation.
Seeing the question as a theoretical one and attempting to find a definition which works has some similarities to the situation where you’re building a house and someone keeps wanting you to stop and define, from a theoretical point of view, the difference between the kitchen and the dining room. Now, some people do stuff in the dining room that others do in the kitchen, and some do stuff in the kitchen that others do in the dining room. Still other people don’t really have any theoretical problems because they have a kitchen/dining area where they do both kitchen and dining room stuff.
A more fruitful way of working with the question of the difference between quality assurance (or monitoring) and program evaluation is to attempt to identify all of the stuff (tasks) that you would do under each of these. Once you’ve done that, you can then decide whether or not you need to spend a lot of time defining the difference between the two if everybody concerned is clear about which of the underlying tasks are, and are not, being done by whom. (more…)