Standardized visual evaluation plans - quick and effective October 9, 2008
Posted by Paul Duignan in : Using the approach, Communicating outcomes models, Doing evaluation more efficiently, Systematic Outcomes Analysis, Outcomes models, DoView, Easy Outcomes, Evaluation planning, Uncategorized , add a comment
I’ve not had much time to blog recently due to building a number of large outcomes models for public sector organizations; having input into the further development of DoView; and presenting at international evaluation conferences on Easy Outcomes, DoView and related evaluation and outcomes topics. A lot has been happening though, from version 1.14, DoView is now able to create web page versions of its visual outcomes models. I’ll do several postings showing how this new feature can be used. The first is that now, once an outcomes model has been built in DoView, the user can quickly create a web page version of the same model and then have it put up on an intranet or the internet. You can see (and use) a number of examples at OutcomesModels.org. The second great thing is that you can now produce visual evaluation plans will save you a great deal of time. I delivered a paper on this at the recent European Evaluation Society Biennial Conference in Lisbon. (more…)
A Systematic Outcomes Analysis framework for psychotherapy evaluation February 12, 2008
Posted by Paul Duignan in : Doing evaluation more efficiently, Outcomes systems architecture, Systematic Outcomes Analysis, Outcomes models, Easy Outcomes, Evaluation planning, DoView , 2comments
In my last blog posting (which you should read before this one) I talked about using Systematic Outcomes Analysis to define the basic tasks one needs to do in quality assurance, monitoring and evaluation and how this can avoid the need for a protracted theoretical discussion about the difference between quality assurance and program evaluation. I was using the example of an illustrative Systematic Outcomes Analysis framework I set up based on an outcomes logic model in regard to psychotherapy which I’ve posted on the Outcomes Models site. Here’s the PDF of the DoView file. Using the Systematic Outcomes Analysis approach, indicators and evaluation questions are mapped onto the outcomes logic model (indicators are marked with a yellow icon and evaluation questions with a green circular icon). This blog posting looks in more detail at ways stakeholders can use such a framework once it’s been developed. (more…)
What’s the Easy Outcomes site all about? September 20, 2007
Posted by Paul Duignan in : Doing evaluation more efficiently, Evaluation planning, Easy Outcomes, DoView , add a commentI’ve realized that I’ve not put up a post about the Easy Outcomes site even though it’s been up for a few weeks. Easy Outcomes takes the Systematic Outcomes Analysis approach and applies it using DoView outcomes software. We developed it in response to feedback that the Systematic Outcomes Analysis site is a bit intense for those who just want to work out how to develop a good evaluation plan, but don’t want to get buried in the detail of the approach too soon. (more…)
New Systematic Outcomes Analysis case study - evaluating an academic research rating scheme September 20, 2007
Posted by Paul Duignan in : Doing evaluation more efficiently, Systematic Outcomes Analysis, Evaluation planning, DoView , 1 comment so farI’ve recently put up the link to another case study on the Systematic Outcomes Analysis site. This is a report detailing an evaluation and monitoring plan for the Performance Based Research Fund (PBRF). The PBRF is a national academic research output assessment system and the monitoring and evaluation framework was produced for the government organization responsible for the tertiary sector - the Tertiary Education Commission.
The report is a good example of a comprehensive use of Systematic Outcomes Analysis (in the report the approach went by one of its earlier names - REMLogic). One of the interesting things about the PBRF is that it’s an evaluation system in its own right and developing an evaluation framework for an evaluation system presents an interesting technical challenge. I’ll put up a blog post about that challenge some time in the future. (more…)
Potential challenges to Systematic Outcomes Analysis July 23, 2007
Posted by Paul Duignan in : Systematic Outcomes Analysis, Evaluation planning , 1 comment so farSystematic Outcomes Analysis claims to provide a standardized approach to outcomes, monitoring and evaluation planning [see my previous post Can outcomes, monitoring and evaluation planning be standardized?]. What are the challenges people are likely to make to this claim and can they be answered? I’ve set out some of the major potential challenges below and provided some thoughts on each of them:
1. It is not possible to have a standardized approach to outcomes, monitoring and evaluation planning, every situation is unique.
This is the argument that evaluation is a ‘craft’ needing a skilled evaluation planner to tailor an evaluation to fit the unique situation. How can a ‘cook-book’ standardized approach do justice to the complexity of real world programs. This challenge should not be accepted until it’s been proved to be true. The best way to try to prove it is to try out a system like Systematic Outcomes Analysis to see if it does, in fact, fail when being used to plan particular types of evaluations. (more…)
Can outcomes, monitoring and evaluation planning be standardized? July 22, 2007
Posted by Paul Duignan in : Systematic Outcomes Analysis, Standards, Evaluation planning , 1 comment so farI was involved in an interesting discussion recently with a group of evalutors about whether outcomes, monitoring and evaluation planning can be standardized. In my experience, much evaluation planning starts from a blank slate with evaluators and project staff sitting around wondering about how they’re going to evaluate a specific program. Or, in other cases, for budgetary or other reasons, people who are not trained in evaluation have to work their way through basic texts about evaluation trying to work out how to do an evaluation. This all takes a great deal of time. Does it have to be like this? I’m not sure that it does.
Every time an organization sets up an accounting system, you don’t get the feeling that accountants have to build the entire accounting system from scratch. They simply put in place a number of basic building blocks of such systems and tailor them to the requirements of the particular organization. Why should monitoring and evaluation be any different? What I’ve been trying to do over a number of years in developing Systematic Outcomes Analysis is to develop such a standardized approach. (more…)