The good old ‘one pager’ contraint again July 9, 2009
Posted by Paul Duignan in : Communicating outcomes models, Doing evaluation more efficiently, DoView, Easy Outcomes, Outcomes models, Outcomes systems architecture, Using the approach , add a commentI’ve been up against the good old ‘one pager’ constraint again in a couple of contexts recently. This is where there is the demand that the outcomes for an organization or program be ‘put on one page’. It comes in various forms and is often a demand from senior management or a perceived demand from them to ‘keep things simple’. Now, there’s nothing wrong with the idea of summarizing things and paper overviews play a role in that. But such things should be seen as one of a range of different types of summaries and products which are produced by an underlying outcomes system, not the beginning and end of an outcomes effort. (more…)
Separating analysis from writing in philanthropic grant applications July 8, 2009
Posted by Paul Duignan in : Communicating outcomes models, Easy Outcomes, Grant writing, Philanthropy, Strategic planning , add a commentThe other night I was doing some pro bono work with a small non-profit organization which is attempting to restore a significant landmark building and promote its use for educational and community purposes. I was building a DoView results map (outcomes model) for them in real-time in the course of one of their monthly meetings. They’re using the visual model as their strategic planning approach (instead of the traditionally long narrative text-based plans many people in such small organizations spend many hours sweating over).
Building a DoView model is a very quick way for a community organization to build a simple but effective strategic plan. Once built they can then show others the model and quickly communicate that they’ve thought through what they’re planning to do. (more…)
Impact evaluation on full program roll-out versus just on piloting – two paradigms June 10, 2009
Posted by Paul Duignan in : Attribution, Doing evaluation more efficiently, Easy Outcomes, Impact evaluation, Outcomes systems architecture, Using the approach , add a commentI’ve just posted an article on the two paradigms in regard to impact/outcome evaluation and full program roll-out. What this is about is making a distinction between designing an evaluation which can provide impact/outcome evaluation information about full program roll-out versus a paradigm where you do impact/outcome evaluation just on piloting and then in regard to full program roll-out you just make sure that best practice is implemented. I once was involved in the evaluation of an overall program which had over 900 component programs. The way that we went about evaluating it was, in my view, wrong. (more…)
The error of limiting focus to only the attributable June 8, 2009
Posted by Paul Duignan in : Accountability, Attribution, Easy Outcomes, Indicators, Measurement, Outcomes systems architecture, Reporting systems, Using the approach , add a commentI am continuing to develop a set of articles which outline various problems which are often built into the outcomes systems which I see. The one I have just put up is on the Error of Limiting Focus to Only the Attributable. This is where the whole emphasis of a performance management or other outcomes system is just on holding a provider to account for a list of demonstrably attributable indicators (often called outputs, deliverables, or key performance indicators). This often creates problems. (more…)
Over-simplifications in outcomes, monitoring and evaluation June 3, 2009
Posted by Paul Duignan in : Accountability, Attribution, Doing evaluation more efficiently, Easy Outcomes, Evaluation planning, Outcomes systems architecture, Outcomes theory, Reporting systems, Using the approach , add a commentAn evaluation colleague Patrica Rogers commented on an earlier blog posting of mine in which I was claiming that what I am trying to do it to make outcomes, monitoring and evaluation work ‘easier’. She challenged me on that idea and pointed out that often what we are having to deal with is over-simplification in the way people are working with outcomes, monitoring and evaluation. Her comment inspired me to work up an article on over-simplification in outcomes and evaluation and after getting underway with it I realized all of the different ways in which people approach outcomes, monitoring and evaluation with over-simplified approaches and the problems which these cause. (more…)
Visual model of what I’m trying to do with my outcomes work April 28, 2009
Posted by Paul Duignan in : Blog info, Communicating outcomes models, DoView, Easy Outcomes, Outcomes models, Outcomes theory, Using the approach , 2commentsThought that I would apply a taste of my own medicine to my own work, so I drew a visual outcomes model of what it is that I’m trying to do with my work in the outcomes area. It is here. At the top is my high level outcome ‘Make working with outcomes, monitoring, evaluation etc. easier’ and below that is all of the lower-level steps I am using to get to this high-level outcome. I have included hyperlinks out to the various web sites where I am attempting to do the things listed in the lower-level steps.
Paul Duignan, PhD
Outcomes and Evaluation Blog (OutcomesBlog.org)
Don’t assume that impact evaluation should always be done April 26, 2009
Posted by Paul Duignan in : Attribution, Doing evaluation more efficiently, Easy Outcomes, Evaluation planning, Impact evaluation, Outcomes systems architecture , add a commentImpact evaluation – evaluation which looks at whether changes in high-level outcomes can be attributed to a particular program, organization or other intervention – is a particularly useful type of evaluation when done properly. It clearly tells us what works, and what doesn’t work, and this information can be used in decision-making about which programs should, and should not, be funded in the future. However, particularly at the present time, with all of the enthusiasm for evidence-based practice, many people mistakenly assume that impact evaluation should always be attempted in regard to any program, organization or other intervention. Assuming this is a serious mistake. I’ve just put up an article in the Outcomes Theory Knowledge Base which sets out in detail why it is and the way we should approach assessing when impact evaluation should be done. (more…)
Social Innovation, evaluation and outcomes April 6, 2009
Posted by Paul Duignan in : DoView, Easy Outcomes, Impact evaluation, Outcomes models, Outcomes theory, Outcomes theory & politics, Research influening policy, Using the approach , add a commentI attended a launch of the New Zealand national Center for Social Innovation last night. Geoff Mulgan from the Young Foundation (a similar center in the U.K.) talked about social innovation. The social innovation movement is about getting stakeholders and sectors together to do things differently to achieve better social outcomes. Already a dynamic movement, it has recently received a shot in the arm from the global economic melt-down – traditional ways of doing things are increasingly being questioned and people are looking for new solutions. A number of points made by Geoff and in the subsequent discussion are particularly relevant to outcomes and evaluation were: (more…)
Why just about every indicator system in the world needs to be fixed! April 5, 2009
Posted by Paul Duignan in : Accountability, Communicating outcomes models, Doing evaluation more efficiently, Easy Outcomes, Evaluation planning, Indicators, Measurement, Outcomes models, Outcomes systems architecture, Reporting systems, Standards, Using the approach , 1 comment so farI’ve just posted a new article in the Outcomes Theory Knowledge Base on why it is essential to map indicators onto an underlying visual outcomes model. I blogged a little while ago about why we should be wary of too-tidy indicator sets and in the article I explain why. The vast majority of indicator systems in the world suffer from the problem set out in the article – they are just a straight list of indicators set out in tabular format. They give the user no idea as to whether a number of important steps and outcomes are not being measured. Those using such systems remain blissfully unaware of this. In my view, all these straight indicator sets need to be fixed. It’s not particularly difficult, it just requires some work. How to draw the underlying outcomes models is set out in the outcomes model standards and how to then use such models for indicator mapping and many other things is described in detail in the applied version of outcomes theory – Easy Outcomes. (more…)
Using visual outcomes models to describe and communicate best practice April 3, 2009
Posted by Paul Duignan in : Communicating outcomes models, Doing evaluation more efficiently, DoView, Easy Outcomes, Evaluation planning, Outcomes models, Outcomes systems architecture, Outcomes theory, Reporting systems, Research influening policy, Strategic planning, Using the approach , 4commentsYesterday I blogged about what is meant by the term ‘best practice’. As I said then, I think that there is some conceptual confusion out there about it, and I am not sure that I have yet tidied it up my own thinking about it in a satisfactory way. However, the great thing is that regardless of how we define it, the idea of identifying the types of things that people currently call best practice and communicating these between programs is a great idea. The most difficult thing in many cases is to get best practice actually applied to on the ground after we have identified it. I have put up an Outcomes Theory Knowledge Base article (link to the article is at the bottom of this posting) on the issue suggesting that an efficient way of describing and communicating best practice may be to use visual outcomes models (a type of logic model). (more…)