Using large visualized outcomes models in policy development December 18, 2008
Posted by Paul Duignan in : Outcomes systems architecture, Strategic planning, Communicating outcomes models, Outcomes models, DoView , trackback
I recently presented a paper to the European Evaluation Society Conference in Lisbon on the use of large visualized outcomes models in policy development. The conference theme was on ways that evaluation could contribute to policy development. I see visualized outcomes models as one way that evaluators can contribute. Check out the full paper here. Evaluators may see themselves as ideally suited to contribute to high level policy making, however, regardless of evaluators’ views, the powerful players already involved in policy development (stakeholders, politicians, senior public servants, policy analysts, economists and others) need to be convinced that the evaluation discipline can add real value to the policy making process. What actual value can evaluation add?
Evaluation can already claim expertise in evaluating specific programs and evaluators are also sometimes involved in creating summaries of evaluation evidence through meta-analyses of sets of individual evaluations. However, this type of work tends to be viewed as inputs into the high-level decision-making around policy issues. Evaluation’s case that it also has something important to add at the high level would be advanced if it could bring to the table a powerful tool that makes policy-making easier.Well-visualized comprehensive ‘outcomes models’ are one specific tool with the potential to play a powerful structuring role at the centre of policy-making. Such models are versions of the logic models, programs logics, intervention logics, programs theories and theories of change that many evaluators use in their daily work. However, for use in higher-level policy-making, outcomes models need to be built according to a specific set of guidelines.
These guidelines free models from the constraints evaluators have often worked within when building such models (e.g. only including measurable and attributable outcomes, limiting the number of levels within the model, limiting the model to a single printed page). For such models to serve their purpose in high-level policy-making they need to be visualized in outcomes modeling software which allows them to be used in real-time deliberations by policy-makers. Creating and using such models with policy-making groups is a specialist area of technical expertise. Evaluators, given their experience with logic modeling, together with their other expertise, are well positioned to take a lead role in building and supervising the use of such models within the policy-making environment. To take advantage of this they’ll need to become adept at constructing the type of outcomes models that are most useful in these settings.
I have been developing a number of such high-level models with government ministries using DoView outcomes and evaluation software, which I was involved in developing. However, most of the models have not yet been released for me to make them publicly available on the web at www.OutcomesModels.org. I’m working on permission to get some of them released at the moment. In the meantime you can get a flavor of the type of model by looking at OutcomesModels.org, even though the models there tend to be just at the program level. One model up there at the moment, a model for a national school self/external evaluation system is at a somewhat higher level. The type of models I am talking about are at a high level for a sector or policy area. Once constructed, they allow policy makers to do a number of things such as identify which projects map onto which outcomes (to discuss gaps and overlaps in policy deliver) (see a simple example here). In addition, policy makers can use them to identify indicators for monitoring, evaluation questions etc.
Paul Duignan
Comments»
[…] I’ve talked about the use of large scale outcomes models for various purposes such as overall policy development, evidence-based practice and monitoring and evaluation. The next version of DoView outcomes and […]