In monitoring and evaluation there are, of course, situations where there is little time or resources to do such analysis. One of the main points I try to push is that there should be transparency about the status of the information that people are using for decision-making. So where it is not particularly robust they should be informed of this. This is why I always think that sets of performance indicators should be mapped back onto an underlying visual model. (As argued in this article http://knol.google.com/k/paul-duignan-phd/indicators-why-they-should-be-mapped/2m7zd68aaz774/72 and this YoutTube video http://www.outcomescentral.org/performanceindicators5.html).

We still need to catch up for that talk some time! Sorry I have been busy with many things including writing a first draft of a book about my work.

As you say, I think that we need some way of working across the different perspectives, otherwise people are just going to keep talking past each other.

There is powerful growth of the Randomista approach which is coming in at least part from people who would not call themselves evaluators, economists in particular.

I think that as the evaluation profession went through a long period of thinking about different impact evaluation designs and methodologies, that that process needs to happen now on a much larger scale.

In particular more sophisticated thinking, for instance, that in some situations you are better off doing smart formative evaluation (for those not familiar with the term – evaluation to ensure that a program is well formed – sometimes also called implementation evaluation) than attempting to do an impact evaluation where it is not appropriate, feasible or affordable.

It is great that so many people are getting into evaluation, evaluators just need to look to engage all of the different people now thinking intensely about evaluation.

I enjoyed your blog post.

It’s amazing to me how the word “rigor” is still equated with randomized control trials and quasi-experimental designs, even after Lincoln and Guba so diligently described how to ensure reliability and validity regardless of the evaluation approach or method. I see it most decisively as I assist organizations with writing the evaluation plans for grant applications. It most certainly still rings true with who you call the Randomistas.

I believe that using outcomes planning approaches as you describe would help clear the air and assist both the Randomistas and Non-Randomistas in coming to terms with doing what is valuable for the project instead of having tunnel vision.