This summer I had the privilege of attending a Wilton Park conference on “New frontiers for evaluation in an era of market-driven development”. The conference theme was a response to the realization that the evaluation field is failing to meet the needs of the growing arena of social enterprise and impact investing. Coming from the development world, I have to confess that we’ve been having these conversations for a long time now, albeit leading to little changes in actual practice. There was a lot of discussion about reforming the evaluation sector, including a call to move towards approaches that blend “monitoring and evaluation through the use of continuous data capture and evaluative analysis.”
The case fo
r reform is also reflected in a recent report by Root Capital. Its call for ‘client-centric’ approach to evaluation is grounded in the practical truth that the value of evaluation is realized by people. It asks, who uses the evaluation? How is it used? In particular, they found that their clients – mainly smallholder agriculture cooperatives – are called upon to contribute a great deal to evaluation activities, but rarely experience direct benefits.
Root Capital, and a growing number of other development actors, are actively trying to change this dismal cost-to-benefit equation for evaluation.
An interesting menu of alternative principles and practices is emerging, including:
- In M&E activities, stop extracting information and engage in dialogue;
- Lighten the data collection burden on everyone while investing more in taking corrective actions based on what evidence you already have;
- Share M&E findings in ways that people like and use;
- Invest more in light-touch continuous data collection of a very few indicators and less in periodic deep-dive evaluative research (the medical analogy would be take your weight every day at home for 10 seconds and get a full half-day physical exam at the clinic once a year);
- Be smarter about using the data that does not require any effort to collect and do more triangulation of different data sets to discover the correlations that will increase the likelihood that you will act on the data.
Keystone’s Constituent Voice approach puts these principles to practice. To give you one example (disguised to avoid embarrassment), we worked with a social enterprise in East Africa to build a feedback loop with the communities where it works. Through community feedback, we found that perceptions about the fairness of the company correlated to the level of vandali
sm that the company experienced. After consulting with the community to design and take corrective actions, survey responses improved and, most importantly, vandalism decreased. Since the community benefitted directly from the social enterprise, in which it had an ownership stake, this was a win-win for the company and the community. This became po
ssible because the approach was both client-centric and provided real-time data that could be directly acted upon.
I am not saying that we should ditch impact evaluations, but rather that we should complement them with feedback practices of this sort that are quicker, shorter and, let me add, far more engaging and useful for those involved.
Natalia Kiryttopoulou is a Senior Consultant with Keystone Accountability.