The post Beyond Accountability: Feedback as transformation appeared first on Alliance magazine.
Beyond Accountability: Feedback as Transformation is the title of Alliance magazine’s June 2015 special feature, and this was the topic of the Alliance breakfast club, kindly hosted by the Shell Foundation.
The panellists were David Bonbright, co-founder and CEO of Keystone Accountability and one of the guest editors for the special feature, and Nina Schuler, Social Development Adviser from the Finance Performance and Impact Department at the Department for International Development (DFID).
The emerging field of constituent feedback
The discussion focused around mapping the contours of this emerging field and the challenges of gathering the right evidence for funders to consider when making decisions.
David Bonbright’s Keystone Accountability pioneered the Constituent Voice (CV) methodology, drawing on tested techniques used by for-profit customer satisfaction organizations and adapting them to the context of development and the non-profit sector. He described CV as the ‘lovechild of participatory development and customer satisfaction’. An ‘uber-shift’ towards an evidence-based approach to managing social change is taking place, he said. Part of this transformation is actually gathering less data, in favour of ‘rigorously tracking a smaller number of key performance indicators (KPIs), and validating them with constituents’.
DFID has been an early adopter of beneficiary feedback and constituent voice. While acknowledging that international development has a long history of gathering feedback (participatory development), Schuler emphasized the importance of refining feedback methods, improving the quality of information collected, and finding ways to link feedback to decision making. She highlighted the ways in which DFID is using beneficiary feedback across its programme cycle – trying to strengthen opportunities to engage beneficiaries at all stages. To support testing and learning, DFID is supporting pilot activities in seven countries through GPAF (UKAID Direct) using different approaches to beneficiary feedback to improve programme performance.
DFID has recently introduced a Better Delivery Agenda, which narrows down a huge volume of rules of practice to just 37, placing greater emphasis on the judgement of staff. The main aim of this is to encourage better decisions. Beneficiary feedback/CV can be seen as one tool to help inform staff decision making.
Watch the discussion here:
[youtube https://www.youtube.com/watch?v=BqHVfeyUUvY&w=560&h=315]
Why aren’t more organizations collecting it?
Almost all foundations and non-profits agree that getting beneficiary feedback is a good idea, so why don’t more of them do it? Bonbright mentioned that Charity Navigator is currently the best source of empirical information for non-profit feedback practices. Having applied its feedback-related criteria to some 7,000 US charities, it found that only 7 per cent collect constituent feedback systematically, with even fewer systematically utilizing the data gathered.
Jane Steele of Paul Hamlyn Foundation gave an interesting perspective on this. There is an onus on foundations to gather feedback from their beneficiaries – their grantees – and thereby set an example to grantees, demonstrating that they are subject to the same type of accountability, she said. Foundations also have a responsibility to help grantees to get more from their feedback by providing help in improving the quality of evidence and analysis.
A key challenge comes from power relationships, with grantees and beneficiaries afraid to be honest, fearing the withdrawal of funding if they give negative feedback. Bonbright illustrated this problem from his experience with a re-offenders education programme in New York City. A service user expressed his fear of not receiving a job placement if he gave negative feedback about the programme, explaining that ‘if I saw three other guys give bad reviews and still get jobs, I might do it too’. ‘We must earn the candour’ of constituents, he said.
Myrna Atalla from venture philanthropy organization Alfanar raised the important problem of costs. As well as measuring social impact, organizations must also evaluate efficiency and financial performance, so it can be difficult to justify significant financial investment in constituent feedback. Survey fatigue is another well-documented problem, with beneficiaries tired of endless requests for information.
Net Promoter Score for non-profits?
Bonbright emphasized that the aim is to collect less data not more. He talked about the Net Promoter Score – the ultimate question for eliciting customer feedback in the for-profit sector: ‘Would you recommend this product or service to a friend?’ Can an equivalent question be found for the social sector?
The key, he said, is to formulate the right predictive questions in order to make interventions earlier and improve social impact. For example, the question ‘Do students treat the teacher in this classroom with respect?’ is a far better predictor of student outcomes than the question ‘Do you like your teacher?’ Organizations must ensure that feedback survey questions ‘correlate to their desired outcomes’ and will provide the insight to allow timely interventions.
Schuler highlighted the problem of making these predictive questions ones that you know you want the answer to. She also mentioned the risk of organizations developing ‘tunnel vision’ from a fixation on meeting KPIs and losing sight of larger issues and opportunities. She was keen to show the value of adaptive programming: ‘social change won’t happen in linear fashion’. She highlighted the importance of the challenges around beneficiary feedback and raised the importance of context and expectations. She invited the audience to consider a hypothetical comparison between feedback from beneficiaries of a healthcare centre in Tanzania and feedback from users of the NHS in the UK. Even low-quality healthcare may fare well if the beneficiaries face the prospect of having nothing instead, whereas people accustomed to good healthcare will only be satisfied with improvements in the service. Schuler rightly said that it is crucial to ‘let beneficiaries know what they are meant to be receiving so they can make a better judgement and improve service delivery’. Grantee feedback needs to be triangulated with wider metrics, she emphasized.
Bonbright echoed these sentiments. ‘Combining information with context makes impact far more powerful,’ he said. He also stressed the importance of people understanding why they are giving feedback.
What about technology, one audience member asked? Of course, this is opening up the possibilities for collecting constituent feedback, but it can bring its own problems. Schuler instanced gathering feedback from beneficiaries through SMS messaging. Although this can increase response rate, there is a risk that a disproportionate amount of evidence is contributed by young men, unintentionally marginalizing women and older people.
Looking ahead
To conclude, both were asked how they’d like to see constituent feedback in five years’ time. Schuler would hope to see it embedded in management processes – established as ‘a legitimate and rigorous process’, ‘part and parcel of management’. People from the business world can’t understand that we don’t already do this, she said.
Bonbright wants to see constituent feedback continue to evolve as a ‘systematic, light-touch, real-time process’. He is also keen for it to ‘get personal’. He gave the example of Cisco Systems, where all staff are required to end each touch point with a client with ‘Did you get what you wanted from this [fill in the blank]?’ This changes the dynamic of every interaction with a client, closing the feedback loop in real time.
Most of all, he hopes to see a well-developed benchmark feedback data set available for trustees and other people involved with non-profits to access easily in the public domain.