Hear our chief executive, David Bonbright, in conversation with Business of Giving host Denver Frederick in a new podcast from the radio interview that aired on Sunday evening, 24 January. Denver was very well prepared and we think the result is a breezy, informative “state of the field”. Please let us know what you think!
[soundcloud url=”https://api.soundcloud.com/tracks/243096044″ params=”color=ff5500&auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false” width=”100%” height=”166″ iframe=”true” /]
Update, January 28:
Denver Fredrick has shared the interview transcript with us, also available on his blog:
The following is a conversation between David Bonbright, founder and CEO of Keystone Accountability and Denver Frederick, host of “The Business of Giving” on AM 970 The Answer in New York City:
Denver Frederick: In the world of philanthropy, there’s been a lot of talk in recent years about impact and outcomes. Is the social program that is being implemented working, and where is the data that proves it? There is one data point that is often missing, however. It is the opinion and feedback from those receiving the service.
That is all changing. One person who has been leading that change is my next guest. He is David Bonbright, the founder and CEO of Keystone Accountability. David, welcome to The Business of Giving.
David Bonbright: Thank you for that kind introduction Denver. I’m delighted to be here.
Denver: Tell our listeners what is Constituent Voice, and how does it work.
David: Thank you. Constituent Voice is what happens when you listen systematically to those you serve, when you are careful in the way that you solicit their views and then respond to their concerns. As a result of that, they feel engaged with your organization, loyal to your organization, and ready to be involved to do the things that you want them to do.
Denver: What is Constituent Voice good at solving, and what things does it not address?
David: That’s an important point. It’s not a silver bullet that’s going to solve all the challenges of social change and deep societal problem solving in the world. Let me illustrate what I mean by that. If you are vaccinating an entire population, you have the ability to march into the community, line people up, and stick them in the arm, and produce the vaccination. Your relationship with them is not a key factor.
However, if you are trying to get people to change their intimate behaviors, how they make love, how they cook their food, and so on, your relationship with them is going to be critical to whether or not they are going to give a damn what you’re saying.
Denver: You made a point before, which I thought was very interesting. I think I’d always thought about Constituent Voice, about getting some data and insights as to whether a program is working.
But you’re seeming to suggest that the mere act of engaging in this process, and getting the front line workers involved, and getting the recipients involved, and having everybody empowered in the transformation is of equal if not greater importance. Would that be right?
David: Spot on. I always say that Constituent Voice is a triple threat. As you say, it’s directly contributing to the thing you want because of the way you engage with people and it deepens their commitment to you and your work.
Secondly, it gives you measures. It gives you data that’s helpful for measuring, and for understanding whether or not you’re making a difference, because people can shed some light on that. Not all light, and you need to be looking at other data to understand impact and outcomes, but it’s an important piece of that puzzle.
Thirdly, it enables you to improve. It gives you that insight piece. You discover something you didn’t know about what’s actually happening, how the service is working. “Gosh, we should start that course a little later,” or, “We need to actually have different office hours for this kind of service.” Those things that actually improve the day‑to‑day, as I call it, of what you’re doing, which has a knock on effect to give you better results.
Denver: The customer satisfaction movement in this country began probably in the early 1960s out of the consumer rights movement. The question I have for you is what took 50 years to get this to the non‑profit and social sector? What took so long?
David: I ask that question a lot. I came to this whole topic out of the social justice and human rights context. I’m a human rights lawyer by training, and I did a lot of grant making in the social justice space internationally. The way I think and the way I’m oriented toward things is very much around recognizing people’s rights, and their voice, and their leadership in whatever’s going to happen.
As the ’90s came along and we started to look at measurement much more seriously, and there was a lot of pressure around impact assessment as you noted in your introduction, I looked at the systems that were being put in place. They didn’t seem to be responsive to this more voice of people point of view.
I step back and I said, “Who’s solved this problem?” Sure enough, some people pointed me toward business, consumer‑facing business in particular, and the customer satisfaction industry.
I went back, and studied, and discovered that back in the early ’60s when it began, the situation there in business and consumer‑facing businesses is very similar to the situation we are in the non‑profit sector today, which is that the people who ran the businesses just didn’t think that they had to know this. Their view was, “Hey, we designed this stuff. It’s good. We know what we’re doing. They buy it and use it.”
Denver: The end.
David: The end. Then this lovely consumer rights movement came along and changed everything. It’s interesting, one of the reasons, the most obvious conclusion from the analogy to customer satisfaction, is that there’s nothing like a rights movements for the people who receive services from governments and non‑profits in the same way.
Denver: Interesting.
David: There never will be, I don’t think. That’s not the nature of the way this plays out. You don’t have a right to philanthropy in the same way that people have a right to be informed about consumer products, and so on. Yet, at a practical level, if organizations did this, it would work better. They would achieve their variance better.
I think the argument we have to make and demonstrate through our work is that this is the smart thing to do. You will get better outcomes. Same to a non‑profit’s it’s the right thing to do isn’t convincing them. It hasn’t convinced them. We’ve really got to make the case on the smart thing by showing them that it’s easy, cost‑effective, and powerful in terms of its results.
Denver: Are they afraid of how funders might respond? There’s a lot of charities out there. I know that you intend this to be a learning situation. But sometimes these things become grades. They become failing grades.
You would hope that the grant makers, and the foundations, and the funders would not look at it that way. But I would have to think that the fear of being penalized and losing funding has to be at the core of a lot of these, at their hesitation to move forward.
David: Absolutely. I hear this again and again when potential clients or clients are being frank with me. “What happens if they say they don’t like us, and what happens if our donors hear about that? It’s all downside. You talked, David, about all these wonderful things that are going to happen in terms of the results, but that’s down the road. If I lose my funding, no more.”
It’s a real fear. The truth is that there are very few donors that will behave that way. There’s lots of natural experiments that are going on out there that show the opposite is the case, actually. Donors are not stupid. They know that charities are putting a fiction out there about their work.
In some ways, it’s a huge relief when an organization stands up and says, “Hey, it’s not perfect. I’m going to show you the warts and all, and I’m going to tell you how we’re going to fix it. You watch and see what people say next month because we’re changing things and our scores are going to get better.”
I always say to our clients, “Watch the change over time. Don’t worry about the scores you get today. In fact, the lower scores, the better at the beginning because you got nowhere to go but up.”
Denver: That’s exactly right. Why don’t you give us a couple of examples? You’re talking about clients. You’ve been at this for a decade plus. Give us a few examples where people have reached out to the beneficiaries of the recipients of a service, learnt something, and have improved the delivery of their services.
David: I can give you a great example here from New York City. Right here in New York City, the Center for Employment Opportunities, 25‑year‑old, very effective organization that helps men transition out of prison and into stable jobs. They wanted to deepen the way they listen to those folks.
For the past year, they’ve been piloting at it very purposefully, very carefully in one city. It’s been working well. They’ve rolled it out across New York City. They’re now expanding it to other cities where they’re working.
Just to give you an example. In the first week of their program, they have an intensive course they call “Life Skills.” They introduced Constituent Voice surveys into the course and quickly learn that people were struggling with the starting time of the course.
It came up really quickly, one of your discovery points. There’s huge penalties for the participants in the program to miss or to be late because they end up not graduating and falling out of the program.
What did they do? They changed the time. When they changed the time, they shifted an hour. Very inconvenient for the staff, by the way, but very convenient for their participants. But they were so committed to being responsive and listening that they did it. Of course attendance improved and the results are better.
Denver: Let me ask you about candor. There is this inherent power struggle. I think we all act this way, whether you’re in their program or not. If you “Kiss up to management,” there might be some benefits. But if I rip them, there may be some unpleasant consequences. This is a brand new initiative you’re laying on me right now. Can I really trust you and do you have a different end motive in mind?
David: That’s beautiful. You’re channeling many, many people who receive surveys in the context of our work. You’re absolutely channeling them right now, Denver. This is the key. If people understand that it’s in their interest to be candid with you, then they will give you a candid answer.
When you go out and send out a survey and run a poll, or do however it is you’re going to do it, do an interview, if you haven’t done the homework with them to make sure they understand that, then you can’t expect candor. I always say you have to earn people’s candor in this context.
But, it doesn’t take a lot to do that. If you’re careful, and you explain it, and you show the setting, and you show that when people give feedback, good feedback change, things are changed, and improved, and so on, it doesn’t take long. In my experience, it takes three to six months of going at this to earn people’s trust so that they see that it works.
Denver: This is something which I think a lot of our listeners can relate to and that would be, could you just give us five minutes to fill out this survey. Is there a danger after the novelty of this approach wears of, of survey fatigue, people just get tired of this?
David: Part of the key here is that you can’t ask a lot of questions. The historical pattern which has been dominated by social science is to use the survey as a research exercise, we don’t do that. We ask one, maybe two questions at the most at any given survey instance so…
Denver: Give us an example of a question that you would ask?
David: One of my favorite questions is what I call the voice question which is, “To what extent is it worth your while to engage with this organization to make it better for you and your family.” If people are telling you, “Yeah, I’m in there,” it’s a 10, then the organization knows it’s being responsive to those people. Those people are giving their heart, they’re getting engaged, but if they’re saying six, you got some work to do.
Denver: Let’s say I take a survey and I’m an organization that’s helping the homeless and I get a six, how do I know I’m doing well or not so well? Is there anybody I can compare it to who’s in the same arena? Can I benchmark those results against somebody else’s?
David: First of all, the assumption behind that question is very important one, which is that benchmarking is really critical to understanding what qualitative data really means. If you don’t have the comparison you don’t know and we don’t have that now. You can compare it against yourself overtime. If you keep asking that question you can watch how scores go up…
Denver: Like the Fitbit, the way it works.
David: Exactly, and that’s great. Every organization can do that on its own and it’s sufficient, by the way, to really validate this. However, we’re working on creating that framework for benchmark, so we saw this as an important problem and an easily solvable one.
We’ve created a website called the Feedback Commons which is now operating, feedbackcommons.org, where any organization can go and sign up and share its feedback data. Basically, the way it works is that there’s a menu of questions that organizations have found useful and data associated with those questions that other organizations have asked.
If you choose to use that question and go away and get your survey and then come back and put your data in, you can benchmark yourself against the others. If you’re asking a different question from anybody else, you can’t get benchmarks.
The way we’ve solved that is we’re encouraging people to come before they start, choose questions that are already popular, that are already been tested and proven, and then use those so that they can get their benchmarks.
Denver: That is very smart. David, have there been instances when they’ve gotten the recipients’ feedback and where the institution is listening and they hear what somebody wants that’s not there, or maybe something that they’re providing that is just a waste of money, frankly.
At end of the day they’ve actually responded to this feedback and have changed the deployment of the way they allocate and deploy their resources?
David: Absolutely. It takes a while to develop conviction about the data you’re hearing in order to change your program or your strategy, because after all you’ve invested a great deal on this. I can give you lots of examples now of organizations that are starting to do that after about nine months to a year, typically.
Maybe the best example we have from our experience comes from the humanitarian context, because when you have a humanitarian crisis it’s a hothouse situation, things happen very fast. We were invited by the British government to support the efforts that they were doing to respond to the Ebola crisis in Sierra Leone.
We went in and we started surveying frontline health workers and then doing a text‑based pulling of the wider citizenry. At first, the scientists and the medical people who were running the Ebola in Sierra Leone didn’t pay any attention to what we were doing, because they didn’t trust the feedback from the citizens or the workers.
When we showed them what people were saying both the frontline health workers and the citizens, they were saying the same thing and it absolutely mirrored exactly the incidence of new disease. They said, “Whoa, people know what’s going on out there.” They knocked on our door and they said, “Would you mind asking some questions for us that we have about how to improve the response.”
We said, “Sure.” We added those questions to the thing, they happen to be about the quarantine procedure. From the answers that we got from that, they changed the quarantine procedure within six weeks. They changed the quarantine procedure, quarantine processes improved and we beat the virus.
Denver: What a great story that is!
David: I’m not saying that’s the only thing. By the way, I don’t want anyone to misinterpret it, but it was a good example of how a better use of feedback from people enabled a rapid correction in a system and produced a better result.
Denver: That really crystallizes it for me. I know that you’re very excited, you’re on the cusp of, I think, something which is really beginning to gain momentum. Give us an idea of some of the forces that are at work right now which are going to propel this and accelerate this, and I would imagine technology would be included in there somewhere.
David: Absolutely. There’s a number of things that are going on that are tremendously exciting. You alluded to donors as a worry in an earlier part of this conversation, but one of the really exciting things that are happening is that organized philanthropy in United States and indeed the UK, where I live, is starting to recognize that there is value in this.
They’re starting to ask their grantees about it and they’re starting to support the grantees to do it. There’s something that was formed a couple of years ago, a consortium of funders led by Fay Twersky at the Hewlett Foundation and Hilary Pennington at the Ford Foundation called, “The Fund for Shared Insight.”
I think they now got eight or nine foundations that are members. They’re really starting to change the way the larger foundation sector in the US is thinking about this. This is huge. It sends a big signal to everybody, very important.
Second thing that’s happening is three or four years ago a number of organizations came together and set up kind of an incubator and a place where we can really improve the practice of doing this and make the case for it called “Feedback Labs.” Terrific organization, we now have some staff. It’s based in Washington DC.
We’ve set up a chapter in London and I like to think of it as the first call for people who want to find out more about how you do feedback and how to make the case to your funders and all of that, feedbacklabs.org for those of you who might want to visit the website.
There is, if you will, the practitioner community is forming and building its craft. Technology as you say is key to this. We’re using cell phone surveys and diverse settings now and it’s very inexpensive, it brings the cost of getting feedback from people down. Actually, one of the nice things about it is you can’t ask long surveys on a text message. It’s forcing people to do what they should be doing anyway…
Denver: There you go.
David: …which is just going for that one clean signal and then managing to that. The last piece is really important, this methodology is falling into place because the how’s are important and it’s not just about the data.
The data is just the first step. It’s about collecting the data and then going back to people, talking with them about what you’re hearing and together discovering ways to improve. Constituent Voice and feedback when it’s done best is not a data exercise, it’s an engagement exercise.
Denver: It’s a circular. It goes back and forth and it never really ends.
David: Exactly. Now we have the methodology very well laid out, anybody can get it and read about it, and people are starting to recognize that. By the way it fits with a larger thing that’s happening in performance management generally in the nonprofit sector, which is this whole lean movement of moving away from top‑down blueprints where you match according to a plan for a year and then you review it or even longer, God forbid.
Denver: Three and five year plan…
[crosstalk]
David: Yeah, exactly. The point is that now people are starting to really understand that it’s about constant iteration of small steps, getting feedback of different kinds not just hearing from people but looking at what’s happening and then making changes all the time. Tiny steps all the time, iterating, iterating, iterating.
Denver: I think for the first time in history the adaptive ability and mindset is the one that’s going to win the day as opposed to the one who can rigidly march to a particular plan. With all the excitement around Constituent Voice are there any things you are wary of, are there any unintended consequences that give you cause for concern?
David: There is a long list of worries, I haven’t seen any unintended consequences yet, I’m sure they will come and I’m sure I’m blind to them.
Denver: That’s the unintended part of the question.
David: You don’t know what you don’t know. What I think, I always worry about quality that we are out there waving this flag and we have the Feedback Commons, anybody can come and use the questions but you don’t really know how faithful they are executing. We need to figure out our quality assurance model for this emerging field as it comes…
Denver: That’s a good point. Because if you ever get a bad name from people who try to take real, real shortcuts and then say it didn’t work.
David: Exactly. I think we’re pretty early days on getting the word out. Getting the support for the resources behind it, we need to build some infrastructure to support this thing. We’ve done the Feedback Commons on a bootstrap to get to this point.
The Feedback Labs need support. Nobody is really stepping up and funding this infrastructure in a serious way beside the “Funds For Shared Insight” group. It’s nerve racking for us to have such a narrow financial base for this.
We carved this out of our consulting practice. We’ve tried to build the field on the side of our consulting practice. I think one of my biggest worries is getting a significant three to five‑year influx of money and to the kind of make that infrastructure available.
Denver: We’ve talked on that in previous shows, with all the excitement around initiatives like this. It just seems like nobody ever wants to fund the backbone organization to make it work. They want the results, they want the program but they don’t want to understand the critical role that the administrative function is.
You have written about predictive questions that an organization can manage up to. Tell us what a predictive question is and give us an example if you could.
David: The best and most empirically validated example comes from customer satisfaction. It’s the recommended question. How likely would you be to recommend this company to a friend or relative? It turns out that answers to these questions directly correlate to what buyers constantly do.
They reach out to their pockets and buy in line with what they see in their question. So that question predicts what consumers are going to do. That’s a home run because now you know that if people are saying this they are going to do that and I can work now work towards that and manage with that data.
In the case of Ebola, we found some of those questions. We got in the answer on the quarantine question. We got answer to questions what people wanted to see before they were comfortable coming into quarantine, that when the Government acted on it people came into quarantine. So it was correct. It predicted, it was accurate to what they were saying.
That’s what I mean by predictive questions. Something that I answer today that your organization can rely on to know if it acts on it, it will get these results tomorrow.
Denver: I think there was another great example that I saw from the Gates foundation and Education.
David: Oh yeah.
Denver: Tell us about that.
David: Wonderful example. The Gates Foundation was trying to understand how to improve teacher’s effectiveness and it was studying it and spent a lot of money at it doing various kinds of studies. It had this smart idea that it was going to be mastering teachers in the classroom which was going to be the best tool for really helping understand whether teacher was effective or not.
When all the work was said and done, the thing that turned out to be the most predictive of teacher performance was student feedback and in particular asking the right questions.
You don’t ask students “Do you like your teacher?” It’s the wrong question. The question that turned out to correlate directly to students’ outcomes on standardized tests which is not the only measure of learning, let me be clear. But it’s a very clear and objective measure of learning, was the answer to the question, “To what extent do you and your classmates respect this teacher in the classroom?” It’s not the exact wording but it was to that effect.
Essentially, the answer to the question described a setting in the classroom. That really correlates to good learning. If this teacher is able to create that setting, then you are going to have good learning outcomes and that was the kind of genius of the question.
Denver: Tell us about Keystone Accountability. How you got started with this gig and who you work with and a little bit about your company.
David: Thank you. Keystone is a nonprofit with three legal entities registered, one in the US, one in the UK and one in South Africa. Where I have done a lot work historically, we came into this evaluation, was 10 years ago to say, “Guys we need a way to measure that’s really more responsive to people on the ground and relevant to people on the ground.”
We kind of fluster away for a few years and eventually came up with this methodology Constituent Voice. Essentially what we do now is we very simply. We just promote Constituent Voice through consulting services. We help organizations put those systems in place.
We work on field building activates by creating things like the Feedback Commons. We try to get the word out. We write about our work that’s essentially…it’s pretty simple. I think we are the first specialists consulting firm that’s focuses exclusively on feedback in the social space. If you will we are kind of the J.D. Powers of the nonprofit world.
Denver: David Bonbright the CEO of the Keystone Accountability. Thanks so much for being here this evening. This is a topic that I think very few people know much about and you were more than enlightening, it was a real pleasure having you on this show.
David: Thanks Denver, and I hope it’s something people hear more about going forward.
Denver: I am sure they will. We’ll be back with more of The Business of Giving right after this.
Announcer: The Business of Giving can be heard every Sunday evening between 6:00 PM and 7:00 PM Eastern on AM 970 The Answer in New York and on iHeartRadio. You can follow us @BizofGiv on Twitter and at Facebook.com/BusinessOfGiving.