Keystone believes in the power of data and benchmarking to improve how we do development. However, it is not always easy to come by. The International Aid Transparency Initiative is supposed to make information about aid easier to access, use and understand. It is the first (and only) standard for how organizations and governments are supposed to publish information about how they spend their money, who it goes to, and what it’s for.
However, agencies like USAID, DFID, and the World Bank put out lists of transactions years after they happen, and there is often no text description of the money’s intended purpose. It’s more like making sense of your monthly budget by reading your bank balance, but in this obscure XML format (see right).
IATI’s website says a mere 472 organizations (out of four million worldwide) have registered their data sets (about 0.01% global coverage). If you wanted to take this data and add up the money spent on HIV drugs, for example, or see all the relevant actors to fighting child abuse in East Africa – it would take you days of work. And after you did all that work, how would you know whether the answer was any good?
The team at AidSight devised a solution – in their spare time. They scoured the internet and found thousands of IATI documents then put this into a single database. They explored the data using algorithms, machine learning, and their brains – to generate features that would allow them to connect the data together. They devised heuristics fill in the missing pieces of data sets. For example, if two organizations mention working together, and one describes the work, and both seem to be funded by the same government agency, that one description of the work is a reasonable approximation to the other organizations work. They validated the world’s data. Then they applied a battery of tests to every organization’s data and gave them a letter score (A-F) for quality.
The search engine lets you name an organization, or place, or type of aid work, or some combination and it quickly builds a network map of everyone who matches. In 60 seconds, I was able to find all the HIV/AIDS related funding data for Africa and decide that this CDC data set is not worth downloading. It is missing everything except for project titles, dollar amounts, currency, and language. I’ll never know where it went (besides Africa) or when it was spent.
Keystone Accountability has been training organizations for a long time on how to carry out successful work by getting multiple perspectives, listening to the people served, and by looking at reference data when deciding whether to act.
Aidsight’s IATI report card gives us a badly needed benchmark score on all organizations. The team were also able to identify some 30,000 organizations (implicit in data from the 2000 that reported) – giving us a much fuller view of international development. The average score is a C-. We finally have definitive evidence that they have been making a rather pathetic effort to publish useful data about the way the world spends over US$200 billion each year to help the poorest of the poor.
The benchmarks also give useful, specific feedback to every organization on what it can do to improve.
Here the organization named Hivos has a pretty good score, but there are specific things it omits. In the network it doesn’t work with any of the other 94 named organizations that address the problem of child abuse in Uganda. Really? We ought to be more aware of what others are doing, because Keystone Accountability’s INGO partnership survey shows that when organizations do work together, they accomplish more of what the people we aim to serve ask of us.
AidSight is a wonderful example of how combining data sets can power up what they can be used for. Taken individually, IATI data sets reveal very little about the problems and solutions in the world. But collectively, they quickly expose the bad actors that do as little as possible to share vital information that could help us fight poverty, disease, and abuse. Collectively, the data set is powerful enough to create new data (data scientists call this ‘imputing’ data) from patterns in existing data.
It’s an inspiration for the work I’ve been doing at the Feedback Commons to make merging surveys from many NGOs into a common data set as easy as clicking a button.
Thanks to the AidSight team – Natarajan Krishnaswami, Nick Hamlin, Minhchau Dang, Glenn ‘Ted’ Dunmire.
To read a longer version of this blog see here.