Changing how peacebuilding organizations measure success could save aid projects that are stuck trying to meet rigid, dated, and increasingly arbitrary goals in conflict zones.
- By Andrew BlumAndrew Blum is the vice president for program management and evaluation at the United States Institute of Peace. The views expressed here are his own.
Ukraine, Iraq, Afghanistan, Libya, Syria, Nigeria, the Central African Republic, South Sudan — a depressing list, which seems to grow each day. It can be read as shorthand for human suffering and international tragedy. For the multitude of conflict prevention and humanitarian organizations that are committed to preventing the calamities that have struck these countries, the list is a sobering reminder of how much work needs to be done.
But it is also a reminder that this work demands continuous evaluation. The governments, foundations, and individuals that fund international aid work demand assurance that their money is being spent wisely; any hope for success demands being able to deploy smart, well-run programs. And doing that means being able to hold agencies and organizations accountable.
There’s a paradox, however. The challenges inherent to working in conflict zones means that strengthening the current approach to accountability — judging success against promises made years ahead of time — will create less effective programs, not better ones. The paradox is caused by a stable, slow-moving system, like the U.S. government, colliding with the unstable, rapidly changing conditions in conflict zones.
Virtually all of the work funded by the U.S. government or other international funders in areas of conflict follows a certain model: An agency identifies a problem and designs programming to address it, then hires a for-profit contractor or non-profit NGO to make it happen. Along with other independent agencies, such as inspectors general, the funding agency then tries to hold the hired organization accountable for achieving the objectives described in the original agreement. In the end, the results are supposed to check off the boxes from the initial plan, regardless of what might have happened in the interim.
Needless to say, this doesn’t always work out.
The rigid approach to implementing projects, not so surprisingly, has contributed to some well-documented failures. The final report of the Special Inspector General for Iraq (SIGIR) documented hundreds of abandoned projects that Iraqis are not using, including a $40 million prison that "will never hold a single Iraqi prisoner." In her new book Peaceland, Severine Autesserre describes another illuminating failure: The United Nations sought to increase security in the eastern part of the Democratic Republic of Congo (DRC) by deploying additional police officers to the region. However, the police received no training, and the government of the DRC saw them as U.N. police, so refused to pay them. As Autesserre documents, the untrained, unpaid police became just another group preying on the local population, but at the end of the project, because the stated goal was to establish the force in the first place, the U.N. still claimed it as a success.
Lurking behind these failures is funding agencies’ normal accountability mechanisms, which simply don’t work when applied to conflict-affected areas because they make it very difficult to adapt programming to changing circumstances. As Andrew Natsios, the former administrator of the U.S. Agency for International Development (USAID), has argued, the accountability system in Washington, which he calls the counter-bureaucracy, "misapplies a domestic management lens to aid programs." Holding an organization accountable for building a highway bridge in Minnesota, for instance, requires a different approach than holding an organization accountable for building a Ministry of Justice in Libya.
Still, the need for accountability won’t change — working in challenging places cannot mean that organizations get a pass for not doing the job. But the way that success is measured and applied needs to evolve. So how should the U.S. government hold peacebuilders, contractors and NGOs, accountable in a way that actually makes sense?
To oversimplify a bit, the current approach is built around two basic questions that funding agencies ask: Did you do what you promised you would, and did it achieve the results you said it would? As a result, in the field, the plan drafted by the government — called a "scope-of-work" — guides every decision that contractors and NGOs make. Given how long planning and procurement can take for projects like the prison in Iraq, this means that contractors are often trying to fulfill promises made up to three years before projects even start.
To fix this process — that is, make it more responsive and agile — those questions should be focused on how the project achieved results in an unstable environment. The first question to program directors should be: What results did you achieve? Then, how did the project adapt to be most effective, given the changing context in which you are operating? Finally, what evidence do you have that supports your decisions regarding adapting your project?
Given the rigid, detailed planning processes that are the norm in U.S. government-funded projects, that may sound like a radical departure, but many of the building blocks are already in place. There are already examples of NGOs, international organizations, and U.S. government agencies that have adopted the flexible, adaptive programming that is required to be effective in conflict zones. The NGO Partners for Democratic Change, for instance, often uses a model that is based on establishing a permanent partner "center" within a conflict zone, such as Yemen or Colombia, as opposed to implementing a project with a set of pre-determined activities. A recent evaluation of this model claimed that, "as process experts, the Centers were able to adapt their programs and services to meet new needs and take advantage of new opportunities." Religious networks, in part because they have independent sources of funding, have been employing a similar, so-called "window of opportunity" model for decades. This model relies on continuous presence in an area and the ability to respond flexibly to opportunities as they arise.
The U.S. Institute of Peace, where I work, uses a similar model, implementing something called Justice and Security Dialogues in six different conflict zones. The project, which consists of ongoing discussions between police and community organizations, is organized as a platform, as opposed to a strict set of activities. The nature of the dialogue and the problems addressed are regularly adapted based on the feedback received from the community and security services. This adaptation makes the dialogues better, and it makes it more likely they will succeed.
The truth of the matter is that it is easier than ever to get reliable feedback on how projects are working. New tools, including SMS-based cell phone surveys, civilian-controlled satellites and drones, social media, and groundbreaking big data projects (such as the GDELT initiative), all provide new and powerful ways to gather data within conflict contexts on both program activities and broader context. And gathering this information will only get easier. The challenge, therefore, is creating a feedback loop, and the only way this can happen is if projects are allowed to adapt in response to the data they gather.
Finally, there is hard evidence that adaptive projects are simply more effective. In a recent blog post, Duncan Green, a strategic advisor for Oxfam Great Britain, discusses a study of the evaluations of 10,000 development projects. The paper reports that, in general, giving program implementers flexibility to adapt to changing realities makes their projects more effective. That impact is stronger in complex environments like conflict zones. A recent evaluation of USAID reconciliation programming similarly found that "programs are most effective when they are adaptively implemented."
Peacebuilders thus know how to implement adaptive programming, and they know it works. But many of the organizations doing good programming are succeeding despite the demands of their funders, compliance officers, or inspectors general. They find ways to be more flexible, while at the same time doing just enough to keep the "counter-bureaucracy" happy. The shift that is needed is an approach to accountability that doesn’t just create space at the margins for creative organizations to do flexible programming, but which demands that all organizations do this sort of programming. It must then require that organizations provide evidence explaining programming decisions they made and the results achieved. (As a bonus, this approach can safeguard taxpayers’ money.)
As rallying cries go, "More, but different accountability!" probably won’t inspire the masses to march in the streets. For peacebuilders, though, who are used to the long, hard, often dangerous tasks involved in helping countries put themselves back together, it might resonate. It’s time to stop making the hard work even harder.
Andrew Blum is the vice president for program management and evaluation at the United States Institute of Peace. The views expressed here are his own.