¹ Coralie Bryant, quoted in “Asking the Do-Gooders to Prove They Do Good,” New York Times, January 3, 2004, p. A9.

² This paper does not discuss international humanitarian (“emergency”) aid. Fiona Terry, Condemned to Repeat? The Paradox of Humanitarian Action (Ithaca: Cornell University Press, 2002) is an excellent discussion of the issues that have raised concerns about accountability in humanitarian aid agencies. See also the Humanitarian Accountability Partnership International website; available at http://www.hapinternational.org. I bracket for the time being the point that a significant proportion of what is counted as international development aid is not given with the aim of bettering the conditions of the poor, but is given for political and strategic reasons.

³ Fleshman, Michael, “Africa Pushes for Better Aid Quality,” Africa Recovery 17,no. 4 (2004),p.18Google Scholar. In this section, and in sections 3 and 4, I draw on Leif Wenar, “What We Owe to Distant Others,” Politics, Philosophy and Economics 2, no. 3 (2003), pp. 283–304; and Leif Wenar, “The Basic Structure as Object: Institutions and Humanitarian Concern,” Canadian Journal of Philosophy, supp. vol (forthcoming).

⁴ Charles Handy, Understanding Organizations, Fourth ed. (New York: Penguin Global, 2005); and Basil Edward Cracknell, Evaluating Development Aid: Issues, Problems and Solutions (New Delhi: Sage, 2000).

⁵ It is not uncommon for projects to face internal as well as environmental challenges for example when a funder fails to follow through on its commitment to provide resources during the implementation of the project.

⁶ Norman Uphoff, “Why NGOs Are Not a Third Sector: A Sectoral Analysis with Some Thoughts on Accountability, Sustainability, and Evaluation,” in Michael Edwards and David Hulme, eds.,Beyond the Magic Bullet: NGO Performance and Accountability: Introduction and Overview (London: Earthscan, 1995), pp. 23–29; and Peter Oakley, The Danish NGO Impact Study: A Review of Danish NGO Activities in Developing Countries Overview Report (Oxford: INTRAC, 1999), pp. 81–83.

⁷ Susan Rose-Ackerman, “Governance and Corruption,” in Bjorn Lomborg, ed., Global Crises, Global Solutions (Cambridge: Cambridge University Press, 2004), pp. 301–44.

⁸ Nazmul Chaudhury and Jeffrey Hammer, “Ghost Doctors: Absenteeism in Bangladeshi Health Facilities,” World Bank Policy Research Working Paper no. 3065 (Washington, D.C.: World Bank, 2003).

⁹ See Denise Dresser, “Bringing the Poor Back In: National Solidarity as a Strategy of Regime Legitimation,” in Wayne Cornelius, Ann Craig, and Jonathan Fox, eds., Transforming State-Society Relations in Mexico: The National Solidarity Strategy (La Jolla, Calif.: Center for US-Mexican Studies, 1994), pp. 143–66. Alberto Diaz-Cayeros and Beatriz Magaloni estimate that the PRONASOL funds, if perfectly targeted as monetary transfers, would have alleviated one third of the severe poverty in Mexico (where 15 percent of the population was under the $1 per day international poverty line during the period when the program was implemented). See Alberto Diaz-Cayeros and Beatriz Magaloni, “The Politics of Public Spending: Part II, The Programa Nacional de Solidaridad (PRONASOL) in Mexico,” World Bank Background Paper no. 28013 (Washington, D.C.: World Bank, 2003), pp. 2–3.

¹⁰ Jean-Paul Azam, Shantayan Devarajan, and Stephen O’Connell, “Aid Dependence Reconsidered,” World Bank Policy Research Working Paper no. 2144 (Washington, D.C.: World Bank, 1999); Adam, Christopher andO’Connell, Stephen, “Aid, Taxation, and Development in Sub-Saharan Africa,” Economics and Politics 11,no. 3 (1999), pp.225–53CrossRefGoogle Scholar.

¹¹ Raghuram G. Rajan and Arvind Subramanian, “Aid and Growth: What Does the Cross-Country Evidence Really Show?” National Bureau of Economic Research Working Paper no. 11513 (Cambridge, Mass.: NBER, 2005).

¹² Fleshman, “Africa Pushes for Better Aid Quality,” p. 18; and Michael Edwards and David Hulme, “NGO Performance and Accountability: Introduction and Overview,” in Michael Edwards and Alan Fowler, eds., The Earthscan Reader on NGO Management (London: Earthscan, 2002), p. 190.

¹³ The report reads, “Whatever the projects’ individual qualities, the collective impact was chaotic, because of the huge administrative burden on the Tanzanian government and because the projects employed large numbers of the most qualified people in the country, many of whom had been lured away from the public sector.” UNDP, Development Effectiveness Report (New York: United Nations, 2003), p. 41.

¹⁴ See, for example, Mosley, Paul,Hudson, John, and Horrell, Sara, “Aid, The Public Sector and the Market in Less Developed Countries,” The Economic Journal 97,no. 387 (1987), pp.616–41CrossRefGoogle Scholar; Boone, Peter, “Politics and the Effectiveness of Foreign Aid,” European Economic Review 40,no. 2 (1996), pp.289–329CrossRefGoogle Scholar; Craig Burnside and David Dollar, “Aid, Policies, and Growth: Revisiting the Evidence,” World Bank Policy Research Working Paper 3251 (Washington, D.C.: World Bank, 2004), pp. 847–68, available at econ.worldbank.org/external/default/main?pagePK=64165259&piPK=64165421&menuPK=64166093&theSitePK=469372&entityID=000009486_20040421103444; David Roodman, “The Anarchy of Numbers: Aid, Development, and Cross-Country Empirics,” Center for International Development Working Paper 32 (Cambridge, Mass.: CID, 2004), available at ideas.repec.org/p/wpa/wuwpdc/0412003.html; Rajan and Subramanian, “Aid and Growth: What Does the Cross-Country Evidence Really Show?”; and Edwards and Fowler, The Earthscan Reader, p. 190.

¹⁵ Grant, Ruth andKeohane, Robert, “Accountability and Abuses of Power in World Politics,” American Political Science Review 99,no. 1 (February 2005), pp.29–43.CrossRefGoogle Scholar

¹⁶ Robert D. Behn, Rethinking Democratic Accountability (Washington, D.C.: Brookings Institution, 2001), p. 3; and Andreas Schedler, “Conceptualizing Accountability,” in Andreas Schedler, Larry Diamond, and Marc F. Plattner, The Self-Restraining State: Power and Accountability in New Democracies (Boulder, Colo.: Lynne Rienner, 1999), pp. 13–28; Ronald Oakerson, “Governance Structures for Enhancing Accountability and Responsiveness,” in James L. Perry, Handbook of Public Administration (San Francisco: Jossey-Bass, 1989), p. 114; Norman Daniels and James Sabin, Setting Limits Fairly (Oxford: Oxford University Press 2002), pp. 45–63.

¹⁷ Some authors writing on development use the term “accountability” in a looser sense, requiring only that an accountable agent show itself to be responding to the demands of what it is responsible for; or in an “agency” sense, requiring that one actor be acting on behalf of another. See Hugo Slim, “By What Authority? The Legitimacy and Accountability of Non-Governmental Organizations,” Journal of Humanitarian Assistance (March 2002), available at http://www.jha.ac/articles/a082.htm; Bernard Manin, Adam Przeworski, and Susan Stokes, eds., Democracy, Accountability, and Representation (Cambridge: Cambridge University Press, 1999), pp. 8–10; Jon Elster, “Accountability in Ancient Athenian Politics,” in Manin, Przeworski, and Stokes, eds., Democracy, Accountability and Representation, pp. 253–78; James Fearon, “Electoral Accountability and the Control of Politicians: Selecting Good Types versus Sanctioning Poor Performance,” in Manin, Przeworski, and Stokes, eds., Democracy, Accountability and Representation, pp. 55–97; Mulgan, Richard, “Accountability: An Ever-Expanding Concept?” Public Administration 78,no. 3 (2002), pp.555–73CrossRefGoogle Scholar; Simon Burall and Caroline Neligan, “The Accountability of International Organizations,” GPPi Research Paper Series, no. 2 (Geneva and Berlin: Global Public Policy Institute, 2005), available at http://www.globalpublicpolicy.net/index.php?id=106.

¹⁸ See Elster, “Accountability in Ancient Athenian Politics”; Edwards and Hulme, “NGO Performance and Accountability,” p. 196; Ebrahim, Alnoor, “Accountability in Practice: Mechanisms for NGOs,” World Development 31,no. 5 (2003), pp.813–29CrossRefGoogle Scholar; Tina Wallace and Jennifer Chapman, “An Investigation into the Reality Behind NGO Rhetoric of Downward Accountability,” in Lucy Earle, ed., Creativity and Constraint: Grassroots Monitoring and Evaluation and the International Aid Area (Oxford: INTRAC, 2004); and Lisa Jordan, “Mechanisms for NGO Accountability,” GPPi Research Paper Series no. 3 (Cambridge, Mass.: Global Public Policy Institute, 2005), available at http://www.globalpublicpolicy.net/fileadmin/gppi/Jordan_Lisa_05022005.pdf. For a sensitive and wide-ranging analysis of the costs of accountability, see Onora O’Neill, A Question of Trust: The BBC Reith Lectures 2002 (Cambridge: Cambridge University Press, 2002).

¹⁹ Titus Alexander, “In the Name of the People: Strengthening Global Accountability,” One World Trust Pamphlet (London: One World Trust, 2000); available at http://www.oneworldtrust.org.

²⁰ Occasionally a nonspecialist book that describes the failings of NGO development efforts attracts some public attention; e.g., Alex de Waal, Famine Crimes: Politics and the Disaster Relief Industry in Africa (Bloomington, Ind.: Indiana University Press, 1998); and Michael Maren, The Road to Hell: The Ravaging Effects of Foreign Aid and International Charity (New York: Free Press, 2002). The main effect of such books seems to be to reinforce a generalized aid skepticism in some segments of the public instead of leading to sanctions on specific NGOs.

²¹ Ackerman, John M., “Co-Governance for Accountability: Beyond ‘Exit’ and ‘Voice,’” World Development 32,no. 3, (2004), pp.447–63CrossRefGoogle Scholar.

²² Woods, Ngaire, “Making the IMF and the World Bank More Accountable,” International Affairs 77,no. 1, (2001), pp.83–100CrossRefGoogle Scholar; Devesh Kapur, “The Changing Anatomy of Governance of the World Bank,” in Jonathan R. Pincus and Jeffrey A. Winters, eds., Reinventing the World Bank (Ithaca: Cornell University Press, 2002), pp. 54–75; Bruce Rich, “The World Bank under James Wolfensohn,” in ibid., pp. 26–53; and Jonathan Fox, “The World Bank Inspection Panel and the Limits of Accountability,” in ibid., pp. 131–63.

²³ OECD, OECD Action for a Shared Development Agenda (Paris: OECD, 2002); available at http://www.oecd.org/document/46/0,2340,en_2649_33721_2088942_1_1_1_1,00.html.

²⁴ USAID controlled $5.7 billion of the $12.6 billion US foreign aid budget in 2004. The Millennium Challenge Corporation is now growing rapidly, and its ascendancy will represent a major shift in how US aid is administered. See Curt Tarnoff and Larry Nowels, “Foreign Aid: An Introductory Overview of US Programs and Policy,” (Washington, D.C.: Congressional Research Service Report for Congress, 2004); available at usinfo.state.gov/usa/infousa.trade.files/98-916.pdf.

²⁵ Here I draw on Paul Clements, “Development as if Impact Mattered” (Dissertation, Woodrow Wilson School, Princeton University, 1996).

²⁶ Schraeder, Peter J.,Hook, Steven W., andTaylor, Bruce, “Clarifying the Foreign Aid Puzzle: A Comparison of American, Japanese, French, and Swedish Aid Flows,” World Politics 50,no. 2 (1998), pp.294–323CrossRefGoogle Scholar. The Congressional Research Service reports: “Most US foreign aid is used for procurement of US goods and services, although amounts of aid coming back to the United States differ by program. No exact figure is available due to difficulties in tracking procurement item by item, but some general estimates are possible for individual programs, though these may differ year to year . . . Food assistance commodities are purchased wholly in the United States, and most expenditures for shipping those commodities to recipient countries go entirely to US freight companies. Under current legislation, three-fourths of all food aid must be shipped by US carriers. On this basis, a rough estimate suggests that more than 90%—at least $1 billion in FY2004—of food aid expenditures will be spent in the United States . . . Most bilateral development assistance and the ESF, NIS and SEED components of economic, political and security assistance support programs in developing countries and the new European democracies, respectively. Although a small proportion of funding for these programs results in transfers of US dollars, the services of experts and project management personnel, and much of the required equipment, is procured from the United States. According to USAID, 81% of total USAID procurement between October 2002 and September 2003 under these programs came from US sources.” Tarnoff and Nowels, “Foreign Aid,” pp. 18–19.

²⁷ See ibid., pp. 294–323; and Alesina, Alberto andDollar, David, “Who Gives Foreign Aid to Whom and Why?” Journal of Economic Growth 5,no. 1 (2000), pp.33–63.CrossRefGoogle Scholar

²⁸ Jonathan A. Fox and L. David Brown, The Struggle for Accountability: The World Bank, NGOs and Grassroots Movements (Cambridge: MIT Press, 1998); Robert Dahl, “Can International Organizations be Democratic?” in Ian Shapiro and Casiano Hacker-Cordon, eds., Democracy's Edges (Cambridge: Cambridge University Press, 1999), pp. 19–36; and Grant and Keohane, “Accountability and Abuses of Power in World Politics.” For critical views on these institutions generally see the Bretton Woods Project website (http://www.brettonwoodsproject.org). The World Bank has an Inspection Panel, to which individuals and groups can protest that a Bank-funded project has harmed or will harm their interests (available at wbln0018.worldbank.org/IPN/IPNWeb.nsf?OpenDatabase). The effectiveness of the Inspection Panel as a mechanism of accountability has been controversial; see Dana Clark, Jonathan Fox, and Kay Treakle, eds., Demanding Accountability: Civil Society Claims and the World Bank Inspection Panel (Lanham, Md.: Rowman & Littlefield, 2003).

²⁹ Grant and Keohane, “Accountability and Abuses of Power in World Politics”; Wapner, Paul,Spero, Peter J.,Spar, Debora andDail, James, andKingsbury, Benedict, symposium on “The Democratic Accountability of Non-Governmental Organizations,” Chicago Journal of International Law 3,no. 1 (2002)Google Scholar. For views critical of NGO accountability, see the papers from the American Enterprise Institute's June 11, 2003 conference in Washington, D.C., “We’re Not from the Government, But We’re Here to Help You”; available at http://www.ngowatch.org/ngo.htm.

The discussion of NGOs here focuses primarily on how NGOs fund and implement development projects. Several major NGOs are increasingly engaged in political advocacy campaigns, and accountability for such campaigns will remain limited by the great difficulties of measuring their impacts on poverty. It should also be noted that while the study of NGOs reveals much about the state of accountability in development aid, discussions of NGO effectiveness should be kept in context. NGOs account for a relatively small percentage of global aid expenditure, the most commonly cited figure being 15 percent of the total.

³⁰ Edwards and Hulme, “NGO Performance and Accountability.”

³¹ See ibid., p. 190: “Internal evaluations are rarely released and what is released comes closer to propaganda than rigorous assessment”; and Slim, “By What Authority?”

³² John de Coninck, Current Procedures and Policies Dominating Aid: Building Strong Relationships and Enabling NGOs to Meet their Stated Aims? (Uganda: Community Development Resource Network, 2004); and Wallace and Chapman, “An Investigation into the Reality Behind NGO Rhetoric of Downward Accountability,” pp. 23–46.

³³ Ian Smillie, “Changing Partners: Northern NGOs, Northern Governments,” in Ian Smillie and Henry Helmich, eds., Non-Governmental Organizations and Governments: Stakeholders for Development (Paris: OECD, 1993), pp. 155–92.

³⁴ See ibid.; Edwards and Hulme, “NGO Performance and Accountability”; and Shiras, Peter, “The New Realities of Non-Profit Accountability,” Alliance 8,no. 4 (2003)Google Scholar, available at http://www.allavida.org/alliance/dec03c.html.

³⁵ See John Hailey and Mia Sorgenfrei, “Measuring Success: Issues in Performance Measurement,” INTRAC Occasional Paper Series no. 44 (Oxford: INTRAC, 2004), available at http://www.intrac.org/resources_database.php?id=61, which also contains a history of performance measuring standards in the private and public sectors.

³⁶ Ebrahim, “Accountability in Practice,” pp. 813–29.

³⁷ See Edwards and Hulme, eds., Beyond the Magic Bullet; Coninck, Current Procedures and Policies Dominating Aid; and Jordan, “Mechanisms for NGO Accountability.”

³⁸ Richard Chambers, Whose Reality Counts? Putting the Last First (London: ITDG Press, 1997); Somesh Kumar, Methods of Community Participation (London: ITDG Press, 2003); Chris Roche, Impact Assessment for Development Agencies: Learning to Value Change (Sterling, Va.: Stylus, 1999); Frits Wils, “Scaling Up Mainstream Accountability: The Challenge for NGOs,” in Edwards and Hulme, eds., Beyond the Magic Bullet, pp. 53–62. For some doubts see Frances Cleaver, “Beyond Partnership: Getting Real about NGO Relationships in the Aid System,” in Edwards and Fowler, eds., The Earthscan Reader, pp. 225–40.

³⁹ Wallace and Chapman, “An Investigation into the Reality Behind NGO Rhetoric of Downward Accountability.”

⁴⁰ Burall and Neligan, “The Accountability of International Organizations.”

⁴¹ Roche, Impact Assessment.

⁴² Yedla Padmavathi, “Whose Dreams? Whose Voices? Involving Children in Project Management,” and Jennifer Chapman, Rosalind David, and Antonella Mancini, “Transforming Practice in ActionAid: Experiences and Challenges in Rethinking Learning, Monitoring and Accountability Systems,” in Earle, ed., Creativity and Constraint.

⁴³ Norman Uphoff, “Why NGOs Are Not a Third Sector: A Sectoral Analysis with Some Thoughts on Accountability, Sustainability, and Evaluation,” in Edwards and Hulme, eds., Beyond the Magic Bullet, p. 20. There is anecdotal evidence that in some sub-Saharan African states, the saturation of aid agencies on the ground is great enough that community leaders do have some power to choose among the various projects that different agencies offer. This may provide an accountability mechanism analogous to that of consumer choice among for-profit corporations, as community leaders have the power to choose the “service” that will serve them or their community best. One might speculate that this mechanism of accountability might become more important were significantly more aid funds channeled through NGOs.

However, as it stands the power of choice mostly goes in the other direction: NGOs can choose which communities to work in, and are less likely to choose to work in communities that have the capacity to sanction them. For instance, NGOs would be much less willing to work in jurisdictions where there is a World Bank–style Inspection Panel (see note 28 above), or still less where they could be sued for negligence (as Kunibert Raffer recommends with respect to the international financial institutions, in his “Reforming the Bretton Woods Institutions,” Zagreb International Review of Economics and Business Special Issue [2002], pp. 97–109; available at http://www.networkideas.org/featart/may2005/Bretton_Woods.pdf).

⁴⁴ OECD, “Paris Declaration on Aid Effectiveness” (Paris: OECD, 2005), available at http://www.oecd.org/dataoecd/11/41/34428351.pdf; Ackerman, “Co-Governance for Accountability,” pp. 447–63; and Anne Marie Goetz and Rob Jenkins, “Hybrid Forms of Accountability and Human Development: Citizen Engagement of a New Agenda,” Background Paper for Human Development Report 2002 (New York: UNDP, 2002).

⁴⁵ See Guillermo O'Donnell, “Horizontal Accountability in New Democracies,” in Schedler, Diamond, and Plattner, eds., The Self-Restraining State; and Ackerman, “Co-Governance for Accountability,” pp. 447–63.

⁴⁶ I draw here on a proposal for an evaluation association co-authored with Paul Clements. Clements came up with the idea for the association, and is the driving force behind the proposal.

⁴⁷ Roche, Impact Assessment; and Simon Starling, “Balancing Measurement, Management, and Accountability: Lessoned Learned from Save the Children UK's Impact Assessment Framework” (presented at INTRAC Conference, Oxford, March 31 to April 4, 2003); available at http://www.intrac.org/docs/Starling.pdf. The use of randomized evaluation methods to address the problem of counterfactuals is a promising recent development. See Esther Duflo and Michael Kremer, “Use of Randomization in the Evaluation of Development Effectiveness,” in George Pitman, Osvaldo Feinstein, and Gregory Ingram, eds., Evaluating Development Effectiveness (New Brunswick, N.J.: Transaction Publishers, 2005), pp. 205–31.

⁴⁸ Jerker Carlsson, Gunnar Kohlin, and Anders Ekbom, The Political Economy of Evaluation: International Aid Agencies and the Effectiveness of Aid (London: Macmillan Press, 1994); Clements, Paul, “Informational Standards in Development Agency Management,” World Development 27 (1999), pp.1359–81.CrossRefGoogle Scholar

⁴⁹ Carlsson et al., The Political Economy of Evaluation, p. 180, report on the related process of project appraisal that: “Even an appraisal system as rigorous as the World Bank's is in practice continuously being manipulated, because it is subordinated to the individual interests of POs [project officers] (getting projects to the Board) as well as the organization's own objectives (meeting the disbursement targets). . . . Individuals are rational in the sense that they defend their, or their group's, interests.” The authors of this book do not allege that evaluations are positively biased so much that evaluations are based on such inconsistent assumptions and methodologies as to be practically useless.

⁵⁰ Clements, “Informational Standards,” pp. 1359–81.

⁵¹ World Bank, 2004 Annual Review of Development Effectiveness: The World Bank's Contribution to Poverty Reduction (Washington, D.C., World Bank, 2005), pp. 51–52; available at http://www.worldbank.org/oed/arde/2004/main_report.html.

⁵² Roger C. Riddell, Stein-Erik Kruse, Timo Kyllönen, Satu Ojanperä, and Jean-Louis Vielajus, “Searching for Impact and Methods: NGO Evaluation Synthesis Study” (Helsinki: Institute of Development Studies, 2001), p. 99, available at http://www.valt.helsinki.fi/ids/ngo, also say (p. 24): “If there is one consistent theme to come out of the majority of the country case studies it is that for the sheer numbers of evaluations that have been carried out, there are very few rigorous studies which examine impact: improvements in the lives and livelihoods of the beneficiaries.” Similar conclusions are reached in Oakley, The Danish NGO Impact Study, pp. 29–51, an overview study of Danish development aid.

⁵³ Rick Davies, “Monitoring and Evaluating NGO Achievements,” in Vandana Desai and Robert B. Potter, eds., The Arnold Companion to Development Studies (London: Hodder Arnold, 2002), p. 524; also see Stein-Erik Kruse, “Meta-Evaluations of NGO Experience: Results and Challenges,” in Pitman, Feinstein, and Ingram, eds., Evaluating Development Effectiveness, pp. 109–26.

⁵⁴ Cracknell, Evaluating Development Aid, contains a good introduction to the history of evaluation. One sees in this book how evaluation professionals have within the short span of forty years invented, standardized, and refined the fundamental techniques of project analysis, as well as the basic norms of professionalism. See Roche, Impact Assessment, on the even more recent evolution of (long-term) impact assessment.

⁵⁵ One reviewer for this journal has raised the possibility that project evaluation faces too many technical and methodological difficulties to be rigorous and reliable except in very rare circumstances. Yet given that proposals like the current one seem feasible and their potential benefits appear to be significant, it seems too early to give in to skepticism concerning the possibilities for making progress on evaluation.