«Development, Security, and Cooperation Policy and Global Affairs THE NATIONAL ACADEMIES PRESS 500 Fifth Street, N.W. Washington, DC 20001 NOTICE: The ...»
Besides such a multicountry case study design, the committee also believes that there are other ways for USAID to learn from its past DG activities. These include discussions of outside studies of DG assistance, such as those undertaken by the Carnegie Endowment (e.g., Carothers 2006, 2007) or other nations’ development agencies, statistical analyses of international data, and surveys. These also include making better use of the experience of USAID DG mission personnel by engaging in regular meetings in which DG officers could share and discuss their own experiences with democracy assistance. Although not adequate for determining the impact of specific projects, such sources can provide valuable insights regarding problems of program implementation, responses to rapidly changing conditions in the field, issues in the reception of DG programs, or the shifting contexts in which such programs are carried out. USAID should include these varied sources of information as part of the regular organizational learning activities recommended in Chapter 8.
LEARNING FROM THE PAST The committee thus recommends the use of more diverse and theoretically structured clusters of case studies of democratization and democracy assistance to develop hypotheses to guide democracy assistance planning in a diverse range of settings. Whether USAID chooses to support such studies or gather them from ongoing academic research, it is important to look at how democracy assistance functions in a range of different initial conditions and trajectories of political change. Such case studies should seek to map out long-term trajectories of political change and to place democracy assistance in the context of national and international factors affecting those trajectories, rather than focus mainly on specific democracy assistance programs.
REFERENCESAbbink, J., and Hesseling, G., eds. 2000. Election Obseration and Democratization in Africa.
New York: St. Martin’s Press.
Asia Watch. 2002. Cambodia’s Commune Elections: Setting the Stage for the 2003 National Elections. HRW Index 14(4). Available at: http://www.hrw.org/reports/00/camb)0/. Accessed on January 10, 2008.
Barro, R.J. 1999. Determinants of Democracy. Journal of Political Economy 6:158-183.
Barya, J.J., Opolot, S.J., and Otim, P.O. 2004. The Limits of “No Party” Politics: The Role of International Assistance in Uganda’s Democratisation Process. Working Paper 28. Conflict Research Unit, Netherlands Institute of International Relations, Clingendael.
Berman, S.E. 1997. Civil Society and the Collapse of the Weimar Republic. World Politics 49(3):401-429.
Berman, S.E. 2001. Modernization in Historical Perspective: The Case of Imperial Germany.
World Politics 53(2):431-462.
Blair, H., and Hansen, G. 1994. Weighing in on the Scales of Justice: Strategic Approaches for Donor-Supported Rule of Law Programs. CDIE Program and Operations Assessment Report No. 7. Washington, DC: USAID.
Boix, C., and Stokes, S. 2003. Endogenous Democratization. World Politics 55(4):517-549.
Carothers, T. 1999. Aiding Democracy Abroad: The Learning Cure. Washington, DC: Carnegie Endowment.
Carothers, T. 2004. Critical Mission: Essays on Democracy Promotion. Washington, DC: Carnegie Endowment.
Carothers, T. 2006. Confronting the Weakest Link: Aiding Political Parties in New Democracies.
Washington, DC: Carnegie Endowment.
Carothers, T. 2007. The “Sequencing” Fallacy. Journal of Democracy 18(1):12-27.
Carter, L. 2001. Linking USAID Democracy Program Impact to Political Change: A Synthesis of Findings from Three Case Studies. Revised draft (unpublished).
Carter, L., Silver, R., and Smith, Z. 2003. Linking USAID Democracy Program Impact to Political Change: A Synthesis of Findings from Six Case Studies. Washington, DC: Management Systems International.
CDDRL (Center for Democracy, Development, and the Rule of Law). 2006. Project Prospectus. Unpublished.
Dobbins, J. 2003. America’s Role in Nation Building: From Germany to Iraq. Santa Monica, CA:
IMPROVING DEMOCRACY ASSISTANCE Dougherty, B.K. 2004. Searching for Answers: Sierra Leone’s Truth & Reconciliation Commission. African Studies Quarterly 8(1). Online. Available at http://web.africa.ufl.edu/asq/ /ia.hm.
Doyle, A.C. 1998. The Adenture of the Beryl Coronet, The Adentures of Sherlock Holmes. Oxford World’s Classics. New York: Oxford University Press.
Epstein, D., Bates, R., Goldstone, J.A., Kristensen, I., and Halloran, S. 2006. Democratic Transitions. American Journal of Political Science 50:551-569.
Finkel, S.E., Pérez-Liñán, A., and Seligson, M.A. 2007. The Effects of U.S. Foreign Assistance on Democracy Building, 1990-2003. World Politics 59(3):404-439.
George, A., and Bennett, A. 2005. Case Studies and Theory Deelopment in the Social Sciences.
Cambridge, MA: MIT Press.
Goldstone, J.A. 1998. Initial Conditions, General Laws, Path-Dependence, and Explanation in Historical Sociology. American Journal of Sociology 104:829-845.
Goldstone, J.A. 2003. Comparative Historical Analysis and Knowledge Accumulation in the Study of Revolutions. Pp. 41-90 in Comparatie Historical Analysis, D. Reuschemeyer and J. Mahoney, eds. Cambridge: Cambridge University Press.
Goldstone, J.A. 2007. Trajectories of Democracy and Development: New Insights from Graphic Analysis. Paper presented to Wilton House, UK, October 23.
Goldstone, J.A., and Ulfelder, J. 2004. How to Construct Stable Democracies. Washington Quarterly 28(1):9-20.
Haggard, S., and Kauffman, R.R. 1995. The Political Economy of Democratic Transitions.
Princeton, NJ: Princeton University Press.
Kumar, K. 1998. Post-Conflict Elections, Democratization, and International Assistance.
Boulder: Lynne Rienner.
Lippman, H., and Emmert, J. 1997. Assisting Legislatures in Deeloping Countries: A Framework for Program Planning and Implementation. Washington, DC: USAID.
McFaul, M. 2005. Transitions from Post-Communism. Journal of Democracy 16(3):5-19.
McFaul, M. 2006. The 00 Presidential Elections in Ukraine and the Orange Reolution: The Role of U.S. Assistance. Washington, DC: USAID, Office for Democracy and Governance.
McMahon, E., Beale, S., and Menelik-Swanson, G. 2004. Ethiopia Pre-Election Assessment
Report. Washington, DC: International Foundation for Election Systems. Available at:
Mendelson, S.E., and Glenn, J.K., eds. 2002. The Power and Limits of NGOs: A Critical Look at Building Democracies in Eastern Europe and Eurasia. New York: Colombia University Press.
O’Neill, W.G. 2003. International Human Rights Assistance: A Review of Donor Activities and Lessons Learned. Working Paper No. 18. The Hague, Netherlands: Clingendael Institute.
Przeworski, A., Alvarez, M.E., Cheibub, J.A., and Limongi, F. 2000. Democracy and Deelopment: Political Institutions and Well-Being in the World, 0-0. Cambridge: Cambridge University Press.
Rice, C. 2006. Transformational Diplomacy. U.S. Department of State. Available at: http:// www.state.go/r/pa/prs/ps/00/.htm.
Whitehead, L. ed. 1996. The International Dimensions of Democracy: Europe and the Americas.
Oxford: Oxford University Press.
Youngs, R. 2004. International Democracy and the West: The Role of Goernments, NGOs and Multinationals. Oxford: Oxford University Press.
Zakaria, F. 2004. The Future of Freedom. New York: Norton.
de Zeeuw, J., and Kumar, K. 2006. Promoting Democracy in Post-Conflict Societies. Boulder:
Methodologies of Impact Evaluation
INTRODUCTIONThis chapter presents a guide to impact evaluations as they are currently practiced in the field of foreign assistance. The committee recognizes, as stated before, that the application of impact evaluations to foreign assistance in general, and to democracy and governance (DG) projects in particular, is controversial. The purpose of this chapter is thus to present the range of impact evaluation designs, as a prelude to the results of the committee’s field teams’ exploration of their potential application as part of the mix of evaluations and assessments undertaken by the U.S. Agency for International Development (USAID) presented in the next two chapters.
The highest standard of credible inference in impact evaluation is achieved when the number of people, villages, neighborhoods, or other groupings is large enough, and the project design flexible enough, to allow randomized assignment to treatment and nontreatment groups. Yet the committee realizes that this method is often not practical for many DG projects. Thus this chapter also examines credible inference designs for cases where randomization is not possible and for projects with a small number of units—or even a single case—involved in the project.
Some of the material in this chapter is somewhat technical, but this is necessary for this chapter to serve, as the committee hopes it will, as a guide to the design of useful and credible impact evaluations for DG missions and implementers. The technical terms used here are defined in the chapter text and also in the Glossary at the end of the report. Also,
examples are provided to show how such designs have already been implemented in the field for various foreign assistance and democracy assistance programs.
IMPORTANCE OF SOUND AND CREDIBLE IMPACT
EvALUATIONS FOR Dg ASSISTANCEAs discussed in some detail in Chapter 2, until 1995 USAID required evaluations of all its projects, including those in DG, to assess their effectiveness in meeting program goals. Most of the evaluations, however, were process evaluations: post-hoc assessments by teams of outside experts who sought to examine how a project unfolded and whether (and why) it met anticipated goals. While these were valuable examinations of how projects were implemented and their perceived effects, such evaluations generally could not provide the evidence of impact that would result from sound impact assessments. This was because in most cases they lacked necessary baseline data from before the project was begun and because in almost all cases they did not examine appropriate comparison groups to determine what most likely would have occurred in the absence of the projects (see Bollen et al  for a review of past DG evaluations).
As noted, the number of such evaluations undertaken by USAID has declined in recent years. Evaluations are now optional and are conducted mainly at the discretion of individual missions for specific purposes, such as when a major project is ending and a follow-on is expected or when a DG officer feels that something has “gone wrong” and wants to understand and document the reasons for the problem. Such special evaluations can have substantial value for management purposes, but the committee believes that USAID is overlooking a major opportunity to learn systematically from its experience about project success and failure by not making impact evaluations a significant part of its monitoring and evaluation (M&E) activities where appropriate and feasible. Such impact evaluations could be particularly useful to provide insights into the effects of its largest-scale and most frequently used projects and to test key development hypotheses that guide its programming.