Relative Effectiveness of Bilateral and Multilateral Aid on Development Outcomes

Aid donors are interested in understanding whether allocating aid via bilateral or multilateral channels might be more effective for achieving development goals. We review 45 papers that empirically test the associations between bilateral and multilateral aid flows and various development outcomes including gross domestic product growth, governance indicators, human development indicators and levels of non&#8208;aid investment flows. Findings suggest that differences between countries and regions, time periods, aid objectives, and individual donor organizations all may influence the effectiveness of aid delivered bilaterally and multilaterally. We find, however, no consistent evidence that either bilateral or multilateral aid is more effective overall.


Introduction
Foreign aid is the most important source of international funding for countries with government spending below PPP 1 US$500 per person per year, accounting for roughly 70% of funding from international sources (Coppard et al., 2013). According to the most recent available data from the Organization for Economic Cooperation and Development (OECD) Development Assistance Committee (DAC) Creditor Reporting System (CRS), total net Overseas Development Assistance (ODA) flows 2 from DAC countries to developing countries were US$137.2 billion in 2014. While most of this aid was distributed bilaterally (directly from donor countries to recipient countries, or to multilateral organizations with donor-imposed restrictions on its use), US$42.6 billion (31%) went to multilateral organizations without any use restrictions and was distributed as multilateral ODA. This share of multilateral ODA has been largely consistent since at least 1998 (OECD, 2015). 3 Donors and recipients appear increasingly concerned about the accountability and effectiveness of aid, as highlighted by the 2005 Paris Declaration on Aid Effectiveness, the 2008 High Level Forum on Aid Effectiveness, the 2011 Busan Partnership Agreement for Effective Development Cooperation, and the emergence and expansion of aid monitoring and reporting organizations including the International Aid Transparency Initiative and the OECD-DAC CRS. Initiatives by the OECD, the Center for Global Development (CGD), the United Kingdom Department for International Development (DFID) and others have sought to review the performance of donor organizations against a variety of indicators believed to affect aid effectiveness. These reviews typically evaluate donor organizations individually, but often draw more general comparisons between bilateral and multilateral donors, as multilateral aid is frequently characterized as being relatively more focused on supporting development outcomes in developing countries, while bilateral aid is seen as more likely to be allocated based on donor strategic interests (Alesina and Dollar, 2000;Burnside and Dollar, 2000;Milner and Tingley, 2013;Schraeder et al., 1998). 4 In this paper we review the empirical evidence for whether disbursing aid via bilateral or multilateral channels is more effective at achieving development outcomes. While acknowledging that aid effectiveness varies among specific donor organizations (regardless of funding channel), we summarize the major hypotheses from the literature for why either bilateral or multilateral aid might generally be more effective. We then analyze 45 empirical studies testing the relative effectiveness of bilateral vs multilateral aid, summarizing findings by various aspects of study methodology. Our results indicate that aid effectiveness may vary by country or region and by time periods studied, but we find no consistent evidence that either bilateral aid or multilateral aid is more effective, either overall or by study methodology.

Background
We define "aid effectiveness" as the ability to achieve targeted development outcomes as measured by indicators such as GDP growth, increases in measures of human development, and increases in private investment flows. We start with the premise that increases in the likelihood of achieving a desired development outcome through aid spending can be driven by either (i) attracting more aid funds towards a given outcome, or (ii) using existing aid funds more efficiently in support of that outcome. The major theoretical arguments in the literature on aid effectiveness can be classified according to these two broad methods: either considering whether characteristics of bilateral vs multilateral aid contribute to increased aid volumes (either overall funding, or the share mobilized for development goals as opposed to donor geopolitical goals), or considering whether the use of bilateral vs multilateral channels increases the cost-effectiveness of aid in achieving development goals (Table 1).
As summarized in Table 1, bilateral aid is hypothesized to increase funding flows to development outcomes by appealing to donors' strategic interests (Alesina and Dollar, 2000;Barder, 2012;Schraeder et al., 1998), including donor countries' desire for more direct accountability (Christensen et al., 2011;OECD, 2013). In some cases, historical connections with particular developing countries, e.g. former colonies (Cassen, 1994), are also believed to increase donor country willingness to contribute aid via bilateral channels.
However, critics argue that the continued practice of "tying" bilateral aid, often in the form of restrictions requiring aid funds be used to purchase goods from the donor country, can reduce the value of aid to recipients (Clay et al., 2008;OECD, 2014). Others note that the strategic or geopolitical orientation of aid delivered through bilateral channels (Barder, 2012;Berth elemy, 2006;Clay et al., 2008;Chung et al., 2015;Fleck and Kilby, 2006;Rodrik, 1995) and greater fragmentation of bilateral aid (Acharya et al., 2006;Addison et al., 2015;Barder, 2012;Houerou, 2008;OECD, 2012a) might also decrease cost-effectiveness. That said, several authors contend that direct accountability to donors (Barder, 2012), combined with

Notes:
A "+" indicates that providing aid through a particular channel is argued to increase either funding volumes or cost-effectiveness, while a "-" indicates the opposite. 1428 Pierre E. Biscaye, Travis W. Reynolds, and C. Leigh Anderson bilateral donors' institutional compatibility with recipient countries where they have long-standing historical relationships (Cassen, 1994), and the greater volumes of aid disbursed by bilateral donors (Kharas, 2010;OECD, 2013) all serve to increase bilateral aid cost-effectiveness relative to multilateral aid. Multilateral aid channels meanwhile are hypothesized to promote increased funding flows targeting development outcomes (rather than donor country interests) owing to of the development orientation associated with multilateral organizations (Burnside and Dollar, 2000;Milner and Tingley, 2013). Combined with common multilateral organization practices such as pro-development conditionality (Charron, 2011;Ram, 2003;Rodrik;1995) and expansive accountability mechanisms (OECD, 2013;Publish What You Fund, 2016), the relatively more direct and transparent links between multilateral aid and development outcomes may increase some donors' willingness to contribute aid via multilateral rather than bilateral channels, and attract other sources of development finance (Griffith-Jones et al., 2008).
Similarly, in terms of cost-effectiveness the development orientation of multilateral aid (Milner and Tingley, 2013;Miquel-Florensa, 2007), prodevelopment conditionality (Burnside and Dollar, 2000), and the perceived legitimacy of multilateral aid from the perspective of recipient countries (Barder, 2012;Bird et al., 2000;OECD, 2013;Rodrik, 1995) are all believed to increase the cost-effectiveness of multilateral aid relative to bilateral aid. Other factors including the relatively lower fragmentation of multilateral aid (Addison et al., 2015;OECD, 2012a), combined with the specialization and expertise of multilateral aid organizations (Askarov and Doucouliagos, 2015b;Barder, 2012;OECD, 2013;Rodrik, 1995;Uneze, 2012) are further argued to increase multilateral aid costeffectiveness.
These and other theoretical arguments around factors expected to influence aid effectiveness have formed the basis of recent calls for aid reform, including calls for aligning aid with recipient country priorities, using recipient country systems to disperse aid, untying aid, increasing aid transparency and more generally depoliticizing the allocation of aid (OECD, 2008(OECD, , 2012b. Although some studies have compared the aid allocation practices of bilateral vs multilateral aid donors to evaluate how they perform against various criteria (Easterly and Pfutze, 2008;Hashmi et al., 2014;OECD, 2011;OECD/UNDP, 2014), few studies have empirically examined if and how differences in the practices of bilateral vs multilateral donors might lead to improved development outcomes.
Many of the studies that do evaluate impacts of aid do not directly compare the effectiveness of multilateral and bilateral donors. The DFID (2011) multilateral aid review, for example, assessed the "value for money" of UK aid delivered through 43 multilateral organizations. The OECD DAC conducts periodic reviews of the individual development cooperation efforts of its 29 members. Several donor organizations also commission regular independent evaluations, 5 in addition to internal reports and evaluations at different levels. These studies use a variety of evaluation methods, making it difficult to compare donor effectiveness, and do not answer the question of whether aid is more effective when delivered bilaterally or multilaterally.
This research contributes to the literature on aid effectiveness by reviewing the relatively small number of empirical studies comparing the development impacts of aid delivered through bilateral and multilateral channels and evaluating whether there is consistent evidence that either channel is more effective overall. Most of these studies do not directly test the theoretical arguments related to aid BILATERAL AND MULTILATERAL AID EFFECTIVENESS 1429 effectiveness as outlined above. Thus, we analyze the evidence from the studies reviewed on whether bilateral or multilateral aid is relatively more or less effective at improving development outcomes, but do not discuss causal mechanisms behind differences in effectiveness.

Methods
We searched for studies empirically comparing the effectiveness of bilateral vs multilateral aid on Google Scholar, JSTOR, ScienceDirect, Scopus, EconLit, Web of Science and PAIS International using the following search string: ("multilateral aid" AND "bilateral aid") AND (evaluation OR analysis OR effectiveness OR evidence OR impact). These search terms were chosen to focus the results on studies that compared the effectiveness of bilateral vs multilateral aid overall, rather than studies that focused on particular bilateral donors or multilateral agencies. A somewhat broader literature that focused on the allocation of aid or impacts for donor countries, as opposed to the effectiveness of aid in recipient countries, was excluded from the analysis. Studies focused on theory rather than conducting empirical analyses, or that evaluated the effectiveness of aggregate aid only, were also excluded.
Screening the titles and abstracts of search results provided a final sample of 45 studies that: (i) evaluate the relationship between aid and development outcomes in recipient countries; (ii) include empirical data; and (iii) include and distinguish between bilateral and multilateral aid in at least part of their analysis. The sample studies look at different subsets of developing countries and years and employ a wide variety of estimation procedures (summarized in Table 2).
We first summarize findings across all 45 studies. 6 A "more effective" finding indicates either that the authors find significant positive associations for both aid channels but the magnitude of the association is significantly larger for one aid channel, 7 or that the association is significant for only one aid channel. A "mixed" finding indicates that the relative effectiveness of multilateral and bilateral aid differs depending on the model or outcome measure used. A "no difference" finding indicates that the study does not find a statistically significant difference 8 between the impacts of multilateral and bilateral aid for a given outcome. Several studies use multiple models or specifications. In these cases, we follow the authors in reporting the findings that they emphasize as most robust.
We next explore the relationship between study methods and findings. Past research by Roodman (2007) found that cross-country regressions of total aid on growth can be especially sensitive to different instrumental variable techniques, aid measures, data structuring, regression specification (including the timing of aid's effect on growth), treatment of outliers and sample variations. We present tables summarizing study findings by various methodological factors including: (i) hypothesized more effective aid channel, (ii) outcome area, (iii) years of data, (iv) number of countries, (v) sample of countries, (vi) measure of aid, and (vii) primary estimation model. Because the number of studies we review is small, we are limited in our ability to conduct significance tests to determine whether the findings are independent of all methodological choices. We conduct analysis of variance (ANOVA) tests for whether the mean number of countries and mean number of years of data included in the studies is the same across studies with different findings, as these are continuous variables. For summary tables with categorical variables, we conduct Fisher's exact tests (more accurate than v 2 tests when the

Results
Overall, nine studies find that bilateral aid is more effective for a particular outcome measure, 13 find that multilateral aid is more effective and 13 find no significant difference in effectiveness. Ten studies report mixed findings. Table 3 summarizes the findings on the effectiveness of bilateral vs multilateral aid by the authors' hypothesized more effective channel (section A) and by outcome area studied (section B). Thirty-three studies hypothesize either bilateral or multilateral aid to be more effective. In several cases the authors do not explicitly state their hypotheses, but hypotheses can be inferred from their reviews of literature or interpretations of findings. The results of Fisher's exact test (p = 0.002) suggests that findings differ significantly by author hypothesis. Authors who hypothesize a particular aid channel to be more effective appear to be more likely to report findings supporting their hypothesis, though this is not always the case. The 12 studies that do not put forward any hypotheses regarding bilateral or multilateral aid effectiveness are split evenly between finding either bilateral or multilateral aid more effective and reporting no significant difference in effectiveness, with two reporting mixed findings.
Findings do not differ systematically by outcome area studied (p = 0.845). Over half of the studies (23) analyze the effect of aid on a measure related to GDP growth, but an equal number find that multilateral and bilateral are more effective (five studies each), and a greater number report either mixed findings (six studies) or no significant difference in effectiveness (seven). While none of the studies that consider outcomes related to recipient country governance find bilateral aid to be more effective, an equal number find no significant difference in effectiveness as find multilateral aid more effective (three studies each).
We next explored whether this lack of consensus is related to differences in methodology, 9 first conducting ANOVA tests for whether the mean number of countries and mean number of years of data included in the studies vary across studies with different findings.
We find that the mean years of data does not vary significantly across study findings (Prob > F = 0.782). Half or more of studies report mixed findings or no significant difference between bilateral and multilateral aid, regardless of the number of years of data (Table 4A). Four studies (Charron, 2011;Heady, 2008;Rajan and Subramanian, 2008;Ratha, 2011) note that their findings change when they consider subsets of years separately. These differences either indicate that the effectiveness of aid channels is changing over time, that certain events (such as the end of the Cold War) affect the impact of aid on development outcomes (such as by influencing whether aid is distributed for geopolitical or development purposes), or that other factors not included in the models (such as FDI, remittances, or commodity prices) may be driving the changes in country outcomes. Studies that evaluate aid effectiveness over large periods of time may therefore mask variations in effectiveness in different periods.
Study findings do, however, differ significantly with the mean number of countries included in their samples (Prob > F = 0.031). Studies finding bilateral aid to be more effective have the lowest average number of countries in their sample, 1436 Pierre E. Biscaye, Travis W. Reynolds, and C. Leigh Anderson 32.6, including four studies that look at five or fewer countries. In each of these four studies (Feeny, 2005;Gang and Khan, 1990;Gounder, 2001;Khan, 1998), the authors acknowledge that the countries included receive the vast majority of their aid from bilateral sources, which may decrease the likelihood of significant findings on the impact of multilateral aid. Studies finding multilateral aid to be more effective include an average of 45.6 countries and studies that find no difference in effectiveness include an average of 47.8. Studies reporting mixed findings include an average of 81.3 countries in their sample, as seven of 14 studies that looked at more than 75 countries report mixed findings (Table 4B).
The samples of countries selected by the studies vary widely. Though 21 studies specify looking at "developing countries," the number of countries included by these studies varies from 14 to 137. Other samples considered include "transition economies," "low and middle income countries" and for 16 studies, a regional subset of countries (Table 4C). We observe a variety of findings among studies looking at samples of "developing countries". All three studies of small island developing states find bilateral aid more effective, but none of the studies of sub-Saharan African countries reach this conclusion. Thus, the relative effectiveness of bilateral and multilateral aid may differ depending on the context and findings from large samples may mask differences in effectiveness among subsets of countries. Table 5 breaks down findings by the measure of aid used. The findings of studies using measures of total gross ODA are nearly evenly divided. All 14 studies using total net ODA, which subtracts repayment of loan principals from gross ODA to better reflect actual aid inflows, report either mixed findings or no significant  Gebhard et al. (2008), however, argue that "comparing aggregate aid flows to such indicators as economic growth overlooks the specific impacts of aid projects not specifically designed to improve economic growth" (p. 2). Seven studies look at subsets of either gross or net ODA, in attempts to evaluate aid flows against outcomes they are intended to impact. One study looking just at gross primary education disbursements finds that bilateral aid is more effective at increasing primary school enrollment (Christensen et al., 2011). Four studies do not count humanitarian aid, arguing that this type of aid is less likely to target longerterm development outcomes such as GDP growth. Of these, one subtracts humanitarian aid from gross ODA and has mixed findings (Jeffrey, 2015), two subtract it from net ODA and both find multilateral aid more effective (Heady, 2005(Heady, , 2008, and one subtracts both humanitarian aid and technical cooperation from net ODA and finds bilateral aid more effective (Jeanneney and Tapsoba, 2012). Two studies use net aid transfers, which subtract net interest as well as principal repayments from ODA. One finds no difference in aid effectiveness (Wako, 2011), and the other also subtracts technical cooperation and finds bilateral aid more effective (Rodrik, 1995). Twenty-nine studies divide aid by GDP of the recipient country and six divide aid by population to control for differences in how aid volumes affect countries of different sizes, while nine do not normalize their measure of aid. Twenty-three take the average (or in a few instances, sum) of aid and other variables over a certain number of years and test associations in these multi-year periods to attempt to account for the procyclical nature of aid (United Nations Conference on Trade and Development (UNCTAD), 2010) and for delays between aid being received and its effect on the outcome measure of interest. Twenty-four studies use lagged measures Notes: Askarov and Doucouliagos' (2013) meta-analysis is not included in this table. The authors find that multilateral aid is more effective. Rodrik (1995) does not report the number of countries in his sample, so is not included in section B. He finds that bilateral aid is more effective.
1438 Pierre E. Biscaye, Travis W. Reynolds, and C. Leigh Anderson of aid and test associations between outcomes in a given year and aid volumes from previous years. This latter approach is also used to reduce the risk of endogeneity bias from the possibility that changes in outcome variables may also cause changes in aid flows (such as an increase in poverty leading to an increase in aid, or more aid going to countries with a lower GDP per capita), under the logic that lagged aid is exogenous to future outcomes (Askarov and Doucouliagos, 2015a). We do not observe any clear relationship between any of these manipulations of the aid measure and study findings (Table 5, Sections B-D). Fourteen studies take an average of aid over multiple years and also include lagged measures of aid in their models-two find that bilateral aid is more effective, four find multilateral more effective, four report mixed findings and four report no significant difference. While these studies have the advantage of smoothing out fluctuations in aid and may avoid some endogeneity bias, these specifications do not lead to any further consensus on the relative effectiveness of bilateral and multilateral aid.
The reviewed studies use a variety of econometric models to evaluate aid effectiveness. We focused on results from the primary or most robust models, as identified by the study authors. The most common approach (17 studies) is ordinary least squares (OLS) regression, often with fixed effects controlling for aid recipient country and year or time period. Many studies use instrumental variable (IV) models, including two-stage or three-stage least squares (four studies) and the generalized method of moments (GMM-11 studies), in attempts to control for endogeneity in the relationship between aid flows and outcome measures. Other BILATERAL AND MULTILATERAL AID EFFECTIVENESS 1439 econometric models include quantile regression, autoregressive distributed lag, cointegrated vector autoregressive models, vector error correction models, feasible generalized least squares, hierarchical linear modeling and seemingly unrelated regressions estimation (see Table 2). While each study estimates models differently (e.g. varying in the choice of instruments or control variables), for comparison we grouped together studies using OLS, studies using IV models other than GMM, studies using GMM and studies using other types of models (Table 6). We find no clear pattern in the results of these studies, though a larger share of studies using OLS finds multilateral aid to be more effective and studies using GMM are more likely to report mixed findings or no significant difference.
Many studies within a given outcome area base their models on seminal studies and use a similar set of control variables and approaches to instrumentation. To examine the possible relationship between model specification and study findings, we reviewed five studies that use a similar basic analytical model, first presented by Burnside and Dollar (2000). The Burnside and Dollar model includes time and country fixed effects and regresses per capita real GDP growth on logged initial real per capita GDP, aid receipts relative to GDP, an "economic policy index," 10 the interaction of aid receipts and the economic policy index, and other exogenous variables that might affect growth and aid. 11 Burnside and Dollar (2000) look at a sample of 56 developing countries from 1970 to 1993 using 4-year averages for all variables. To account for endogeneity in aid flows, they use two-stage least squares (2SLS) to construct an estimate of aid using various instruments. They find that bilateral aid has a significant positive impact on government consumption, while the impact of multilateral aid is not significant, but they do not separately evaluate the association between bilateral and multilateral aid and GDP growth.
Many other studies reference this model and five studies in our review build directly on the Burnside and Dollar approach. Ram (2003) uses the same model and sample as Burnside and Dollar (2000), but separates out bilateral and multilateral aid without interacting these with the economic policy index. He also uses simple OLS rather than 2SLS, noting that Burnside and Dollar did not find evidence of significant endogeneity between aid and growth. Ram (2003) concludes that multilateral aid has a large and significant negative impact on GDP growth and bilateral aid has a large and significant positive impact. The same author later conducted another study (Ram, 2004) using the same sample and model as Ram (2003) but interacting aid with the policy index and ultimately reporting similar results. 1440 Pierre E. Biscaye, Travis W. Reynolds, and C. Leigh Anderson Headey (2005) starts from the Burnside and Dollar model and sample but adds data from 1993 to 2001, and does not interact aid with policy variables. He argues that the use of "strategy" instruments as in Burnside and Dollar (2000) and other studies cannot explain multilateral aid flows and that IV approaches can also induce multicollinearity between aid and other explanatory variables included both as instruments and as exogenous variables. He addresses the issue of endogeneity by using lagged measures of aid inflows excluding humanitarian aid. He finds that lagged measures of both aid channels have a significant and positive impact on GDP growth, and that multilateral aid has roughly twice the effect of bilateral aid. When he controls for political and strategic influences for bilateral aid using a motivation index, he finds that bilateral and multilateral aid flows have similar mean effectiveness levels. In a subsequent paper using the same model and sample, Headey (2008) finds that while multilateral aid has a positive and significant impact on GDP growth both during and after the Cold War, the effect of bilateral aid is only positive and significant after the Cold War, which he attributes to a reduction in the geopolitical orientation of bilateral aid.
Jeffrey (2015) starts from the Burnside and Dollar model but directly interacts the measures of economic policy and a measure of governance quality with bilateral and multilateral aid, rather than constructing a policy index. The author uses a larger sample of 120 developing countries from 1974 to 2009 and uses OLS. The relationships between the uninteracted aid variables and GDP growth are not significant in the full sample, but multilateral aid has a significant and positive impact in a subsample of 53 low-income countries, while bilateral aid has a negative but not significant impact. In addition, the author reports that a good policy environment spurs bilateral aid's effectiveness but hurts multilateral aid's effectiveness, though the bilateral aid interaction terms are only significant in the full sample and the multilateral aid interaction terms are only significant in the subsample.
These studies illustrate how model specification choices influence study findings, even among studies using similar models and samples. The choice of control variables, interaction terms and approaches to addressing endogeneity can significantly affect results when estimating impacts of aid on something as complicated as GDP growth, for example. A study testing the sensitivity of findings to various model specifications could further shed light on how these decisions may influence aid effectiveness results.

Discussion and Conclusion
In keeping with the disagreements in the theoretical literature, we find no empirical consensus on the relative effectiveness of bilateral vs multilateral aid in any particular outcome areas. Authors that hypothesize that a particular channel will be more effective are more likely to report findings in line with their hypothesis, but this is not always the case.
While this lack of consensus appears to hold across most aspects of study methodologies, our results indicate that the relative effectiveness of aid channels may vary across countries. We find, for example, that studies looking at fewer countries are more likely to conclude that bilateral aid is more effective. This finding may be driven in part by the four studies in our sample with only five or fewer countries included and where the vast majority of aid comes from bilateral sources-a fact that the authors acknowledge makes a significant finding for BILATERAL AND MULTILATERAL AID EFFECTIVENESS 1441 multilateral aid effectiveness less likely. Other patterns across studies suggest potentially meaningful differences in aid effectiveness by region. For example, none of the seven studies that look exclusively at sub-Saharan Africa find bilateral aid to be more effective. Thus, the sample of countries studied may influence study results. Similarly, though we find no association between the number of years of data analyzed and study findings, four studies report that looking at subsets of years changed their findings, indicating variation in the relative effectiveness of aid channels over time.
In addition, it is possible that looking at the impact of total aid flows on a single outcome measure may mask important variation in effectiveness across different subsets of aid flows. The variety of findings among the seven studies analyzing subsets of gross or net aid suggests that the relative effectiveness of bilateral and multilateral aid may vary depending on the specific outcomes targeted by aid flows, so looking at the impact. While studies may be limited by issues of data availability and quality (Addison et al., 2015), future studies that analyze the relationship between sector-specific aid and sector-specific outcomes, as in Basnett et al.'s (2012) study of the effectiveness of aid for trade, may provide a better picture of when and if bilateral or multilateral aid might be more effective (Clemens et al., 2012;Gebhard et al., 2008).
Another factor not considered in this paper that might influence findings on the relative effectiveness of bilateral vs multilateral donors is differences among individual donors themselves. Studies of aid efficiency, transparency and coordination with recipient countries reveal a wide range of performance by individual multilateral and bilateral donor organizations. Our analysis reviews findings for total aid delivered bilaterally and multilaterally, thus likely masking variations in effectiveness among donors. A few of the studies we reviewed do also separately analyze the effectiveness of particular donors (Kosack, 2003;Minoiu andReddy, 2007, 2010;Okada and Samreth, 2012;Ratha, 2001;Rodrik, 1995;Wamboye et al., 2013), but look at only a few donors. Future studies could seek to evaluate more systematically and empirically and compare individual donors or conduct a meta-analysis of existing donor aid effectiveness studies, though such analyses would likely be limited by the availability of comparable data and by challenges in causally connecting aid and activities of particular donors to country-level outcomes. Such studies might follow DFID's approach to reviewing multilateral aid "value for money" (DFID, 2011) by using quantitative data as much as possible but supplementing them with qualitative data from surveys and interviews in recipient countries, external evaluations and organizational reporting.
Ultimately, we cannot draw strong conclusions about the relative overall effectiveness of bilateral versus multilateral aid at improving recipient country development outcomes. What we can conclude is that given renewed international attention to the sustainable development goals, if more efficient resource allocation is a priority, there is now an opportunity to respond to some of the questions about data and methodology raised in the few cross-country studies of aid effectiveness undertaken to date. Our findings suggest that there is variation in, and therefore value in accounting for, the effectiveness of bilateral and multilateral aid across countries and regions and across time periods. While the body of evidence reviewed here does not consider the relative cost-effectiveness of these aid channels, the different transaction costs they impose, or how they are perceived within donor countries, further research connecting aid flows empirically to development 1442 Pierre E. Biscaye, Travis W. Reynolds, and C. Leigh Anderson outcomes could be a useful complement to the DFID and OECD reviews of aid organizations that attempt to answer some of these other questions.