Unraveling the dimensions of supplier involvement and their effects on NPD performance: a meta-analysis

We study the relationship between supplier involvement in New Product Development and performance. The current literature is scattered and fragmented with studies reporting mixed empirical evidence for a variety of concepts related to ‘Early Supplier Involvement’. We conduct a systematic review and meta-analysis of the existing literature to reconcile conflicted findings, revise and refine theoretical perspectives, and provide evidence-based scholarly and practical implications. To achieve these aims, we unravel the general relationship by considering three factors. First, we delineate different types of performance outcomes, mainly related to NPD efficiency (e.g., speed) and NPD effectiveness (e.g., product quality). Second, we distinguish between the moment and the extent of supplier involvement, related to different theoretical perspectives on external knowledge integration. Third, we disentangle multiple levels of analysis that are seemingly obscured in the literature, specifically the project and organizational levels. We find that extensive supplier involvement has positive effects on NPD efficiency and effectiveness, whereas earlier supplier involvement only to some degree affects NPD efficiency and not effectiveness. In conclusion, our meta-analysis based on 11,420 observations from 51 studies provides strong theoretical and practical insights on the important phenomenon of supplier involvement.


INTRODUCTION
Developing new products has increasingly become an inter-organizational activity, with focal firms seeking collaboration with external sources of knowledge, such as suppliers, to enhance their knowledge base and extend their development capabilities (Hoegl and Wagner 2005;Johnsen 2009; Un, Cuervo-Cazurra, and Asakawa 2010).For example, automotive companies have employed their first-tier suppliers to develop parts and components for new car models (Clark 1989;Clark and Fujimoto 1991;Johnsen 2009;Jacobides, MacDuffie, and Tae 2016).
More recently, Boeing started a collaboration with car seat manufacturer Adient to develop and manufacture seats to cut delays in aircraft delivery times (Hepher 2018).This practice of integrating upstream supply chain partners in product development has become known as 'Early Supplier Involvement': the participation of suppliers in their customer's new product development (NPD) projects (Handfield et al. 1999;Monczka et al. 2000).The overall purpose of this paper is to examine the impact of supplier involvement in New Product Development on (NPD) performance.
While supplier involvement is generally believed to be beneficial for achieving better new products faster, prior researchand empirical evidence in particularis fragmented and scattered.Contrary to popular belief, there is as of yet no "overwhelming evidence" to support the positive effects of supplier involvement on New Product Development (cf.Johnsen 2009, 193).In particular, research employs a divergent and inconsistent terminology and shows mixed and heterogeneous results (Eisenhardt and Tabrizi 1995;Hartley, Zirger, and Kamath 1997;Koufteros, Cheng, and Lai 2007;White et al. 2008;Yan and Dooley 2013).The lack of consensus in the literature warrants a structured review and meta-analysis of the prior empirical literature on the relationship between supplier involvement and NPD performance.In conducting such a review, we consider three factors.
First, almost all of the early literature on supplier involvement investigated the impacts on lead time, speed, time-to-market, or development costs, i.e.NPD efficiency (Imai, Nonaka, and Takeushi 1985;Clark 1989;Womack, Jones, and Roos 1990;Clark and Fujimoto 1991;cf. Johnsen 2009: Tables 1 and 2).However, subsequent research has included outcomes related to the newly developed product itself, including quality, product target cost, and manufacturability, i.e.NPD effectiveness (Swink 1999;Takeishi 2002;Hoegl and Wagner 2005;Van Echtelt et al. 2008).NPD efficiency and NPD effectiveness are two very different outcomes with likely tradeoffs (Langerak and Hultink 2006) but prior studies have not adequately recognized this, nor theorized distinct paths to these outcomes.We aim to unravel the relationship between supplier involvement and performance by clearly distinguishing different (NPD) performance outcomes.
Second, many different definitions of supplier involvement exist, with the majority of studies referring to aspects related to the earliness of involvement (moment, timing, cf.Bidault, Despres, and Butler 1998b;LaBahn and Krapfel 2000;Wynstra and Ten Pierick 2000;Parker, Zsidisin, and Ragatz 2008) or to aspects related to the extent of involvement (supplier development responsibility, design integration, cf.Clark 1989;Koufteros, Cheng, and Lai 2007;Parker, Zsidisin, and Ragatz 2008;Wynstra et al. 2012).While all these different studies have previously been reviewed under the general heading of 'Early Supplier Involvement' (Johnsen 2009), they represent theoretically distinct and practically disparate approaches to integrating supplier knowledge in the product development process (Lichtenthaler and Lichtenthaler 2009), as we will review in depth below.Therefore, we also aim to unravel the relationship between supplier involvement and performance by providing a conceptualization and analysis of the distinct nature of these two dimensions of involvement and their effects on performance.
A third and final issue in synthesizing prior research pertains to differences between levels of analysis that so far are seldom explicitly acknowledged.In particular, while the early literature focused almost exclusively on the contribution of suppliers in the context of a single NPD project, some of the recent literature has examined the effects of organizational-level supplier involvement practices on overall firm performance (e.g., Koufteros, Cheng, and Lai 2007;Wu and Ragatz 2010;Perols, Zimmermann, and Kortmann 2013).Therefore, as a third means to rebuild consensus on the relationship between supplier involvement and performance, we aim to unravel the relationship between supplier involvement and performance by clearly distinguishing between the project and organizational levels of analysis.
To achieve these aims, this paper presents a structured literature review and meta-analysis of the supplier involvement literature.In order to regain a fundamental understanding of the literature, such a review must be conducted at a somewhat more abstract level than individual studies are able to achieve.By elaborating a parsimonious model and empirically analyzing the existing literature we aim to inspire and guide future research in the field (Leuschner, Rogers, and Charvet 2013;Durach, Kembro, and Wieland 2017).We also seek to provide more reliable, evidence-based managerial advice (Rousseau, Manning, and Denyer 2008) that goes beyond the adagio 'the earlier, the better', by focusing on to what extent and when suppliers should be involved in new product development (Brown and Eisenhardt 1995;Primo and Amundson 2002;Johnsen 2009).Finally, we compare the effects of supplier involvement and customer involvement (in the Discussion) to evaluate the effectiveness of different approaches to collaboration in NPD (Chang and Taylor 2016).
A BRIEF LITERATURE REVIEW Johnsen (2009, p. 193) sketches the historical development of research on supplier involvement and concludes that there is "overwhelming evidence to support early and extensive supplier involvement as a key explanatory factor of superior new product performance".We conjecture that a closer inspection of prior research on supplier involvement, as reported below, does not show consensus and employs such a divergence of definitions that the broader picture is obscured.We describe, in turn, the historical development of the field, the unit of analysis and performance outcomes, and the theoretical underpinnings, before developing our hypotheses based on these considerations.
Research into supplier involvement was initially sparked by the observation that Japanese automotive companies outperform their Western counterparts in time-to-market and development cost due to extensive supplier participation in NPD (Clark 1989;Clark and Fujimoto 1991;Iansiti and Clark 1994).Subsequent research has led to a wide-ranging literature on supplier involvement (Birou and Fawcett 1994;Liker et al. 1996;Wasti and Liker 1997; and more recently: White et al. 2008;Yan and Kull 2015) establishing the term Early Supplier Involvement to refer to a set of approaches to solicit the active participation of suppliers during product development (Handfield et al. 1999).
However, this literature does not provide overwhelming support for the positive effects of supplier involvement.Many early studies indeed showed positive effects of supplier involvement on new product development performance (Imai, Nonaka, and Takeushi 1985;Takeuchi and Nonaka 1986;Clark and Fujimoto 1991).However, subsequent research has not only failed to confirm positive returns, but has also reported disadvantages and negative effects on NPD performance (Eisenhardt and Tabrizi 1995;Callahan and Moretton 2001;White et al. 2008;Yeniyurt, Henke, and Yalcinkaya 2013;Tavani, Sharifi, and Ismail 2014).For example, Eisenhardt and Tabrizi (1995) found an overall negative correlation between supplier involvement and development speed, with a positive effect only in very mature segments of the electronics industry.Other research in the field reported non-significant findings with effects (very close) to zero (Hoegl and Wagner 2005;Cruz-González, López-Sáez, and Navas-López 2015;Yan and Kull 2015).This short recap of the state-of-the-art of the literature gives rise to a pressing concern that the overall body of research paints a blurry picture of supplier involvement.Figure 1 includes exemplary research for both early and later research that provides negative findings (left side), null and insignificant findings (middle), and overall positive findings (right).This figure shows that the literature has reported contradictory outcomes of supplier involvement and shows that there is no consensus of the effects of supplier involvement.Note that only a selection of (seminal) research has been included in the figure; this initial observation of heterogeneous effects inspired our full meta-analysis. - The mixed nature of the empirical results has been acknowledged in prior research also as a primary reason to conduct their study (e.g., Primo and Amundson 2002;Hoegl and Wagner 2005), but even that has not helped to converge the scattered literature.We posit that a systematic review of the literature can help to rebuild consensus in the field by clarifying inconsistent usage of definitions and explain seemingly contradictory findings due to differences in theory-informed conceptualizations and research designs across studies (Durach, Kembro, and Wieland 2017).
Supplier involvement, as well as other NPD process characteristics, can lead to multiple types of performance outcomes.We can distinguish between performance outcomes of the NPD project related to the development process (efficiency) and the developed product (effectiveness) (Brown and Eisenhardt 1995).NPD efficiency can be defined as the adherence to project targets and the use of fewer project resources such as financial resources and time (Hoegl and Wagner 2005).NPD effectiveness refers to the resulting product's quality and performance in the market (Olson, Walker, and Ruekert 1995;Hoegl and Wagner 2005).
Distinguishing between these two performance outcomes serves two aims.First, it allows us to clearly observe that most of the early literature on supplier involvement focused exclusively on explaining differences in efficiency, such as time-to-market (e.g., Imai, Nonaka, and Takeushi 1985;Clark 1989), while only the more recent literature has also included elements of effectiveness, such as product quality (e.g., Hoegl and Wagner 2005;Van Echtelt et al. 2008).Still, the effects of supplier involvement on these different NPD outcomes are usually not theorized separately (Hoegl and Wagner 2005;Johnsen 2009).Second, the distinction helps to acknowledge that managers of NPD projects may not be able to achieve both efficiency and effectiveness at the same levels, due to potential tradeoffs between reaching these two goals (Langerak and Hultink 2006).

HYPOTHESIS DEVELOPMENT
In this research, we draw upon three related streams of literature: open innovation and absorptive capacity, (organizational) knowledge integration, and the capability view.In an inter-organizational context, firms transact knowledge with partnerssuch as suppliersto extend their own knowledge bases (Gulati 1999), including know-how and (technical) information (Kogut and Zander 1992).In order to integrate supplier knowledge into the product development process effectively and efficiently, the focal firm needs to rely on external knowledge integration capabilities, in particular on absorptive and connective capacities (Cohen and Levinthal 1990;Kogut and Zander 1992;Lichtenthaler and Lichtenthaler 2009).
On the one hand, absorptive capacity allows the focal firm to explore and gather ideas and concepts for new products (Cohen and Levinthal 1990) and to engage in inbound open innovation for integrating external knowledge (West and Bogers 2014).On the other hand, firms can effectively maintain knowledge outside of the firm, without acquiring it, by working with alliance partners (Grant and Baden-Fuller 2004), which constitutes a connective or combinative capacity for integrating external knowledge (Kogut and Zander 1992;Lichtenthaler and Lichtenthaler 2009).

The Extent of Supplier Involvement
Connective capacities for accessing external knowledge (Kogut and Zander 1992;Lichtenthaler and Lichtenthaler 2009) in product development can be leveraged by buyers who delegate development responsibility to their suppliers.The extent of supplier involvement refers to the division of labor and tasks between the buyer and suppliers as measured by supplier design responsibility (Clark 1989;Hartley, Zirger, and Kamath 1997;Van Echtelt et al. 2008;Azadegan and Dooley 2010;Wynstra et al. 2012).With suppliers already performing a majority of the manufacturing of components and sub-systems for most original equipment, they are highly capable and knowledgeable, in particular in the details of component design (Takeishi 2002).If suppliers then also perform labor during component development on behalf of the buying firm (i.e.concurrent engineering), the buyer's expenditures in man-hours, cost, and time can be decreased (Clark 1989;Eppinger et al. 1994;Iansiti and Clark 1994).This is practiced by many OEMs for products that consists of a range of components, technologies, and (sub-) systems, such as automotive products (e.g., Honda cars), electronics (e.g., ASML chip machines), and mechanical systems (e.g., Caterpillar machinery).Designing component blueprints and defining production requirements involves knowledge at the detailed component level that typically suppliers possess most extensively (LaBahn and Krapfel 2000; Koufteros, Cheng, and Lai 2007).Hence, connecting to this specialized, external knowledge, through delegating design responsibility to suppliers, requires less development and engineering resources (Clark 1989) and enables parallel execution of development and engineering tasks (Eppinger et al. 1994;Gerwin and Barrowman 2002), and is therefore associated with increased NPD efficiency:

H1: A higher extent of supplier involvement is positively related to NPD efficiency.
A number of studies also examines the effects of higher extents of supplier involvement on product quality, market success, and other aspects of NPD effectiveness (Primo and Amundson 2002;Ragatz, Handfield, and Petersen 2002;Hoegl and Wagner 2005;Johnsen 2009).
However, achieving NPD effectiveness through higher extent of supplier involvement is difficult.Involving suppliers extensively in product development may lead to better products to the extent that (component) suppliers are knowledgeable about the overall product concept and architecture (e.g., with strategic suppliers).A faster developed or higher quality component does not improve the overall product quality per se, which requires further integration and resolutions of (new) component interdependencies at the overall product level (Lakemond, Berggren, and van Weele 2006;Hong and Hartley 2011).Therefore, delegating design responsibilities to suppliers may have some positive effect on NPD effectiveness (e.g., product quality), but less so than on NPD efficiency.Prior studies, however, do not differentiate between the mechanisms to achieve either performance outcome.We therefore submit the following hypothesis: H2: a) A higher extent of supplier involvement is positively related to NPD effectiveness, b) but this effect is weaker than its effect on NPD efficiency.

The Moment of Supplier Involvement
The capacity for absorbing external knowledge (Cohen and Levinthal 1990;Lichtenthaler and Lichtenthaler 2009) is affected by the moment of supplier involvement.Specifically, involving suppliers in earlier phases of the product development process allows the buyer to acquire more ideas and concepts from knowledgeable supply chain actors (Dowlatshahi 1998;Parker, Zsidisin, and Ragatz 2008a;Un, Cuervo-Cazurra, and Asakawa 2010), which is a form of inbound open innovation (Lichtenthaler and Lichtenthaler 2009;West and Bogers 2014).Early supplier involvement has a broad connotation and is used to refer to a range of supplier involvement practices (LaBahn 1992;O'Neal 1993;Bidault, Despres, and Butler 1998a;LaBahn and Krapfel 2000;McIvor and Humphreys 2004).The more formal term moment of supplier involvement is typically operationalized as the earliest of the phases of product development in which a supplier is involved (Handfield et al. 1999; see Figure 2).

H3: An earlier moment of supplier involvement is positively related to NPD effectiveness.
If suppliers are involved earlier, technical and manufacturability issues can be discovered sooner, which makes them easier to fix (Swink 1999).Early discovery of potential problems with product concepts or their technical execution potentially prevents late-hence costly and difficult-changes to the product specifications or delays in operations ramp-up (Swink 1999;Brettel et al. 2011).However, several studies show that early supplier involvement has negative effects on NPD efficiency (Eisenhardt and Tabrizi 1995;Laseter and Ramdas 2002;Koufteros, Rawski, and Rupak 2010;Yan and Kull 2015).Involving suppliers early to discuss new product ideas and concepts requires effective knowledge sharing, is costly to manage, and slows down the overall progress of the project (Hartley, Zirger, and Kamath 1997;Wynstra et al. 2012).On balance, we posit that early supplier involvement will have some positive effect on NPD efficiency, but less so than on NPD effectiveness: H4: a) An earlier moment of supplier involvement is positively related to NPD efficiency but b) this effect is weaker than its effect on NPD effectiveness.
Before introducing the third distinction that helps to unravel this literature, namely between project-level versus organizational-level integration of supplier knowledge, we briefly discuss the relationship between the two dimensions of supplier involvement as discussed until now.

Interrelationship between Extent and Moment of Supplier Involvement
In order to explore the dimensionality of supplier involvement, it is important to also understand how the extent and moment of supplier involvement are related.Only a few studies explicitly study both the extent and the moment of supplier involvement (Hartley, Zirger, and Kamath 1997;Tracey 2004;Cousins and Lawson 2007;Lai et al. 2011).In these studies, the two dimensions are treated as essentially unrelated independent variables.A handful of other studies furthermore analyze how extent and moment are related (Lin 2009;Koufteros, Rawski, and Rupak 2010;Lau, Tang, and Yam 2010;Lau 2014), but the causal direction is ambiguous at best.Some conceptual studies have argued that the timing of a supplier's involvement should be based on the level of design responsibility it receives (Bidault, Despres, and Butler 1998b;Monczka et al. 2000).Based on this discussion, we expect that there will be some positive interrelationship between the extent and the moment of supplier involvement (r ≠ 0), but that this relationship will not be perfect (r < 1).This means that managing supplier involvement requires two separate decisions for the extent and the moment (or timing) of supplier involvement, which can be interrelated to some degree (Wynstra and Ten Pierick 2000;Lakemond, Berggren, and van Weele 2006).Our basis for delineating the dimensions of supplier involvement would be either meaningless if the dimensions are completely distinct (if r = 0) or redundant if they are completely the same (if r = 1).There is no sufficient empirical nor theoretical basis to explicate this as a hypothesis in this study, but we do explore this issue using the meta-analytical approach.

Level of Analysis: Project v. Organization
Historically, the literature on early and extensive supplier involvement, in general, has investigated involvement in a single NPD project (cf.Clark 1989;Liker et al. 1996;Hartley, Zirger, and Kamath 1997), where the interest is in the structure and process of developing a particular product (Brown and Eisenhardt 1995, 343).In other words, the development project is the unit of analysis in most of the literature on supplier involvement.Our hypotheses for the project level have already been posited in H1-H4.
There is also a collection of (relatively recent) literature that investigates supplier involvement as a general organizational practice, e.g., how the integration of suppliers in innovation processes affects a firm's ability to bring products faster to market than competitors (Perols, Zimmermann, and Kortmann 2013).In other words, these studies conceptualize both supplier involvement and performance at the organizational level (Koufteros, Rawski, and Rupak 2010;Un, Cuervo-Cazurra, and Asakawa 2010;Yeniyurt, Henke, and Yalcinkaya 2013).These studies focus on knowledge integration more generally rather than supplier involvement alone, but provide meaningful insights for our current inquiry as well (Un, Cuervo-Cazurra, and Asakawa 2010; Cruz-González, López-Sáez, and Navas-López 2015).For example, this takes the shape of relating organizational-level involvement practices to the capability to develop products that are unique (product innovation) or reliable (product quality) compared to industry averages (Koufteros, Vonderembse, and Jayaram 2005;Koufteros, Cheng, and Lai 2007;Perols, Zimmermann, and Kortmann 2013).
At the project level, in H1-H4, we distinguished between different performance outcomes and dimensions of involvement, but these are not adequate nor empirically addressed at the organizational level.Focal firm performance can be improved either if openness to external knowledge positively affects innovation capabilities (West and Bogers 2014) or if these supplier involvement practices are effectuated in (a series of) NPD projects that result in superior product performance on the long run.Given the relatively abstract level of operationalization in these studies, we can only expect a general relationship between supplier involvement practices and focal firm performance (cf.Durach, Kembro, and Wieland 2017).
We can then also use the test of this hypothesis to compare the findings against the projectlevel relationships posited before in H1-4.

Conceptual Model
Based on the hypotheses introduced above, we can now derive the following conceptual model, Figure 3.At the project level, our main hypotheses can be summarized by a two-by-two matrix, involving two dimensions of supplier involvement (SI) and two types of NPD performance outcomes.Note that parts b of Hypotheses 2 and 4 are not depicted in the model and posit a quantitative difference in the size of the observed correlation between the primary and secondary effects of supplier involvement on NPD efficiency and effectiveness.At the organizational level, SI practices are related to firm performance.

METHODS
In this section, we describe first the selection of studies, second the data extraction and coding, and finally the data analysis.The online supplement contains detailed information on each of these steps as well as a list of included studies and their characteristics (Online Supplement S1).

Study Selection
The procedure to identify and then select relevant empirical research is visualized in Figure 4.
- ---------------------------------Insert Figure 4 Approximately Here-----------------------------------Relevant articles were identified with a search in six electronic databases, using combinations of key words.Additional articles were identified from a purchasing and supply management literature review (Wynstra, Suurmond, and Nullmeier 2019) as well as a snowballing approach to track down (unpublished) papers using reference lists and author contacts.Combined, these two sources provided 793 unique hits, which were scanned based on titles and abstracts for an initial filtering of irrelevant and qualitative research.The remaining 273 articles was examined in full and another 188 articles were excluded from our set (reasons listed in Figure 4).To ensure independence between included studies, several articles were excluded while retaining the original or most exhaustive source (e.g., original source: Yan 2011; follow up publications: Yan and Dooley 2014; Yan and Kull 2015).Finally, 51 studies representing 53 independent samples with effects of supplier involvement on performance were included in this metaanalysis.

Data Extraction and Coding
From each study, we extracted effect sizes and sample sizes (typically correlation coefficients; Lipsey and Wilson 2001;Carney et al. 2011).When zero-order correlation coefficients were not available, we transformed data from regression models into partial correlation coefficients (Carney et al. 2011;Aloe 2014;Suurmond, van Rhee, and Hak 2017).We coded the two sides of the hypothesized relationships, relating each effect to one specific dimension of supplier involvement and one type of NPD outcome.Two coders independently coded each relevant construct, from each study, using a 75 percent content validity threshold (Hunter and Schmidt 2004;Zimmermann and Foerstl 2014) checked against a-priori definitions (see Table 1part A) and achieved adequate initial interrater agreement (79%).We also extracted additional information related to study characteristics and methodology (see Table 1 part B).Secondary data on Hofstede's dimensions of national culture were collected (Hofstede, Hofstede, and Minkov 1997) and linked to a study's country of data collection.These study-level characteristics were used as moderators in a meta-regression (similar to e.g., Storey et al. 2016;Weiss, Hoegl, and Gibbert 2017).

Data Analysis
We conduct our analyses using a mixed effects model, which accounts for random effects (heterogeneity) and multiple levels (dependency of multiple effects from single study) (Hedges and Olkin 1985).We employ Fisher's r-to-z transformation (and back) to ensure accurate results (Fisher 1921).When multiple effect sizes are available from a single sample, the interdependency between these effects are modelled in specifying the mixed effects models using random coefficients (Viechtbauer 2010;Cheung 2019), and where applicable, weightedleast squares (WLS) regression (Lipsey and Wilson 2001;Geyskens et al. 2009).We use R as the computational back-end (Viechtbauer 2010;Wallace et al. 2012).R-code as well as access to the full data are provided on the Open Science Framework (10.17605/OSF.IO/3VP75).

Publication Bias
We performed publication bias analysis to assess threats to the validity of our results caused by the underreporting of statistically insignificant findings (Rothstein, Sutton, and Borenstein 2006).We conducted an 'Egger-style' regression by including the sample size as a predictor in a meta-regression model (Egger et al. 1997).This approach accounts for the multilevel structure of the data and the heterogeneity of the effect sizes, in contrast to some more familiar methods such as a Failsafe number (for the number of unpublished studies averaging nullresults which are required to reduce the overall effect to a statistically non-significant finding) (Rosenthal 1979).The results of the regression show that effect size is not predicted by sample size (β = -0.0002,p = 0.1896) and that publication bias is not a major threat to our findings.
We additionally examined the funnel plots for asymmetric distributions of effects sizes and found no evidence of publication bias (see Online Supplement S1).

RESULTS
In this meta-analysis, we study the effects of supplier involvement on performance.We conduct random-effects meta-regression and meta-analytic subgroup analysis on a total set of 53 samples representing 11,420 observations, see Table 2.The weighted average (or metaanalytic) correlation (r) between supplier involvement and performance is shown in the first row, pooled for all observations regardless of dimension of involvement and level or type of performance outcome.
Based on the total set of observed effects in the first row, we find general support for supplier involvement: there is a positive relationship between supplier involvement and performance: r=0.189.However, the results are also heterogeneous, as evidenced by the significantly large Q and the wide-ranging credibility (or prediction) interval, in Table 2. Given this mixed nature of the findings, further breaking down the effects into subgroups to test specific hypotheses (as in Table 2) and further exploring this variance using meta-regression (as in Table 3) is warranted.

Main results
Project Level.Our findings show that the extent of supplier involvement is positively related to NPD efficiency, in support of Hypothesis 1 (see Table 2).This means that projects in which a larger share of the development responsibilities is delegated to suppliers, exhibit higher efficiency, such as improved project speed.
The extent of supplier involvement is also positively related to NPD effectiveness, in support of Hypothesis 2a.Projects with larger shares of suppliers' responsibilities for development tend to result in better products, such as higher product quality.H2b furthermore posited a quantitative difference in the size of the effects of extent of supplier involvement, and while the difference is in the expected direction (H1>H2a), it is not statistically significant (see superscript a in Table 2: Δr = -0.017,p = 0.597).
Surprisingly, the results do not support Hypothesis 3: the relationship between the moment of supplier involvement and NPD effectiveness is not statistically significant (p > 0.10), the 95% confidence interval of its effect thus overlaps with zero, and the effect size is very small (r < 0.10).In other words, projects in which suppliers are involved earlier do not achieve significantly higher NPD effectiveness.
The results provide support for Hypothesis 4a: there is a positive relationship between the moment of supplier involvement and NPD efficiency, but still the effect is small and the 95% confidence interval is very close to zero.H4b furthermore posited a quantitative difference in the size of the effects of earlier involvement on NPD effectiveness and efficiency, respectively, but the result is in the opposite direction (H3<H4a) and not statically significant (see superscript b in Table 2: Δr = 0.028, p = 0.683).

Dimensionality of supplier involvement.
We also collected data from three studies that report results for both dimensions of supplier involvement and additionally also include data for the interrelationship between the two dimensions (Lin 2009;Koufteros, Rawski, and Rupak 2010;Lau 2014; note that k=3 and N=553).These studies, in summary, find that the two dimensions of involvement are positively related (r=0.415, p<0.001, 95%CI: 0.34-0.48).As we expected, the two dimensions are positively but not perfectly correlated (0<r>1), in other words, they are distinct approaches to supplier involvement that can be managed and decided upon separately.
Organizational level.We also find support for Hypothesis 5: there is a positive relationship between supplier involvement as a general organizational practice and focal firm performance.The results for the organizational and project level are highly similar (r=0.200vs r=0.178) and the difference is not significant (see superscript c in Table 2: Δr = -0.011,p = 0.829).These results show that organizational practices to integrate supplier knowledge in innovation have a positive effect on firm performance, similar in size to specific dimensions of involvement as effectuated in a single project.

Meta-regression analysis
The results indicate that there is substantial variation in the distribution of effect sizes, indicated by the high and significant values of 'Q' in Table 2.In a further analysis, presented in Table 3, we conducted a meta-regression to explain why effects vary across studies by invoking moderators.In this analysis, only study-level characteristics that vary from sample to sample can be included (such as publication status) and project-level characteristics that vary within samples cannot be included (such as product innovativeness).In these models, the intercept represents the average correlation coefficients with all moderators at their baseline.If the moderator's regression coefficient is significant and large, there is evidence that the effect of supplier involvement on performance increases or decreases with the level of the moderator (compared to the baseline).Note that the moderator's regression coefficient indicates the change in correlation coefficient rather than an absolute level of the correlation coefficient (which can be found in Table 2 for some of the moderator levels).
For the subset of effects at the NPD project level in the second column-akin to the second row of Table 2-we find, in addition, that effect sizes from data gathered for the primary purpose of that study are generally smaller.In other words, large collaborative research efforts with more general aims such as HPM 3 (Mishra and Shah 2009;Salvador and Villena 2013) report larger effects for supplier involvement.Similar to organizational-level studies as tested in H5, the measures for supplier involvement in such studies are relatively crude.
Finally, in the meta-regression models in Table 3, we find no evidence that industrial (column 3) or cultural (column 4) context moderates the overall positive effects of supplier involvement.
While prior research emphasized the distinct 'Japanese' approach of supplier involvement and related differences with US or European approaches, our meta-analysis does not find support in the data.As these meta-regression models include many variables relative to the number of observations, the significance of some of the other moderators (e.g., multiple data sources) should also not be over-interpreted.

DISCUSSION
The literature on 'Early Supplier Involvement' has been and continues to be a great inspiration for many practitioners to engage suppliers in their innovation and product development projects.However, upon careful examination the available empirical studies paint a highly scattered and inconsistent picture of expected outcomes of supplier involvement.We therefore provide a systematic literature review and meta-analysis to unravel this relationship along three main lines of inquiry.First, we study different performance outcomes associated with supplier involvement as New Product Development efficiency and effectiveness, respectively.Second, we disentangle the general supplier involvement concept into the extent (e.g., supplier design responsibility) and the moment (e.g., timing, phase) of supplier involvement.Third, we separate observations at the (single) project level from those studies with more general organizational approaches to supplier involvement.Our systematic review represents a first step in formulating an evidence-based conclusion (Rousseau, Manning, and Denyer 2008, 476) on supplier involvement that has both theoretical and practical implications.

Theoretical implications
First and foremost, our review highlights that firms should pursue the integration of supplier knowledge in new product development by accessing-rather than acquiring-that external knowledge.By leveraging connective capacities (Kogut and Zander 1992;Lichtenthaler and Lichtenthaler 2009) through the delegation of specific design and development responsibilities to suppliers (cf.Clark 1989;Wynstra et al. 2012), firms can expect higher NPD efficiency (Hypothesis 1) and NPD effectiveness (Hypothesis 2).On the other hand, our analysis shows that absorbing external knowledge (Cohen and Levinthal 1990) from suppliers through early involvement in NPD does not lead to better products (Hypothesis 3;cf. LaBahn and Krapfel 2000;Parker, Zsidisin, and Ragatz 2008b).Still, earlier involvement of suppliers does have a positive correlation with higher NPD efficiency (Hypothesis 4).Our meta-analysis thereby provides strong evidence for the complementary effect of connective capacity for integrating external knowledge, next to the more established absorptive capacity (Lichtenthaler and Lichtenthaler 2009;West and Bogers 2014).While both capacities are important for innovation, our meta-analysis shows empirically and systematically that accessing knowledge through buyer-supplier relationships can be an important source for competitive advantage (Kogut and Zander 1992;Grant and Baden-Fuller 2004).Previous research on customer involvement provides diametrically opposed results.Chang and Taylor (2016) in their review of customer participation in innovation in Business-to-Consumer contexts, show that consumers contribute significantly to the generation of new ideas and knowledge, but not to the actual efficiency of the development and engineering process (Mishra and Shah 2009;Menguc, Auh, and Yannopoulos 2014).
Secondly, on a more general level, our review provides a comprehensive conceptualization of supplier involvement.We delineate between the extent and the moment of supplier involvement in order to resolve some seeming inconsistencies between previous research findings.A further analysis shows that the two dimensions of involvement are different (Bidault, Despres, and Butler 1998b;Monczka et al. 2000) and that items capturing these latent constructs should not be mixed up (Anderson and Gerbing 1988).This requires future research to carefully distinguish between various practices associated with 'early supplier involvement' and to provide distinct theorization for the aspect of the phenomenon under investigation, which has been lacking to date (Hoegl and Wagner 2005;Johnsen 2009).
Thirdly, our review provides general support for the positive relationship between supplier involvement and performance, across different levels of operationalization.Our analysis shows that firms that report using suppliers as a source of innovation, more generically across projects and organizational units, tend to perform better (Hypothesis 5).However, at this organizational level, the literature lacks a systematic terminology and theorization with scattered findings as a result (Spina, Verganti, and Zotteri 2002;Johnsen 2009;Cruz-González, López-Sáez, and Navas-López 2015).These studies also typically operationalize involvement or collaboration using crude binary measures for suppliers as a source of innovation and we are weary to interpret these findings as causal evidence.
We also examined whether the heterogeneity in effects of supplier involvement can be explained by research designs, industry, or national culture, using meta-regression analysis (Durach, Kembro, and Wieland 2017).Our results show that the effect of supplier involvement does not vary with industrial setting or national culture.Previous research also shows small and mostly insignificant moderation by national culture for the relation between exploitative innovation and firm performance (Mueller, Rosenbusch, and Bausch 2013), which is similar to the typical context of incremental and 'next generation product' innovation investigated in research on supplier involvement (but see Song and Di Benedetto 2008 for involvement in radical innovation).We do not find strong evidence that other research design characteristics influence the general relationship, except that articles from top-ranked journals tend to report somewhat smaller effects of supplier involvement.
In conclusion, our review of the general relationship between supplier involvement and performance provides a simple yet parsimonious understanding based on the distinctions between concepts, levels of analysis, and research designs (Durach, Kembro, and Wieland 2017).This systematic review of the phenomenon thereby identifies science-based conclusions and areas where evidence is contested, which enable the effective use of scientific evidence by scholars and practitioners (Rousseau, Manning, and Denyer 2008).

Managerial implications
To achieve higher NPD performance, managers should consider the division of labor and tasks between their firm and their suppliers (Clark 1989;von Hippel 1990) and appropriately time the involvement of suppliers in their NPD projects (Wynstra and Ten Pierick 2000).There is ample evidence that involving suppliers leads to higher NPD efficiency (speed) and effectiveness (quality), in particular when managers delegate design responsibility to suppliers.
Managers should pursue the integration of specific, component-level supplier knowledge in their projects and organizations generally.Based on our findings, managers should aim to establish buyer-supplier relationships through which they can in particular, access external knowledge during the development of a new product.
Our review also highlights that the benefits of early supplier involvement, as much touted in the academic and business press, are not clear.Earlier involvement as such is not always better and does not lead to higher product quality, financial performance, or product innovativeness.
As the moment and the extent of supplier involvement are also not perfectly correlated, managers can employ these two dimensions to manage a portfolio of involvement approaches (Wynstra and Ten Pierick 2000).
Finally, our results show that the benefits of supplier involvement generalize across various industrial settings and national cultures, even though the practice of supplier involvement may be more widespread or intensively applied in one country versus the other (Clark 1989;Liker et al. 1996;Yan and Kull 2015).This suggests that managers across industrial and national contexts can benefit from appropriately delegating design responsibility to their supply base.

Limitations
In this meta-analysis, the empirical evidence for testing the hypotheses comes from the underlying primary studies.This means that the limitations of these studies also affect the quality and validity of our findings (Malhotra et al. 2014;Bergh et al. 2016).
In particular, concerns can be raised regarding endogeneity and common method bias (Ketokivi and Schroeder 2004;Roberts and Whited 2013;Ketchen, Craighead, and Cheng 2017), as most of the data originates from cross-sectional studies with self-administered questionnaires and a single respondent for each case.Despite these weaknesses, there is theoretical and empirical support to ground the conclusions.In particular, there is a (albeit conceptual) temporal difference between the decision to involve suppliers in NPD and the outcomes of the NPD project, which suggests that causality cannot run in the opposite direction, see again Figure 2.
Omitted variables that correlate to both supplier involvement and performance outcomes, such as supplier capability, could have affected the reported effects (Meade, Behred, and Lance 2009).However, inconsistent reporting of such antecedents across studies prevents us from incorporating them into the model here.Our model including different constructs, levels of analysis, and research designs accounts for the dispersion of effects encountered in this field (Rousseau, Manning, and Denyer 2008;Bergh et al. 2016;Durach, Kembro, and Wieland 2017).
Our review of the empirical evidence has been systematic and aimed to uncover all the literature, irrespective of publication status or journal ranking.As a result, the amount of studies per relationship and our total sample size (see Table 2) is comparable to other recent metaanalyses in the field (e.g., Leuschner, Rogers, and Charvet 2013;Leuschner et al. 2014;Storey et al. 2016;Weiss, Hoegl, and Gibbert 2017).Each of our conclusions is based on results from more than five studies representing more than 900 observations each, which provides appropriate robustness (cf.Leuschner et al. 2014, 26).However, the set of available studies that capture supplier involvement and performance outcomes at the NPD project level is somewhat smaller than in a typical meta-analysis.Therefore, more research is requiredoriginal empirical studies ánd subsequent (updated) meta-analyses-in particular to study the complex relationship between supplier involvement and performance at the project level.

Future research outlook
We provide suggestions for further research on four topics: dimensionality of supplier involvement, managing supplier involvement, contingencies, and empirical contexts (see Table 4).

Dimensionality of Involvement
Future research is required that explicitly incorporates our proposed dimensionality and conducts further empirical testing.In particular, better empirical measures need to be developed in order to test the effect of early supplier involvement.The current, static representation of NPD projects disregards that project phases in reality may be overlapping and recurring (Eisenhardt and Tabrizi 1995;Tatikonda 2008;Potter and Lawson 2013).There is also a need to further interrelate the levels of project and organization outcomes.Research may investigate whether and how, for instance, repeatedly high project efficiency enables organizations to maintain a larger and broader portfolio of NPD projects and thereby possibly increasing market shares.Research could also explore potential negative effects, such as repeated and increasing supplier involvement reducing the internal innovation capabilities of the buying organization.

Managing Involvement
While the focus in this paper is on two design variables regarding supplier involvement, further studies may investigate the subsequent relational and contractual governance of this involvement.Some studies have looked at coordination and communication (Wynstra and Ten Pierick 2000;Lakemond, Berggren, and van Weele 2006), but more can be done to match communication types, intensity, and frequency to different forms of supplier involvement.A related line of research can look into the capabilities of individuals in managing supplier involvement.In particular, traits and characteristics of the project manager, including leadership, will influence the ability of buyers and suppliers to effectively work together (Hülsheger, Anderson, and Salgado 2009;Anderson, Potočnik, and Zhou 2014).Additionally, the vast majority of the studies use a single buyer-supplier relationship within the context of a single project as the unit of analysis.Future research should address the involvement of multiple suppliers, including issues of coordination and control (Wu and Choi 2005;Hong, Pearson, and Carr 2009;Hong and Hartley 2011).

Contingency factors
A third area for future research relates to the contingency factors that may moderate the effects of both designing and managing supplier involvement on performance outcomes.Our focus has been on refining and revising the main effect of supplier involvement on project performance, thereby ignoring some of the initial exploratory findings for specific moderating effects.Our meta-regression provides some preliminary findings for moderators at the study level (Table 3).The most important potential contingency factors operate at the individual project level, while a meta-analysis can only account for differences at the aggregate study levelthe sample of projects in a given study.Prior research has, for instance, studied the effects of supplier involvement in the context of radical innovation and high technological uncertainty, but has found mixed results (Takeishi 2002;Song and Di Benedetto 2008;Johnsen 2009;Menguc, Auh, and Yannopoulos 2014).Future research should simultaneously include a baseline (non-complex; not uncertain context) to investigate the different effects within the same study setting.Finally, our meta-analysis did not find support for a moderating effect of national culture on the relationship between supplier involvement and performance.More research is required using multi-country samples to investigate this in the context of supplier involvement-while simultaneously reporting both aggregate and country-specific results to update future meta-analyses adequately.

Novel empirical contexts
Additional research is needed beyond the industry context of large series assembly operations (Johnsen et al. 2006)

Conclusion
Supplier involvement in new product development has been researched intensely in the past 30 years.In this review, we have summarized, revised, and delineated the relationship between supplier involvement and performance based on prior empirical research.The general support for a positive association highlights the importance of the phenomenon, but additional research is required, as indicated by the (still) large heterogeneity amongst effect sizes and the suggested directions sketched above.T o t a l : 7 9 3 u n i q u e t i t l e s O t h e r s o u r c e s : P S M l i t e r a t u r e r e v i e w S n o w b a l l i n g / a u t h o r s T i t l e / A b s t r a c t E x c l u d e a r t i c l e s : 2 3 3 q u a l i t a t i v e r e s e a r c h 2 8 7 i r r e l e v a n t r e s e a r c h R e m a i n i n g : 2 7 3 a r t i c l e s Using multiple sources of data/respondents mitigates common method bias Some studies ask more general questions, for example 'our suppliers are typically involved heavily in…' Some studies ask the (buyer) respondent to answer for a specific supplier, such as the supplier mostly involved in the project or the (third) largest supplier Some studies collect data from suppliers rather than buyers Study context: The reported correlation(s) between supplier involvement and performance.

FIGURES AND TABLES
Typically the number of respondents or projects analyzed Some studies (9) do not present correlation coefficients but only regression models.Partial correlation formula in footnote 3.
, the typical context of the empirical research on supplier involvement to date.Future research can cover contexts such as Engineer-to-Order (e.g., shipbuilding) or process-based industries (e.g., chemicals), which have different process and product characteristics.Another important context that is virtually absent from the current literature is the area of services.Only two recent studies (Chien and Chen 2010; Hsieh and Tidd 2012) have empirically investigated supplier involvement for service innovation.Hence, our understanding of the effects and mechanisms of supplier involvement in relation to service design and (quality) definition is limited (van der Valk and Rozemeijer 2009; Selviaridis, Spring, and Araujo 2013).
FIGURE 1 Overview of the Scattered Literature

F
M i s s i n g d a t a ( e f f e c t s ) 3 1 : M e t h o d s / Q u a l i t y conducted for this study's purpose Yes / No Some studies report on secondary surveys or data sources, such as the High Performance Manufacturing survey.Data sources:More than one data source (respondent) used to collect data A specific (NPD) project for which the data These characteristics are not always reported.Coded if and only if data collected from single country / industry, 'multiple' otherwise e.g.,China or United States Based upon country, Hofstede's dimensions of culture and Gross Domestic Product per capita were collected from secondary sources.e.g., automotive or electronics