Improving the culture of interdisciplinary collaboration in ecology by expanding measures of success

Authors


Abstract

Interdisciplinary collaboration is essential to understand ecological systems at scales critical to human decision making. Current reward structures are problematic for scientists engaged in interdisciplinary research, particularly early career researchers, because academic culture tends to value only some research outputs, such as primary-authored publications. Here, we present a framework for the costs and benefits of collaboration, with a focus on early career stages, and show how the implementation of novel measures of success can help defray the costs of collaboration. Success measures at team and individual levels include research outputs other than publications, including educational outcomes, dataset creation, outreach products (eg blogs or social media), and the application of scientific results to policy or management activities. Promotion and adoption of new measures of success will require concerted effort by both collaborators and their institutions. Expanded measures should better reflect and reward the important work of both disciplinary and interdisciplinary teams at all career stages, and help sustain and stimulate a collaborative culture within ecology.

Collaboration among individuals and teams is not new within ecology. Ecology is a discipline that has long required interdisciplinary knowledge due to its inherent complexity (Odum and Barrett 1971; Eigenbrode et al. 2007), but interdisciplinarity is becoming increasingly important as the complexity of the ecological problems facing humanity increases (Uriarte et al. 2007; Pennington 2008; Dawson et al. 2011). Interdisciplinary collaboration is “a form of collaboration that combines components of two or more [comparatively self-contained] disciplines” (Nissani 1997; although see Klein [2010] for a more detailed discussion). In practice, interdisciplinary research is almost always collaborative, and may involve many individuals from different disciplines and multiple institutions or nations.

In a nutshell:

  • Interdisciplinary research is an increasingly common form of collaboration and is essential for answering complex environmental questions

  • The costs of interdisciplinary research can be especially high for early career scientists

  • Accepted research success for all collaborative research participants should extend beyond traditional metrics such as primary authorship or project leadership and should include credit for co-authorship, data production, outreach, education, and ongoing mentoring and administrative activities

  • Broader definitions of – and concomitant rewards for – success will more fully acknowledge participation at all career stages and perpetuate interdisciplinary research

Successful collaborative research, whether disciplinary (occurring within a discipline) or interdisciplinary (occurring across disciplines), provides clear overarching benefits to both science and society (Wuchty et al. 2007; Pennington et al. 2013). Recent evidence based on citation rates points to the potential for greater impact from interdisciplinary versus disciplinary collaboration (Porter et al. 2012), but these successes may be countered by other work that shows that the degree of interdisciplinarity in the life sciences and biology can have negative effects on citation rates (Levitt and Thelwall 2008; Larivière and Gingras 2010). The differences in interpretation between studies may arise from various metrics of impact but may also be a result of a lack of consideration of the broad range of research products that arise from interdisciplinary research (ie papers, book chapters, posters, software, and educational training). Because all products cannot be easily quantified, interdisciplinary collaborations may be undervalued.

Numerous examples of productive disciplinary and interdisciplinary collaborations can be found in ecology from the past (Hutchinson and Bonatti 1970; Wright and Bartlein 1993) and present, including research conducted by the US Long Term Ecological Research Network (LTER), working groups of the National Center for Ecological Analysis and Synthesis, the Census of Marine Life, the Socio-Environmental Synthesis Center, and the Neotoma Paleoecology Database project. Each of these efforts and the resulting research activities has generated knowledge that extended beyond what could have been produced by an individual researcher or a team of researchers in a single discipline. The kinds of research questions addressed by large collaborative teams, including many macrosystems ecology research questions, explicitly require interdisciplinary research especially as the tools needed to conduct this type of research become more multifaceted and specialized (Uriarte et al. 2007; Levy et al. 2014).

As collaborative ecological research becomes more common, more interdisciplinary, and distributed across broader geographical regions, the challenges and benefits of collaboration need to be recognized and reconsidered. For instance, the practice of ecology within academia remains largely disciplinary (Reyers et al. 2010). Even within interdisciplinary organizations such as the LTER, early career researchers still largely engage in projects within a single discipline (Romolini et al. 2013). This focus can result in “disciplinary silos”, so-called because individuals are often the intellectual leads of their major research efforts within a single subject area, applying to a disciplinary program or agency for support and submitting publications to specialty-related outlets whose readership is primarily composed of other individuals sharing similar perspectives and disciplinary knowledge. There is growing recognition that bridging and even merging these silos is critical for fostering true interdisciplinary research, as evidenced by the National Science Foundation's (NSF's) development of programs (eg the Integrative Graduate Education and Research Traineeship; Coupled Human–Natural Systems; Science, Engineering and Education for Sustainability; and MacroSystems Biology).

Early career academic scientists are increasingly encouraged to become collaborative in practice and interdisciplinary in approach. However, their success is likely to be evaluated, at least in part, by later career stage scientists and institutional review processes that are deeply rooted in disciplinary approaches to evaluation (ie based on demonstrated independent scientific success within a specific discipline). Given the potential scientific advantages of interdisciplinary collaboration, scientists' efforts (at all career stages) in these endeavors should be rewarded, but existing measures for reward may not be suitable to support and encourage collaboration, particularly for early career researchers.

Our premise here is that successful collaborations, particularly interdisciplinary ones, can be promoted, but costs and benefits for all team participants should be recognized at the outset and placed in the context of both individual and team goals. Furthermore, as noted above, broader institutional recognition of the costs borne by early career researchers who conduct interdisciplinary research is essential and should be accompanied by shifts in the institutional measures of success. We use examples from the literature and our personal experiences with interdisciplinary collaborative teams to inform our discussion. Coauthors of this paper include six early career scientists and four more senior scientists. There has been extensive research on the strategies behind team building and the requirements for understanding philosophical underpinnings to promote interdisciplinary collaborative success (Eigenbrode et al. 2007), but few ecologists have been trained in the needed skills and strategies (but see Cheruvelil et al. 2014). In this paper, we first present a conceptual model of interdisciplinary collaborative costs and benefits that focuses on the early career stage; we then offer strategies for optimizing benefits of interdisciplinary collaborations for early career researchers in particular; and finally we make suggestions for expanding the measures of success to promote interdisciplinary collaborative research. We point out how the current reward structure in academia and other research institutions may be misaligned with the current practice of interdisciplinary collaborative science, especially for early career researchers.

A conceptual model of the costs and benefits of collaboration

An increasing number of ecologists are joining collaborative teams. A cost–benefit framework of more traditional approaches to conducting ecological research has been proposed previously (Peterson 1993), and here we build from this hierarchical, top-down system to depict a more contemporary collaborative framework. Some tangible benefits of collaboration include greater visibility within the scientific community at an earlier career stage, increased publication rates (Porter et al. 2012; but see Levitt and Thelwall 2008), higher probability of participation in future collaborative research projects (Hampton and Parker 2011), and the potential for greater success in obtaining future funding (Bellotti 2011). Less easily measured benefits that we have all experienced, but few have studied, include high personal satisfaction, the creation and fostering of lasting professional relationships, and the inspiration and enjoyment that scientists gain from fruitful collaboration.

An individual will often weigh the benefits against a set of implicit or explicit costs when choosing whether to participate in an interdisciplinary project. If costs exceed benefits then the collaboration may not take place, or the project may fail since individual participants continue to evaluate their net benefit against their investment of effort over time. Thus, while we discuss costs and benefits, we will refer to balancing and assessing net benefits under the assumption that individuals can evaluate these often intangible components.

We maintain that both the costs and benefits of collaboration are likely to vary, depending on career stage (Figure 1); we focus on early career scientists because this cohort is crucially important to future scientific success and is the group facing some of the greatest challenges as a result of the conflicting pressures of interdisciplinary collaboration and entrenched academic culture (Figure 1). Early career researchers are often key participants in collaborative research but traditionally have had less engagement with each project as a result of evolving projects and changing institutions several times during their graduate studies, postgraduate research, and full-time research positions, as opposed to senior researchers who can remain engaged with a project over a longer planning period. Shorter planning windows mean that early career scientists need research projects to come to fruition relatively quickly to benefit their career advancement. For example, graduate students and “soft-money” research scientists often rely heavily on external project funding for salary support rather than direct institutional support. This external support can be a major benefit of participation, while later career researchers are likely to have multiple sources of funding, independent of the collaborative project. The training that occurs over the course of the project can also be a major benefit for early career researchers, but the shorter period of project engagement can increase their vulnerability because of the shorter timeframe within which they can accrue benefits from any given project (ie they must be immediately productive to further their career). A senior scientist often has the luxury to “wait out” periods during which productivity may be lower, with minimal effect on career advancement since evaluation is often focused on progress over longer time periods.

Figure 1.

Several potential benefits and costs of interdisciplinary, collaborative research, shown through time. The length of the cost or benefit represents the time periods over which the cost or benefit may operate during the lifetime of the project (approximately 5 years, plus 5 years of follow-up) and is derived largely from the authors' personal experience; as such, they are meant to act as a guide to the discussion. Of note here is that senior researchers can derive benefits from a larger number of categories over a longer period of time than early career researchers.

When conducting interdisciplinary research, early career scientists must balance several challenges that primarily affect publication rates. Publication rates in the early stages can predict career longevity (Petersen et al. 2011), meaning delays in publication could harm future career prospects for young researchers. Thus, the constraints of a large collaborative project may increase the time to publication of high-impact papers, and time pressures may force researchers to sacrifice other activities (such as gaining experience in teaching or grant writing) to ensure their success within the project. There are many factors that can affect overall speed and level of productivity among a collaborative team; for instance, productivity in interdisciplinary research can decrease as a result of the time necessary to develop the links between teams and individuals required for collective thinking (Pennington 2008). Conflicts among team members due to philosophical differences among individuals or among disciplines (Hinds and Bailey 2003; Eigenbrode et al. 2007) and variations in disciplinary professional reward structures (Llerena and Meyer-Krahmer 2003; Uriarte et al. 2007) can also slow productivity. The geographic separation of project participants can lead to a lower likelihood of continuous project development (Cummings and Kiesler 2005), with concomitant delays in publication. Under intense pressure to publish, the interdependence of project components may mean that a student has less academic freedom than s/he might otherwise because other team members may depend on her/his specific research output to integrate with the larger project syntheses (Figure 1).

Perhaps the most critical challenge for early career scientists is achieving individual recognition for their work on collaborative projects. For example, Merton (1968) described the Matthew Effect, whereby credit for research is most often awarded to the most senior project participant, regardless of who carries out the actual research. This is supported by the finding that secondary authors continue to receive little recognition in interdisciplinary research (Fisher et al. 2012). Figure 1 indicates that the costs borne by early career researchers are higher in proportion to later career researchers, which can put younger scientists at greater risk of failure in interdisciplinary collaborations.

Strategies for optimizing benefits of interdisciplinary collaborations

In this section, we describe four strategies for optimizing the benefits of interdisciplinary collaborations for early career researchers.

Establish clear expectations for individuals and the team

Expectations for such important factors as training, intellectual credit, and timing of research products should be realistic, agreed upon early, and revisited throughout the project (Cheruvelil et al. 2014). Although individuals are likely to have estimated their personal costs, the potential benefits of the team's research efforts should be explicit (to the degree that is possible) so that conflicts do not arise. Developing methods for resolving conflict early in the project cycle can help improve the likelihood of project success (Zucker 2012). Even if internal conflict (ie within the team) is managed, a challenge still remains as to how an individual is rewarded by institutions for his/her research efforts in team projects.

Foster an environment of active mentoring within the team

In interdisciplinary collaboration, the opportunity to mentor extends beyond the traditional supervisor/mentee relationship typical of disciplinary research projects (eg Peterson 1993). For instance, mentoring by senior individuals of early career scientists whom they are not directly supervising is an excellent way for early career, as well as later career, individuals to broaden their network of colleagues. In addition, mentoring across career stages should be encouraged, for example when early career researchers train more senior collaborators from different disciplines or in the use of newly developed tools.

Draft explicit team policies on data sharing and authorship

Peer-reviewed publication is a well-recognized way for intellectual contributions to an effort to be acknowledged (Table 1), and publication and authorship of results may be the most contentious aspect of collaboration (eg Smalheiser et al. 2005). The value of publications across the lifetime of the collaboration may vary (eg papers published early in the research project may be more data-intensive, and cited less often, than broader synthesis papers) and interdisciplinary work is often further penalized by low citation rates when compared to disciplinary research, particularly in the life sciences (Levitt and Thelwall 2008; although see Porter et al. [2012] for a counter example). Lack of credit (or perceived lack of credit) by team members may be balanced by the increased productivity (ie number of publications) of collaborative teams (Hampton and Parker 2011). Without prior agreement on authorship and, in the case of the collection of project data, who is allowed to use the data and in what context, the desire and competing needs to secure primary authorship could cause disagreements – or result in first authorship for those not in greatest need of career advancement. Such conflicts can result in missed opportunities to effectively balance costs and benefits within a team. Pitfalls can partially be remedied by encouraging lead authorship roles among different team members. Lead authorship provides early career researchers, or those on the cusp of reappointment or promotion, with opportunities to gain leadership experience with support and mentoring from more senior personnel who are co-authors. In fact, many of the articles in this Special Issue, including this one, have followed this model; Cheruvelil et al. (2014) provide guidance and examples of authorship and data sharing policies that can be adapted for use by others.

Table 1. Traditional metrics that have been used to evaluate individual scientists conducting disciplinary researchThumbnail image of

Distribute and document the data management workload

Agreements and documentation for managing and distributing project data can help improve participant satisfaction and can potentially improve the speed of publication. Information management includes the management of project data and metadata, paper writing, administrative communication, workshop planning, timelines, paper drafts, posters, presentations, and grant proposals. Although many research projects have begun to include data management plans, ecologists have generally not included them explicitly (or have failed to enact robust plans when they do exist); this has resulted in an underestimation of requisite resources for this task (see Rüegg et al. 2014).

Ensuring a fair balance of costs and benefits among participants may require individuals to give up some benefits to help others within the team balance their costs. Individuals with longer time frames for accruing benefits may be more likely to (or can be encouraged to) cede immediate benefits to individuals with shorter planning windows, which can result in net benefits to the collaboration as a whole. For instance, lead authorships may be less important for more senior researchers, especially those in tenured positions. Changes in institutional culture are also necessary to encourage optimum functioning – and scientific success – of interdisciplinary teams. Tenure and hiring committees and proposal reviewers must recognize that some costs borne by individuals participating in interdisciplinary collaboration, particularly costs associated with publication, are balanced by other benefits that may not be easily measured. Given the critical need for expanded credit for interdisciplinary research, we propose a set of measures for more fully evaluating individual and team success.

Expanding the measures of success

Professional success in academic research careers (hiring, pay raises, promotion and/or tenure, and funding) often hinges on two measures (Table 1): the number of grants secured and dollars awarded as a principal investigator (PI; Shapiro 2006) and peer-reviewed publications (with lead investigator and first authorship being valued most; Adam 2002). A key problem is that many of the contributions of team members in collaborative research are not adequately reflected in these two traditional measures of success (Figure 2). Under many funding structures, only one scientist can be the lead PI on an interdisciplinary collaborative grant. Although this sole designation is a practical measure (so that funding agencies can communicate with the team more efficiently), PI status is often interpreted as sole intellectual leadership. It is also a practical matter that primary authorship cannot be ascribed to multiple team members. The conflation of practical/administrative and intellectual contributions and these narrow perceptions of career-based success (and commensurate rewards) could therefore easily sabotage the quality and output of the science produced by interdisciplinary teams.

Figure 2.

The interactions (red arrows) that are rewarded among individuals, institutions, and funding agencies. The traditional reward system applied to disciplinary-based research (a) is well supported and most of the depicted interactions are valued and rewarded. In contrast, the traditional reward system applied to interdisciplinary-based collaborative research (b) shows that while certain interactions are favored and rewarded (red arrows, similar to those in [a]), there are many interactions that are undervalued (gray arrows). The size and shape of symbols within the collaboration in (b) represent career stage and type of discipline, respectively. Undervaluing collaboration provides weak support for individuals engaged in this kind of research (ie fewer red arrows), even as outlets for interdisciplinary research dissemination increase. By expanding evaluation criteria for interdisciplinary research (Table 2), a more complex set of interactions is supported. These expanded measures of success should support the investment in time and effort required for effective interdisciplinary collaboration.

Collaborative, team-based ecology will achieve greater success if the professional culture – particularly peers in positions to review and reward colleagues – evolves to explicitly value all of the outcomes of successful interdisciplinary research. Some institutions have already begun to recognize interdisciplinarity in tenure evaluations; and those that adopt a broader view of merit, including more aspects of collaborative research, are likely to increase the probability of successful careers, promotion, and retention of scientists in the system. Such a broad view is consistent with the reality that answering relevant questions in ecology (and science in general), and securing funding to do so, increasingly demands interdisciplinary teams. One way to reward science conducted by teams would be for more professional societies to honor entire teams instead of individuals (eg the Nobel Peace Prize that was awarded to the Intergovernmental Panel on Climate Change in 2007 and the American Institute of Biological Sciences award for Distinguished Scientists bestowed on the LTER in 2010). Broader recognition and valuation of collaborative outcomes could ultimately result in improved institutional success in attracting faculty members, increased extramural funding, greater institutional stature, and, most important, encouraging the best scientific research.

Members of interdisciplinary collaborations should be evaluated both on individual performance, using key measures, and on overall team performance, including publications in journals outside of their disciplinary silo. In this way, contributions to leadership, data management, and other essential but “intangible” outcomes can be evaluated as part of the overall team success (Table 2). Because interdisciplinary work relies on output from all team members, the success of individuals in obtaining project-related funding, publishing project-related papers, and training students (for example) is dependent on overall team performance. As such, evaluation of an individual's direct contribution may overlook the role the individual played in supporting the collaboration through activities such as organizing and/or leading workshops, training individuals from other disciplines, and developing interdisciplinary dialogue.

Many underappreciated and underutilized measures of success could be used to value collaborative output, and many existing measures can be broadened to help highlight the role of individuals in large team efforts (Table 2). The major categories of success for a research project are defined here as (1) Research scholarship, (2) Team functioning, leadership, and training, and (3) Public outreach. For each of these outcomes it is possible to assign value to both individual and team outcomes, which may be weighted differently but should be valued nonetheless.

Research scholarship

Broadening what is considered research scholarship beyond publications and grant dollars will benefit both science and society. As the pathways between society and scientists become more diffuse, the forum for discussion moves from academic corridors and into the public sphere, resulting in greater public participation both in the applications and implications of modern ecological research (Gibbons 1999); thus, broader impacts beyond traditional publication metrics become critical. Measures for research scholarship (Table 2) can include data creation and policy outcome indicators, as well as both team and individual outputs. Creating useful databases, statistical analyses or code, merging and synthesizing diverse data streams, and working with natural resource managers and policy makers are other activities that are not traditionally viewed as research productivity yet are important components of modern interdisciplinary research. Some of these research outcomes fall into the “broader impact” criteria described by the NSF, but even with the support of funding agencies, academic culture is slow to respond to these opportunities (Frodeman et al. 2013; Nadkarni and Stasch 2013).

Team functioning, leadership, and training

Administrative and mentoring duties should be explicitly recognized and rewarded. For instance, investigators who serve as team leaders and coordinators and spend time mentoring colleagues (at all career levels) should be given credit for such critically important activities and for training they may receive before undertaking these activities. Meetings and workshops are essential for increasing team cohesiveness (Figure 3), but individuals' investment in planning and orchestrating these meetings may also be undervalued. In interdisciplinary collaboration, the opportunity to mentor extends beyond the traditional supervisor/mentee relationship. Although an important role, mentoring is difficult to assess directly, particularly when it occurs in a nontraditional manner, such as between peers across disciplines. Nevertheless, explicit descriptions of the mentoring activities can assist in its evaluation by academic committees. Giving greater credit for mentoring may facilitate provision of this benefit to early career researchers and provide support for early career researchers who may provide mentoring for more senior participants as part of their activities in the project.

Figure 3.

The network diagrams displayed here indicate increasing connectivity among members of the NSF-funded PalEON project over the course of 3 years, (a) prior to project initiation, based on publication records; (b) following the second PalEON workshop, based on publication and informal interactions; and (c) projected following completion of the PalEON project grant. In panels (a) and (b) the team structure relies strongly on one or few individuals and thus may be less resilient to conflict. Symbols indicate investigator discipline (triangles: paleoecologists; diamonds: statisticians; circles: ecosystem modelers); colors are used to highlight a diversity of career stages and project roles. Increased connectivity in interdisciplinary research projects can improve project resilience but relies on frequent interactions, such as face to face meetings, that require planning and coordination (d).

Public outreach

Public outreach can include providing ongoing educational services, thereby applying scientific results to societal problems through outreach and information dissemination (Uriarte et al. 2007). Public outreach helps to extend the knowledge produced in scientific efforts to the public sphere. Credit for making research results available to the general public will further encourage such activities, whether through blogs, educational materials, or popular science media (eg http://journalistsresource.org; Whitmer et al. 2010). Such outreach may be measured in part through existing alternative metrics that are based on online download, usage, and sharing.

How to use the expanded measures of success

Ultimately, the use of such measures of success requires two partners. First, members of collaborative teams must explain their and others' contributions in clear and meaningful ways. Individuals should promote their own activities and also act as advocates for the research team. Second, peers, review committees, and administrators should use these contribution statements to more accurately evaluate individuals and to help incentivize future collaborative research. There is strong motivation for expanding the definition of success by institutions. Scientific progress, productivity, and funding success are made more likely by collaborative participation, and by extension such successes benefit institutions as well. To foster and incorporate these broader measures of success for interdisciplinary collaborative teams, we recommend that committees deciding the hiring, promotion, tenure, and award of individual scientists should consider the following:

  1. Acknowledge and reward activities critical to the success of collaborative science, such as database creation and management, public outreach, and mentoring.

  2. Recognize that there can be large transaction costs associated with initiating interdisciplinary research that may limit productivity of the individual, at least in the short term.

  3. Recognize that all authors on multi-authored publications have made substantial contributions to the research, and that being one of 10 authors is not necessarily one-tenth the effort of being a sole author. All authors need to be credited and recognized for their contributions. As such, honorary co-authorship should be discouraged (Greenland and Fontanarosa 2012).

  4. Recognize that many successful research careers are no longer defined by single-discipline research, grants, and publications, and that such measures should not have primacy if interdisciplinary, collaborative research is to mature successfully and sustainably.

Acknowledgements

We thank the participants of the March 2012 NSF-MacroSystems Biology PI meeting in Boulder, Colorado, for providing the impetus for this paper and this Special Issue. In particular, we thank E Blood and H Gholz (NSF) for their support and M Bremigan, M Moore, and N Vermeulen for helpful comments on earlier drafts. SJG thanks C Gruzling, N Gruzling, and A Goring for support, and the PalEON project for providing a rewarding collaborative environment. We also acknowledge the MacroSystems Biology Program, in the Emerging Frontiers Division of the Biological Sciences Directorate at NSF, for support. SJG is supported by NSF grant EF1065656, KCW is supported by grant EF1137327, WKD and JR are supported by grant EF1065255, PAS and KSC are supported by grant EF1065986, LCS is supported by grant EF1065864, JSK is supported by grant EF1064998, and RMU is supported by NSF cooperative agreement EF1138160. For author contributions, see WebPanel 1.

Ancillary