International benchmarking of biotech research and research training centers—lessons and perspectives: Results from a european-united states comparative study characterizing the centers and their performance
What are the structural similarities and differences between European and United States biotechnology research centers? How do they perform? What is their impact? These were the basic questions underlying a study launched by the European Commission's Directorate General for Research in 2000 . A U.K.-Germany-U.S. research team tried to provide the answers via a large survey.1
There are many problems associated with collecting any sort of data on biotechnology. On the one hand definitions are ever-changing, making it difficult for statisticians to produce consistent time series. On the other hand there are problems providing policy makers with the indicators they need because changes in biotechnology research and development are happening so rapidly that they can not be tracked and responded to in time. For these reasons the collection of better data and the production of improved indicators on biotechnology are priorities not only for Europe's policy makers but also at the international level, which is the reason why the Organization for Economic and Cooperative Development is also involved in studies on this subject.
DEFINITIONS AND OBSTACLES
The research team here used a definition of biotechnology that has been developed in a previous study and which is basically a list of seven broad biotechnology research areas such as “environmental biotechnology” that are made more specific with up to nine subareas such as “biosafety” or “microbial ecology.” The definition of a “center,” on the other side, proved to be much more complicated, at least for Europe. Whereas the formation of a center is seen as a typical U.S. academic research and research training development strategy, European biotechnology centers were primarily created by providing incentives for existing research centers to move into this area, by establishing new ones, or by relabeling existing ones. Several large public research centers that focused, for example, on traditional agricultural research have shifted their specialization to agricultural biotechnology research and sometimes have reflected this change in a name change as well.
However, not every institution containing the word “biotechnology” in its name or doing research in biotechnology is necessarily a biotechnology research center. A lot of European public research centers perform several research strategies, and biotechnology might well be one among others. How to deal with this reality? The research team opted for a rather strict definition of “biotechnology research center.” In order to qualify, the centers had to have at least 50% of their research activities focusing on biotechnology, be receiving at least 50% of their funding from public sources, and have a specific mission related to biotechnology. The latter could be one or more of the following items: providing education and training, building up the knowledge base, creating a national center of research excellence, or fostering commercialization.
Although the prime interest at the Commission level is focused on its 15 member states, the geographic coverage of the study was extended to include Switzerland, a European powerhouse when it comes to pharmacy and biotechnology. From an initial 714 centers that were identified (504 centers in Europe and 210 in the United States), 324 responded to the survey (Europe, 272; United States, 52). However, due to the definition, not all qualified as biotechnology research centers. One hundred ninety-four European centers and only 32 U.S. centers remained in the sample. Within Europe the numbers vary considerably by country. Although there are 44 German centers (of an initial 139, as identified) included, there are only three Irish centers (from an initial six), nonetheless reflecting the size of the countries.
Although the European sample seems to be representative, a closer analysis of the U.S. centers shows that the ones responding to the survey were not the ones that are traditionally well known and well established. On the contrary, it was found that the U.S. centers that responded did not represent “first movers and major players” or the top quality universities represented among the Council of Biotechnology Centers/Biotechnology Industry Organization. Instead, the respondents represented only “second movers and emerging actors,” or major research universities late to enter biotechnology research and universities without an established research tradition. This does not necessarily render the outcome invalid or obsolete, but the question arises whether the data derived from the partial survey of the U.S. centers can be used as a benchmark or not.
A comparison of European-United States structural features shows that
More than three-quarters of European Union (EU)2 centers and all the U.S. centers were established during the last two decades. The remaining EU centers are even older, some dating back to the turn of the 19th century. They are primarily the ones that have shifted their research focus to biotechnology over time.
Whereas the majority of centers are affiliated with universities, European centers are also affiliated with a range of other types of organizations. The majority of the European centers are unitary centers based in a single institution (65%). Cooperative or “virtual” centers in the same town or in different geographic areas were found, however, only in the European part, but this does not imply that there are no “virtual” centers in the United States.
Most of the European and U.S. centers have multiple missions. They differ very little in their mission to provide education and training, build up the knowledge base, or create a national center of research excellence. A wider gap exists, however, when it comes to commercialization. Only approximately half of the European centers have a mission to foster commercialization, compared with almost three-quarters of the U.S. centers that responded.
When it comes to size, European centers are, on average, significantly larger than U.S. centers in terms of the total number of research staff and budgets.
A quite significant difference was found concerning long term versus short term researchers. Here the U.S. centers clearly favor long term employment, while in Europe more researchers were employed on a short term basis.
There is no significant difference in the average number of staff in “single focus” and “multifocus” centers. Among the single focus centers in both Europe and the United States, those specializing in human biotechnology have the highest number of staff and the highest budget per staff member. “Basic research” and “plant biotechnology” have the lowest budget per research staff member.
Key personnel, defined as those who had published more than 20 publications in the period 1994–1999, were identified in 45% of EU centers but only 22% of the U.S. centers that responded. The average number of key personnel per center is also considerably lower in the United States than in the EU.
In comparing European-United States process features, it was found that
The majority of the centers are involved in several areas of biotechnology research. U.S. centers allocate a very high percentage (50%) of their expenditure to basic research. EU centers spend over 30% of their budgets on human biotechnology, followed by basic research (17%), cell factory (15%), and plant (12%) and animal (10%) biotechnology.
Although most centers are involved in several areas of research, 57% can be regarded as single focus because a significant proportion of their budget is dedicated to one area. The remaining centers allocate their budgets more or less equally to several areas and can be regarded as multifocus.
Most dramatic increases in employment concerned the employment of technicians.
Whereas 48% of the responding centers are not permitted to award Ph.D. degrees, they have numbers of doctoral students similar to those biotechnology centers that are permitted to award Ph.D. degrees. Doctoral students per center and per member of research staff are similar in the EU and the United States. EU centers have double the productivity of U.S. centers in Ph.D.s awarded for both these measures.
Structure + Process = Outcome
Outcome comparisons between European and U.S. centers show that
When it comes to the question of growth or decline of a center, continuous growth characterizes the development of 70% of all centers. Only a handful state a decline or the occurrence of a single decrease in their lifetime.
EU centers account for 7.1% of all EU biotechnology publications in the period 1994–1997, and U.S. centers for 0.3% of all U.S. biotechnology publications. The publications of EU centers received an average of 10.7 citations per publication and U.S. centers 18.2 citations per publication.
In terms of “prestige” activities, i.e. highly active researchers in terms of publications, scientific awards, and memberships of external committees, European centers have more members of editorial and scientific committees. They also send far more researchers to transatlantic conferences than those in the United States.
In terms of industrial activities, a higher percentage of European centers have industrial research collaborations as well as a higher average number of collaborations per center than do U.S. Centers.
U.S. centers have a higher launch rate for spinoff firms than do EU centers.
PERFORMANCE AND IMPACT
The very low output in terms of scientific publications leads to the conclusion that a further analysis is necessary. Are these centers the core biotechnology research centers? The clear answer is no. Although scientific publications account for almost 33% of publication output in Portugal, their average share in Europe is 6.9% and a mere 0.3% in the United States.
An analysis of how various structural features of the centers affected their performance in terms of academic functioning, prestige, industry collaboration, and networking was undertaken for all the centers. Whereas 72% of the centers achieved a high performance in at least one of these areas, only 2% scored highly in all four areas. This is an interesting finding that might lead to more reflective thinking about possibly conflicting goals of public research centers in Europe as well as the United States.
The age of the center, the number of personnel, and the centers' research specializations mattered in achieving a high performance in the above categories. Centers showing a high performance in academic research had up to 25 researchers and were often specialized in human biotechnology. The ones excelling in industry collaboration were approximately of an equal size, specialized in cell factory activities, and were between 12 and 20 years old. The centers excelling in networking were slightly younger and tended to have a single research focus.
In order to estimate the relative efficiency of the European and U.S. centers, inputs (budgets and research staff) and outputs (Ph.D. degrees, publications) relating to the missions that they might undertake (research excellence, research training, fostering commercialization, and knowledge production) were compared. Whereas 47 centers (20%) are efficient in at least one mission, 153 centers (80%) are inefficient. Only 10 centers (4%) are highly efficient in all the missions they undertake. However, no center is efficient in all four missions; all of the 19 centers stating four missions are not efficient in even one of them.
Again, these findings suggest that the goals as set might be too diverse. A further analysis to identify any relationship between efficiency in the four missions found a strong relationship between efficiency in research excellence and research training and a weak relationship between research excellence and fostering commercialization. The researcher team pointed out that the “second movers and emerging actors” group of U.S. centers represents a high proportion of centers efficient at knowledge production and at fostering commercialization.
One of the most important findings, at least for the policy makers, is the relationship between efficiency and having a single research focus. This, again, is strongly related to efficiency in research excellence. A slightly weaker relationship also exists for efficiency in research training and knowledge production. The share of long term research staff in total staff also has a significant but weak influence on centers efficient at achieving research excellence. Industrial excellence on the other hand had no relationship to funding or staff. The research focus, age, or the existence of a technology transfer office was equally not related. The authors concluded that this might be due to the fact that efficiency in industrial commercialization is mainly influenced by intangible characteristics such as management. However, considering the strong orientation of U.S. centers toward fostering commercialization and basic research stresses the importance of performing basic research relevant to industry. Furthermore, a good performance in industry-related activities is perpetuated by academic activities that lead to research excellence and prestige. This will attract them to industrial partners and possibly improve their level of industrial performance.
LESSONS & PERSPECTIVES
Despite the fact that the U.S. first movers in biotechnology were more or less absent from the study, the findings about the European centers and the generation of U.S. second movers gives some clear insights and indications. Although there are notable exceptions among the European centers, the situation grosso modo is that the European centers seem not to be focused enough on becoming serious players. The wish to be an excellent knowledge provider to industry, a provider of top-class education and training, and ardent publishers of scientific literature are goals often assigned to public research centers as well as self-professed claims, and these might be too much for the second movers that are clearly missing the advantages of the now well-established, large, and possibly excellent first movers. The various obligations are of course often put forward by policy makers that want return on scarce or even declining (public) investment. However, there is still a considerable amount of public investment in biotechnology research in Europe that is nonetheless often coupled with several goals. Centers might struggle to perform well in all and, if one uses the results of this benchmarking study, rarely achieve these goals. The finding that small but targeted research centers were among the best performing ones leads the authors to conclude that a strategy of concentrating resources—a strategy often put forward by European policy makers—may not be the best way to proceed. Instead, the fact that U.S. centers at nonprestigious universities are able to achieve high performance and efficiency gives hope that this might happen in European peripheral regions as well. This outcome contrasts the view formulated in Ernst & Young's  2001 report that “… many smaller institutes that struggle to achieve critical mass populate the community.” It might be necessary to analyze the complex of “critical mass” in order to be able to estimate if the current number of centers is too high.
Another issue that intrigues the authors is the surge of technician positions in the centers. An important issue to policy makers and professionals alike, one needs to ask what this surge will imply for biotechnology employment, current curricula, and student enrollment.
The project was lead by J. Senker, P. Patel, and J. Calvert (Science and Technology Policy Research, University of Sussex, Brighton, UK). Partners were S. Hinze and T. Reiss, (Fraunhofer Institute for Systems and Innovation Research, Karlsruhe, Germany), and H. Etzkowitz (State University of New York, Purchase, NY).
The abbreviation used is: EU, European Union.