Track record?


As scientists, we are fortunate to have the opportunity to explore hypotheses – as many as our minds can generate and our lab fellows are willing to test. Although only a fraction of these hypotheses will be proven correct, the serendipitous discoveries made in the course of our explorations will lead us to formulate new models and, of course, new hypotheses. Like Sherlock Holmes, we focus on solving the crime, mechanisms underlying pathogenesis, while also enjoying the mystery. We have the honor to be among the select few that comprise this 24/7 investigative team. Yet, we are required to prove ourselves constantly. Even if we have performed outstanding science for our entire careers, we are obliged to prove time and again that our plans are competitive with those of the best and brightest of the next and the current generation.

Is this really so? Well, yes – if you depend on peer-reviewed grant support. In essence, this is the principle of the peer review system in the US and, in differing configurations, in many other countries. The most important criteria in evaluating scientific grants are the novelty and significance of the problem to be studied, as well as the impact that the studies would have when completed. The ideas to be tested should be exciting and with very few exceptions, they should be backed up by solid preliminary data.

Yet, where does the investigator’s track record fit into this? Apparently, in different systems, this parameter has different weights. NIH Intramural review guidelines devote approximately 50% of their evaluation for the scientist’s track record. In the Extramural system, it appears to be left up to individual peer reviewers to decide how the scientist’s track record is to be emphasized. Additionally, the standards set for young investigators appear to be more flexible. If an applicant has not published significant work during the past decade, and has written an impressive application, should the weak productivity be a major point of discussion, or merely, say, 30% of the overall score? Or, could one argue that if there have not been significant scientific accomplishments in recent years, why would one expect that there would be a different outcome in the future?

How do we define good progress or impressive track record? Currently, it seems that this is too often based on the quantity, not quality, of publications. This metric vastly undervalues the more time-consuming disciplines; for example, the time needed to perform complex studies that integrate genetic mouse models or clinical data into mechanistic biochemical or cell biological studies. Hence, in assessing a scientist’s track record, it should be more important to evaluate the overall past research impact than to count the number of low impact publications. Has the applicant’s research resulted in a paradigm shift? Are his or her contributions translatable to clinical evaluation? Do they present a novel technology? Are they broadly applicable? Have they been cited more than a few times and not just within a small field?

At times when fewer grants are funded, past performance should be key in the evaluation of an idea; however, brilliant it may appear. It takes much to establish a strong track record. A constant stream of high-quality studies that impact the field to the degree others would follow one’s path is a great example. A good place to start would be to define how significant the question one addresses is, and the advances achieved by the answers gathered. It would seem that Medawar’s ‘Advice to a Young Scientist’ is as valid today as when it was first published in 1979: ‘It can be said with complete confidence that any scientist of any age who wants to make important discoveries must study important problems’.