Rating Researchers

For baseball pitchers, it's the earned run average. For National Football League passers, it's the quarterback rating. In each case, a single number gives some sense of a player's value.

How would you quantify the cumulative impact and relevance of a researcher's scientific work? Furthermore, is there a single number that would usefully characterize a scientist's output?

"In a world of limited resources, such quantification (even if potentially distasteful) is often needed for evaluation and comparison purposes," physicist Jorge E. Hirsch of the University of California, San Diego argues in the Nov. 15 Proceedings of the National Academy of Sciences.

Such ratings can be (and already have been) used to measure the impact of research and to justify tenure and funding decisions. Analyses of these measures can also generate rankings of scholarly journals, academic departments, universities, and even countries.

The most accessible information on which to base such judgments is an individual's publication record and the number of times these publications have been cited in other journal articles.

Looking just at the total number of published papers certainly measures productivity, but it doesn't give any sense of the importance or impact of those papers. Looking at the total number of citations gives a sense of impact, but the numbers may be biased by a small number of "big hits" or by review articles, which tend to be highly cited, especially in hot research areas.

Hirsch proposes a new metric, which he calls the "h index," for characterizing the quality of an individual's scientific output. In essence, this score is based on the highest number of papers that a scientist has produced that have had at least this number of citations.

So, a score of h = 30 means that you have published 30 papers, each of which was cited at least 30 times. In effect, published papers with fewer citations than h don't count in the analysis.

"Two individuals with similar h's are comparable in terms of their overall scientific impact," Hirsch says, "even if their total number of papers or their total number of citations is very different."

"Conversely," he adds, "comparing two individuals (of the same scientific age) with a similar number of total papers or of total citation count and very different h values, the one with the higher h is likely to be the more accomplished scientist.

Because publications practices differ from field to field, it's best to compare results within a discipline. Hirsch's calculations, which involve citation data from Thomson Scientific's ISI Web of Knowledge, reveal that the highest h among physicists belongs to Edward Witten of the Institute for Advanced Study in Princeton, N.J. His score is 110.

Other physicists with high h values include Arthur J. Heeger (107), Marvin L. Cohen (94), A.C. Grossard (94), Philip W. Anderson (91), Steven Weinberg (88), Michael E. Fisher (88), Manuel Cardona (86), P.G. deGennes (79), John N. Bahcall (77), Zachary Fisk (75), and D.J. Scalapino (75). Hirsch himself, whose field is superconductivity, has a rating of 49.

For life scientists, the numbers are higher. The following researchers rank near the top in their field: Solomon H. Snyder (191), David Baltimore (160), Robert C. Gallo (154), Pierre Chambon (153), and Bert Vogelstein (151).

The value of h is likely to increase over time, approximately linearly, as a researcher publishes additional papers and garners new citations, Hirsch suggests. This increase can be captured, roughly, by another parameter, m, which is the slope in the linear relation linking h and the number of years, n.

According to Hirsch, a value of m = 1 would characterize a successful scientist. For example, this would mean achieving an h index of 20 after 20 years of scientific activity. Higher values of m would signify higher levels of achievement.

Based on his calculations, Hirsch suggests that, for physicists, a value for h of about 12 might be a useful guideline for tenure decisions at major research universities. A value of about 18 could mean a full professorship, 15–20 could mean a fellowship in the American Physical Society, and 45 or higher could mean membership in the National Academy of Sciences.

Of course, there are caveats, and the system is dependent on the availability of reliable, unbiased, complete citation records.

"Obviously, a single number can never give more than a rough approximation to an individual's multifaceted profile, and many other factors should be considered in combination in evaluating an individual," Hirsch notes.

Nonetheless, he concludes, the h index "may provide a useful yardstick with which to compare, in an unbiased way, different individuals competing for the same resource when an important evaluation criterion is scientific achievement."