Quantitative methods are the foundation of science. They represent a way of knowing the answers to very basic questions such as how big, how fast, how many, etc. The great British scientist Lord Kelvin (William Thomson, 1824-1907) noted, "If you can measure that of which you speak, and can express it by a number, you know something of your subject; but if you cannot measure it, your knowledge is meager and unsatisfactory."

With the introduction of computers after WW II and their rapid improvements in speed and capacity in the past two decades, large databases have been created of all sorts of phenomena, both natural and social. Bibliographic databases, such as ISI's, have been built up over many years now, and are a vast treasure house of insight into what happened in the annals of recent scientific history. In addition to using these databases to find the publications needed for research work at hand, investigators began turning their attention and the tools of their profession (quantitative methods) to science itself. They ask questions such as how big, how fast, how many, but in this case about scientists, nations, institutions, journals, etc.

Publication counts became units for measuring output. Citation counts, as they became available from ISI's Science Citation Index, were recognized as units of influence or impact (when total citations were divided by publications). The sociologist of science, Robert Merton, has term citations "pellets of peer recognition." In the vast majority of cases, a citation recorded in a scientific paper is a positive expression of importance, influence, or utility. Seen in yet another way, it is a type of currency by which scientists repay their intellectual debts. Over the past 30 years a large literature has accumulated on the validity of using citation counts to measure peer recognition and esteem. Citation counts correlate to expert opinion on research excellence to a degree higher than peers agree with each other on this subject. Thus, it is generally a robust measure of scholarly achievement, reflected in the referencing patterns of peer researchers.

It is one thing, simply put, to publish much and to publish in top quality journals. But it is another thing again, and perhaps even more impressive, to have your work actually cited, and frequently cited, by your peers.

Several rudimentary rules for using publication and citation counts should always be kept in mind:

1. Compare like with like: scientists or papers in the same field and papers of the same vintage, since different fields exhibit different average rates of citations and older papers have more time to collect citations than younger papers.

2. Multiple measures (number papers, citations, cites/paper, percent cited vs. uncited) and large datasets are superior to single, thin ones.

3. Relative measures should be used, not merely absolute scores (such as setting citation counts relative to appropriate baseline, or average, scores).

4. Sometimes the area of research is not adequately surveyed by the database examined, in which case the measures will not be robust and could be misleading. 

5. And, most important, that these methods should be used as supplement and not as replacement for careful consideration by informed peers or experts. 

| Return to menu |