Editorial, DRT, 19(3), 2001
Considerable human resources as well as financial resources go into the R&D effort leading to an archival publication. With limited resources it is not surprising that all concerned parties wish to carry out this process as effectively as possible. The principal objective of the authors of papers and their granting agencies is to ensure timely and wide dissemination of the outcome of the research effort after a careful in-depth review. This also avoids the possibility of “re-inventing the wheel,” which is clearly a waste of the limited resources available for R&D.
I believe that another important objective of publication
of engineering R&D and the reason why a reader would read it is that it
provides an insight into or provides useful data or design procedures that are
useful in his/her engineering practice. Academic researchers typically seek out
such publications to identify areas of research that are of interest to them
and to which they believe they can make a contribution. Such interest can lead to another
publication based on ideas published in one or more of prior publications.
Thus, the journal publications receive citations when academics actively
pursuing research in a given area publish their own contributions in the same
or another archived journal. However, an engineer in industry or a researcher
in industry will typically benefit from the publication of a given engineering
paper but this will not lead to a citation. Indeed, the more relevant and hence
valued the publication is to engineers and R&D personnel in practice, the
less likely it is to receive academic attention and hence journal citations.
This is all obvious and logical. In science, the situation is quite different
since most papers published in that discipline are for academic researchers
whose primary business is to publish research papers in archival journals and
thus naturally receive voluminous citations. This is clear even from a cursory
glance at the 1999 figures of impact factors published by Institute for Scientific
Information (ISI), which are well known to academic researchers.
Journals devoted to engineering science and hence
primarily, if not exclusively, directed to the academic research community have
higher impact factors since their readership has as its primary focus the
publication of research papers after positive peer reviews. One paper thus
spawns additional papers, which cite the original paper out of ethical
considerations or to support their own work. I called this “closed-loop”
approach of academic research leading to research by academics in academia for
academics. This model, in my opinion, is not well suited for engineering
research although it may well fit the needs of the scientific community. Thus,
good science-oriented journals typically have high impact factors. Engineering
journals typically have lower impact factors; the magnitude depends on how its
readership is distributed between academia and industry. A journal that
publishes R&D results of interest to industry and the practicing engineer
is clearly unlikely to have a high impact factor when a significant fraction of
its readership is in the category of users (and not publishers) of what is
published. This does not reflect adversely on the value of the journal or the
author’s contribution to the archival literature; it is just in the nature of
the criterion used to calculate the impact factor as defined currently.
There is one even more serious flaw with the total
dependence on the impact factor to assess research or its value. By definition
the number of citations for currently popular or “buzz word” research is
automatically higher as more academics are active in the area. Truly innovative
research is often drowned in the mainstream research and not pursued actively.
More run-of-the-mill research is likely to be cited extensively (as is one with
major flaws that other authors may wish to point out) since most fields cannot
be filled with truly innovative ideas at a given time. Every researcher with
extensive research portfolio and diverse research contributions can easily
reflect and identify his/her own research papers, which he/she considers truly
original and innovative. I suggest that they then examine the number of
citations for such work. If they compare this number with the number of
citations for the more routine research papers, I am sure, in most cases, the
latter will have larger numbers of citations because of the very nature of the
research model. I have personally carried out this exercise and found that the
most original and innovative papers have attracted fewer citations compared to
the ones in mainstream areas, which stretched only modestly the contemporary
knowledge of the subject area. The work that led to numerous industrial
applications in several countries and even building of expensive pilot tests by
industry received little attention of other researchers.
Without going into the numbers
and an in-depth discussion, suffice it to point out the order-of-magnitude of
the impact factors of key archival journals in chemical and mechanical
engineering disciplines for the year 1999. For purely scientific journals the
factors are of order 10, for engineering science journals they are of order one
and for technology and engineering journals they are about half of those for engineering
science journals. This is consistent with the hypothesis I have propounded
here.
I hope I have made a useful
point in this exercise, which deserves discussion and a quantitative analysis
by interested parties. While a useful indicator of the academic quality of the
research published, the impact factor, as defined currently, does not reflect
on the true engineering significance and certainly not on the real engineering
value or impact of the reported research. If engineers in practice do not read
it or use it in any useful way, such research publications – despite the high
impact factor - are unlikely to have direct impact on industrial practice. A
different type of impact factor needs to be invented to assess the real value
of an engineering research publication. A purely statistical number based on
citations alone may be good for the sciences and arts but it fails miserably to
do justice to the true engineering researcher.
I hope that some day some one
will come up with a way to measure the true rather than apparent impact factor
based on number of citations. Indeed, some careful research will be needed
first to determine, in a quantitative manner, the validity of the currently
defined impact factor for engineering research since most of the highly cited
papers never make an impact on the discipline but possibly only on the careers
of the academic researchers. The current impact factor should perhaps be called
academic impact factor since it truly measures it fairly accurately.
Arun S. Mujumdar
Singapore