In the competitive landscape of academic publishing, especially within engineering disciplines, the evaluation of journal quality and research impact is a critical exercise for researchers, academic institutions, and funding bodies. For decades, the Journal Impact Factor (JIF) has served as the dominant metric, often equated with prestige and influence. However, a growing recognition of its limitations has spurred a broader conversation about the need for a diversified set of metrics—ones that can more accurately reflect the nuanced and often applied nature of engineering research. This article explores the traditional Impact Factor, its shortcomings within engineering contexts, and the array of complementary metrics that researchers and evaluators should consider to gain a holistic view of journal and article impact.

Understanding the Impact Factor

The Journal Impact Factor, published annually by Clarivate Analytics in the Journal Citation Reports (JCR), is a ratio of citations to citable items over a defined period. Specifically, it calculates the average number of citations received per article published in a journal during the two preceding years. For example, the 2023 Impact Factor of a journal is the number of citations in 2023 to articles published in 2021 and 2022, divided by the total number of citable articles published in those same years. This simple formula has made the IF an easily understood and widely adopted proxy for journal quality.

Despite its ubiquity, the IF was never designed as a measure of individual article quality. It reflects the performance of a journal as a whole and is heavily influenced by a small number of highly cited papers. In many engineering subfields, citation patterns differ markedly from those in basic sciences: research cycles are longer, applied work may have a slower citation uptake, and conference proceedings often carry more weight than journal articles. Consequently, a high IF may not indicate relevance or usefulness in a specific engineering domain. Furthermore, the two-year window is arbitrary and often too short for engineering research to accumulate citations, particularly in fields like civil or mechanical engineering where validation and field testing take time. For a deeper historical perspective, Clarivate provides extensive documentation on how the IF is calculated and its intended use (Clarivate Impact Factor).

Limitations of the Impact Factor in Engineering

Engineering research is inherently interdisciplinary and application-oriented. A paper proposing a novel bridge design or an efficient manufacturing process may not attract citations as quickly as a fundamental physics discovery, yet its societal and economic impact can be enormous. The Impact Factor's narrow citation window penalizes such applied work. Moreover, the IF does not account for differences in citation norms across subfields. An aerospace engineering journal may have a lower IF than a materials science journal simply because the community is smaller and publishes fewer papers per year, not because the research is less significant.

Another critical limitation is the manipulability of the IF. Editors may encourage authors to cite other articles from the same journal to inflate the metric, a practice known as "citation stacking." Additionally, review articles tend to be cited more heavily than original research, skewing a journal's IF upward if it publishes many reviews. For these reasons, the San Francisco Declaration on Research Assessment (DORA) has called for the elimination of IF-based practices in hiring, promotion, and funding decisions (DORA Principles). Engineering departments, in particular, have been slow to adopt alternative evaluation frameworks, often relying on the prestige of high-IF journals for tenure decisions.

Finally, the IF fails to capture the digital footprint of research. Engineering papers are frequently read, downloaded, and used in patents and standards—activities that are not reflected in academic citations. Metrics that include patent citations, industry adoption, or practice codes are far more relevant to engineering impact, yet they are absent from the IF. This narrow focus can lead to undervaluation of applied engineering work and discourage researchers from publishing in specialized, lower-IF journals that serve their communities best.

Alternative and Complementary Metrics

To address the shortcomings of the Impact Factor, a range of alternative metrics has been developed. Each offers a different perspective on journal and article performance, and together they provide a more complete picture. Below are the most prominent among them, particularly relevant to engineering research.

h-Index

Originally proposed by physicist Jorge E. Hirsch, the h-index measures both the productivity and citation impact of a researcher, journal, or institution. A journal has an h-index of X if X of its articles have each received at least X citations. For example, an h-index of 50 means 50 articles have 50 or more citations, and the remaining articles have fewer than 50. The h-index is robust against outliers because it requires sustained performance. However, it also discriminates against younger journals with fewer total publications, and it does not account for co-authorship or self-citations. In engineering, where collaborative research is common, the h-index can be a useful but imperfect measure. Many database platforms like Scopus and Web of Science automatically calculate h-indices for journals.

Eigenfactor Score and Article Influence Score

The Eigenfactor score, developed by Jevin West and Carl Bergstrom at the University of Washington, goes beyond simple citation counts by weighting citations based on the prestige of the citing journal. It operates on a model similar to Google's PageRank: a citation from a high-impact journal contributes more than a citation from a marginal one. The Eigenfactor is normalized so that the average journal in the JCR has a score of 1.0. For engineering evaluators, this metric accounts for the quality of citations, mitigating the influence of journals that engage in excessive self-citation. A related measure, the Article Influence Score, represents the average influence of a journal's articles over the first five years after publication, thus extending the observation window beyond the IF's two-year limit. The Eigenfactor Project provides detailed methodology and data (Eigenfactor Project).

CiteScore and SCImago Journal Rank (SJR)

CiteScore, published by Elsevier based on Scopus data, is similar to the IF but uses a three-year citation window and includes all document types (not just citable articles). This broader inclusion can benefit engineering journals that publish a mix of research papers, reviews, and case studies. CiteScore is also free to access, unlike the proprietary IF. The SCImago Journal Rank (SJR) is another alternative that incorporates the prestige principle similar to Eigenfactor but uses Scopus data and a different algorithm. SJR weights citations from highly cited journals more heavily and applies a subject field normalization, making it particularly valuable in engineering where cross-disciplinary citations are common. Both CiteScore and SJR are indexed on the SCImago Journal & Country Rank portal, which allows easy comparison across subdisciplines (SCImago Journal Rank).

Altmetrics

Altmetrics capture online attention to research outputs through social media mentions, news articles, blog posts, policy documents, Wikipedia citations, and download counts. For engineering, altmetrics can reveal early impact that precedes formal citation. A paper on a new renewable energy technology might be shared widely on Twitter, discussed in industry blogs, or even cited in a government policy report long before it accumulates academic citations. Platforms like Altmetric.com track these events and provide a weighted score that reflects the source of attention. However, altmetrics are noisy and can be gamed; they complement rather than replace citation-based measures. They are especially useful for showing public engagement with engineering research that has societal relevance. Learn more about Altmetrics at Altmetric.com.

The Case for a Multi-Metric Approach

No single metric is perfect. Relying solely on the Impact Factor can lead to distorted assessments, penalizing engineering journals that serve smaller specialized audiences or emphasize applied work. A multi-metric approach allows evaluators to triangulate different aspects of impact: citation intensity (IF, CiteScore), citation prestige (Eigenfactor, SJR), sustained performance (h-index), and broader societal engagement (altmetrics). For example, a journal with a moderate IF but a high SJR and strong altmetric attention might be more influential in engineering practice than a journal with a high IF but low prestige-weighting. Institutions are increasingly adopting balanced scorecards that combine several of these indicators for tenure and promotion decisions, in line with the DORA recommendations.

Moreover, the multi-metric approach helps researchers choose the right venue for their work. A team working on an applied control systems project may prioritize a journal with strong industry readership and high altmetric mentions over one with a high IF but less relevance. Similarly, a department reviewing candidate productivity can use h-index alongside field-normalized citation percentiles to account for discipline-specific norms. The key is to understand what each metric measures—and what it misses—and to combine them thoughtfully rather than adding them up mindlessly.

Implications for Researchers and Institutions

For researchers, understanding these metrics is essential for strategic publishing. Early-career engineers should be aware that a high IF journal may have a low acceptance rate and long review times, which could delay dissemination of time-sensitive findings. Conversely, publishing in a journal with strong altmetric engagement can increase the visibility of their work among practitioners and policymakers. Researchers should also be cautious about metrics manipulation: journals that boast unusually high IFs or self-citation rates should be scrutinized. The use of the h-index in hiring committees is widespread, but candidates should ensure their profile on platforms like Scopus or Google Scholar is accurate and up-to-date.

Institutions, especially engineering colleges and departments, have a responsibility to move beyond simplistic metrics. Many still use JIF cutoffs for funding allocation or tenure milestones, which can penalize faculty who publish applied work in field-specific journals. A more equitable approach involves peer review of research outputs, complemented by a portfolio of metrics that align with the department's mission. For instance, an institution emphasizing innovation and industry collaboration might weight patent citations and altmetrics more heavily. Faculty evaluation committees should be trained to interpret metrics critically and to recognize their limitations—especially in underrepresented engineering subfields like structural engineering or manufacturing.

Future Directions in Research Evaluation

The landscape of research evaluation is evolving rapidly, driven by open science initiatives and technological advances. One emerging trend is the use of machine learning to identify research impact patterns, such as predicting which articles will become highly cited based on early usage data. Another development is the rise of "responsible metrics" frameworks, which emphasize qualitative peer review alongside quantitative indicators. The European Commission's Open Science Policy and the Leiden Manifesto for research metrics both advocate for context-sensitive evaluation that respects disciplinary differences. Engineering research, with its strong ties to industry and society, can benefit from these trends by incorporating measures like patent citations, standards references, and technology readiness levels (TRLs) into evaluation portfolios.

Furthermore, the increasing adoption of persistent identifiers (ORCID, DOI) and open-access publishing will make citation data more transparent and reproducible. Future metrics may include "impact factors for the global south" that account for research relevant to developing economies, an area where engineering innovation is often critical. As the community moves toward a more holistic view of impact, it is essential that engineering researchers and administrators stay informed about these developments and actively participate in shaping evaluation practices that truly reflect the value of their work.

In summary, while the Impact Factor remains a widely recognized metric, its limitations are particularly acute in engineering research. By embracing a multi-metric approach—combining journal-level indicators like Eigenfactor and SJR, author-level measures like h-index, and alternative metrics like altmetrics—the engineering community can achieve a more accurate and fair assessment of research impact. This broader perspective not only benefits individual researchers but also strengthens the discipline's ability to demonstrate its real-world contributions. As we move forward, the challenge is not to discard the Impact Factor entirely, but to place it within a richer framework that values all forms of engineering scholarship.