Peer review serves as the primary gatekeeper for quality in academic publishing, and nowhere is its function more consequential than in engineering. When a structural design, a materials synthesis, or a software algorithm appears in a peer-reviewed journal, practitioners and researchers alike assume that the work has undergone rigorous scrutiny. Yet the reliability of that scrutiny has been questioned for decades. Inconsistent standards, hidden biases, and the sheer volume of submissions strain a system that often operates behind closed doors. This article examines the reliability of peer review in engineering journal editorial decisions, identifying persistent vulnerabilities and evaluating reforms that could strengthen the process.

The Foundational Role of Peer Review in Engineering

Peer review in engineering journals typically follows one of several models. The most common is single-blind review, where reviewers know the authors’ identities but authors do not know the reviewers’. Double-blind review, in which both sides are anonymized, is increasingly adopted to mitigate bias. Open review, where names are revealed and sometimes comments are published, remains less common but is gaining traction. Regardless of the format, the core purpose is the same: expert evaluation to verify that the research is original, methodologically sound, and appropriately interpreted before it reaches the broader community.

Engineering manuscripts often involve experimental data, computational models, or design analyses that require domain‑specific knowledge. Reviewers are expected to check for technical correctness, reproducibility of results, and adherence to reporting standards such as those for uncertainty quantification or code availability. The journal editor synthesizes these evaluations, weighting them against the journal’s scope and the paper’s potential impact, to make an accept–reject decision. In high‑quality engineering journals, rejection rates exceed 70 % for initial submissions, underscoring the selective role peer review plays.

Yet the process is not purely objective. Editors must interpret conflicting reviewer recommendations, gauge the thoroughness of each review, and sometimes override opinions when they detect bias. The reliability of the entire system hinges on the assumption that reviewers can and will perform their duties impartially and competently.

Key Challenges Undermining Reliability

Reviewer Bias and Its Many Forms

Bias in peer review can take numerous forms. Confirmation bias leads a reviewer to favour results that align with their own previous work or theoretical stance. Institutional bias may cause a reviewer to undervalue research from less‑prestigious universities or from developing countries. Gender and racial biases have also been documented in multiple fields; studies have shown that manuscripts with female first authors or authors with non‑Western names sometimes receive harsher assessments. In engineering, where applied research often has commercial implications, a reviewer with a financial stake in a competing technology may consciously or unconsciously downgrade a submission.

These biases undermine the fundamental fairness of the editorial process. When bias goes unchecked, it not only harms individual authors but also distorts the scientific record, favouring certain topics, methodologies, or research groups over others. Journals that rely solely on single‑blind review are especially vulnerable because reviewers can see the authors’ affiliations and past publications, which may trigger subconscious prejudice.

Inconsistency Across Reviewers

A well‑known study in the journal Nature asked reviewers of the same manuscript to provide evaluations; the inter‑reviewer agreement on whether a paper should be accepted was barely above chance. Inconsistency is a persistent problem in engineering as well. One reviewer might focus on the novelty of the approach, while another emphasizes the experimental validation, and a third critiques the statistical analysis. Without consensus on what constitutes an acceptable submission, editors are left to exercise subjective judgment, which can vary widely between editors and across journals.

This variance is compounded by the lack of standardized review criteria in many engineering fields. Unlike some biomedical disciplines that have adopted structured reporting checklists, engineering journals often leave the review entirely unstructured, allowing reviewers to apply personal criteria. As a result, a paper rejected by one journal due to perceived lack of novelty may be accepted by another with only minor revisions, raising questions about whether peer review filters for genuine quality or merely for alignment with a particular reviewer’s worldview.

Transparency and Accountability Gaps

Traditional anonymous review creates an accountability black box. Reviewers spend little time on the process; one survey found that the median time a reviewer spends on a manuscript is under one hour. With no public record of their comments (and often no requirement to sign them), reviewers have little incentive to provide deep, thoughtful critiques. Moreover, the anonymity barrier makes it difficult to detect collusion or ghost‑review behavior, where authors or editors influence the selection of friendly reviewers.

Editors themselves operate behind the scenes. While most journals now provide a “reject and resubmit” option, the decision rationale is rarely made public. This lack of transparency makes it hard for the broader engineering community to evaluate the fairness and consistency of editorial decisions. It also erodes trust, especially when high‑profile papers are retracted after publication because flaws were missed during initial review.

Time Pressures and Workload

The volume of engineering manuscripts has grown substantially over the past decade. Many journals report a 5‑10 % annual increase in submissions, while the pool of willing reviewers has not expanded proportionately. Editors are forced to send more invitations, resulting in higher rejection rates from potential reviewers. Those who do accept are often overcommitted; the typical reviewer might handle 10–20 papers per year in addition to their own research, teaching, and administrative duties. The pressure to produce a fast turnaround—often expected within two to four weeks—can lead to cursory readings that overlook critical errors.

Time constraints also affect the editor’s ability to investigate suspicious data or request clarifications from authors before making a decision. In engineering, where data sets can be large and analyses complex, a rushed review may fail to detect fabricated data, incorrect modelling assumptions, or inappropriate statistical tests. The result is a system that, while still functional, operates at the edge of reliability.

Methodological Reforms to Bolster Trust

Open Peer Review Models

Open peer review—where reviewer names and sometimes the full review history are published alongside the article—aims to increase accountability. Proponents argue that knowing their identity will become public encourages reviewers to be more thorough and civil. A growing number of engineering journals, such as those published by the journals of the Institute of Electrical and Electronics Engineers (IEEE) and some Springer‑Nature titles, have implemented optional or mandatory open review. Preliminary evidence suggests that open review yields longer, more detailed reviews, though concerns about career reprisals prevent some researchers from accepting invitations.

Hybrid models also exist. For example, some journals anonymize reviewers initially but later, if the paper is published, allow reviewers to opt in to have their name associated with the review. This middle ground preserves the benefits of anonymity during evaluation while creating an incentive for high‑quality reviewing. Journals that have adopted such practices report higher reviewer satisfaction and lower rates of offensive or dismissive comments.

Structured Review Checklists and Training

One of the most direct ways to reduce inconsistency is to provide reviewers with a structured checklist that aligns with the journal’s editorial priorities. Many top‑tier engineering journals now ask reviewers to rate specific aspects—such as significance, novelty, methodology, data availability, and clarity—on a Likert scale, rather than leaving the review completely open‑ended. This approach forces reviewers to address each dimension, making it harder to overlook a critical flaw. The checklist can also be tailored to the manuscript type: a review article will have different criteria than an original research paper, and an experimental study may require additional scrutiny of measurement uncertainty.

Reviewer training programs, such as those offered by the Committee on Publication Ethics (COPE), have been developed to teach best practices. Training modules cover bias awareness, constructive feedback, ethical breaches, and how to handle conflicting reviews. Early studies indicate that trained reviewers produce more reliable assessments, although adoption remains low due to time and resource constraints. Journals could mandate training for all new reviewers, as some medical journals have done, to raise the baseline quality.

Preprint Servers and Post‑Publication Review

Preprint servers like arXiv, engrXiv, and TechRxiv have changed the publishing landscape by making engineering research publicly available before formal peer review. This separation of dissemination from evaluation allows the community to critique findings early, sometimes uncovering errors before the journal review process begins. Post‑publication review platforms such as PubPeer enable comments on published articles, serving as a continuous quality control mechanism. However, the effectiveness of post‑publication review depends on community participation; many published papers receive no comments at all.

Journal editors are increasingly using preprint feedback to inform their initial editorial decisions. Some journals now allow authors to submit a manuscript that has received comments on a preprint platform, and editors factor those comments into their evaluation. This hybrid model can reduce the burden on formal reviewers while leveraging broader expertise. Yet it also introduces risks: public comments can be harsh or misinformed, and authors may feel pressured to respond before the journal process is complete. Balancing openness with author protection remains an ongoing challenge.

Registered Reports

Registered reports represent a radical departure from traditional peer review. In this format, the introduction, methods, and planned analyses are submitted for review before data collection or experiments begin. If accepted, the journal commits to publishing the results regardless of the outcome, provided that the authors follow the approved protocol. This approach eliminates publication bias toward positive results and forces reviewers to evaluate the research question and methodology rather than the final findings.

Registered reports are still rare in engineering, but they have gained traction in psychology and are now being piloted by some engineering journals. For experimental engineering research—for example, in structural engineering or materials science—the model could be particularly valuable because it encourages rigorous pre‑study planning and reduces the temptation to cherry‑pick results. The journal Nature Communications has a dedicated track for registered reports, and several engineering‑focused publishers are exploring similar options. Widespread adoption would require cultural shifts among authors and reviewers, but the potential to improve reliability is significant.

Implications for Engineering Research and Practice

The reliability of peer review directly affects the real‑world trustworthiness of engineering knowledge. When flawed research passes peer review, the consequences can be disastrous. For instance, the collapse of the Walkway Bridge in 1981 was partially attributed to a design that had not undergone rigorous peer validation; a more thorough review might have caught the miscalculation. Similarly, retractions due to data fabrication in engineering journals have led to wasted research investment and eroded public confidence in safety‑critical technologies.

Beyond individual cases, systemic unreliability can distort entire research directions. If reviewers consistently favour incremental improvements over novel approaches, high‑risk, high‑reward projects may never get published, slowing the pace of innovation. On the other hand, a lack of consistency can allow substandard work to enter the literature, forcing later researchers to spend time reproducing and correcting errors. In fields such as software engineering, where replication studies are rare, erroneous claims can persist for years.

Reproducibility initiatives in engineering are gaining momentum. Many funding agencies and journals now require authors to deposit data and code in public repositories. Reviewers are encouraged to check that the shared materials are complete and that the claimed results can be reproduced. Yet without a reliable peer‑review process that enforces these requirements, policies remain aspirational. Journals that implement mandatory data‑ and code‑sharing along with verification checks during review are likely to see higher reproducibility rates—and greater credibility.

Ultimately, the engineering community must recognize that peer review is a human system, inherently imperfect, but one that can be systematically improved. Investment in reviewer training, transparent editorial policies, and innovative review models is not a luxury—it is a necessity for maintaining the trust that society places in engineering research.

Conclusion: Toward a More Robust Peer Review Ecosystem

Peer review remains the bedrock of editorial decision‑making in engineering journals, but its reliability is far from guaranteed. Bias, inconsistency, lack of transparency, and time pressures all threaten the integrity of the process. The good news is that many promising reforms are already being tested: open review, structured checklists, registered reports, and increased use of preprint feedback. By adopting a combination of these strategies, journals can reduce the influence of subjective factors and make editorial decisions more transparent and reproducible.

Authors, reviewers, editors, and publishers each have a role to play. Authors can support reliability by submitting well‑prepared manuscripts and openly sharing data and code. Reviewers can invest time into producing thorough, constructive evaluations and can make use of training resources. Editors can enforce submission standards and diversify reviewer pools to counter bias. Publishers can provide technological infrastructure for transparent review and reward reviewers with recognition or credits.

Efforts such as those by Retraction Watch and the Center for Open Science have already highlighted systemic issues and advocated for change. The engineering community must embrace these reforms, adapt them to the unique demands of engineering research, and continuously evaluate their impact. Only through persistent, evidence‑driven improvement can peer review truly serve as a reliable filter for the knowledge that shapes our built world.