Module 2: The Peer-Review Process

Overview and Learning Objectivees

One of the most important aspects of science is the sharing of research with peers. This allows for a critical review of the information being disseminated, expands the current scientific body of knowledge, provides the foundation needed to reproduce and verify findings, and sets the stage for future discoveries. Communication and collaboration within and between subdisciplines of science can be done by presenting research at scientific meetings or conferences, but this approach often only reaches the limited few who are present. Instead, most research is evaluated and disseminated through peer-reviewed journal articles.

"Scenes from the 2014 Biennial Scientific Conference, Mammoth Hot Springs" by gyeconference is marked with Public Domain Mark 1.0.

Peer-reviewed articles are scientific articles that are reviewed by other scientists, or qualified peers. These peers are often experts in the same research area, who determine whether or not the work done is suitable for publication. Grant proposals, which are requests for research funding, are also subject to peer review. The process of peer review helps to ensure that the research described in a scientific paper or grant proposal is significant, logical, thorough, and scientifically sound following appropriate experimental design methodology. Peer-reviewers also verify that the experimental results and conclusions are consistent with the data presented and the findings of other scientists.

There are many journals, as well as the popular press, that do not use a peer-review system. Whether from paid journals or open-access journals (i.e. journals with articles available without cost), it is important to verify that a journal uses a rigorous peer-review system, as not all do. Results of any studies published in these forums without peer review are not reliable and should not form the basis for other scientific work.

In this module, you will learn more about the steps involved in the peer-review process, who qualifies to peer-review scientific research, what is specifically reviewed, and how the information presented is deemed credible. You will then apply what you’ve learned from Module 1 and Module 2 to review your peers’ scientific experiment developed in the Discussion Forum.

After completing module 2, you should be able to:

identify the steps of the peer review process;
describe who qualifies to peer review scientific research
describe what reviewers look for when reviewing a manuscript
evaluate an experimental design based on your knowledge of the scientific method

Steps of the Peer-Review Process

Video length (2 min. 19 sec). "What are Peer-Reviewed Articles" by "Samchap" is licensed under CC-BY. Video transcript can be obtained here.

In summary, the scientific peer review process typically involves the following steps. First, a researcher submits their manuscript to a scientific journal for consideration. The journal's editor then conducts an initial evaluation to determine if the paper meets the publication's scope and standards. If deemed suitable, the editor assigns the manuscript to expert reviewers (usually three) in the field who evaluate the paper's methodology, results, and conclusions for appropriateness, accuracy, and validity. Reviewers provide constructive feedback, identify strengths and weaknesses, and may request revisions or clarifications. The author addresses the reviewers' comments and revises the manuscript accordingly. The revised version undergoes another round of review or may be accepted for publication based on the initial reviews. If rejected at any stage, the author starts the process over again, either by rewriting their manuscript or submitting to a new journal. Ultimately, the editor makes the final decision on whether to accept, reject, or request further revisions before publication. The peer review process aims to ensure the quality, credibility, and integrity of scientific research before it is disseminated to the broader scientific community and the public.

Who Qualifies to Peer-Review a Scientific Manuscript?

Qualified individuals who possess expertise in the specific field of study covered by the scientific article are typically invited to peer review the manuscript. These individuals are often researchers, scholars, or practitioners who have a deep understanding of the subject matter and have published their own research in reputable journals. They may hold academic positions, such as professors or researchers, and have a track record of conducting high-quality research in the field. Reviewers are commonly selected based on their expertise, reputation, and prior publications in the relevant area. The aim is to have reviewers who can critically evaluate the methodology, results, and conclusions of the manuscript and provide valuable feedback to enhance the quality and credibility of the research. It is important that reviewers have no conflicts of interest that could compromise the objectivity and fairness of the review process.

What Do Reviewers Look for When Reviewing a Manuscript?

Peer reviewers look for several key aspects when reviewing a scientific article. These include:

Methodology: Reviewers evaluate the soundness and appropriateness of the study design, data collection methods, replication, and statistical analysis. They assess whether the methods are rigorous, transparent, and capable of answering the research question.
Validity of Results: Reviewers examine the data presented in the article to assess the validity and reliability of the findings. They determine if the analyses are appropriate, assess whether the results support the claims made by the authors, and determine if any alternative explanations or confounding factors have been adequately addressed.
Significance and Originality: Reviewers consider the novelty and importance of the research. They assess whether the study contributes new knowledge to the field and whether the findings have broader implications or applications. Reviewers also evaluate that the manuscript does not make claims larger than what the data supports.
Clarity and Coherence: Reviewers assess the overall clarity, organization, and coherence of the manuscript. They check whether the data tables, figures, and writing is clear, concise, and well-structured. Reviewers also look for appropriate use of terminology and assess whether the article effectively communicates the research methods, results, and conclusions.
Ethical Considerations: Reviewers ensure that the research has been conducted ethically and adheres to relevant guidelines. They look for proper consent, appropriate treatment of human or animal subjects, and disclosure of potential conflicts of interest.
References and Citations: Reviewers verify the accuracy and completeness of the article's references, ensuring that relevant prior work is properly cited and are credible. They also assess whether the authors have appropriately acknowledged the contributions of other researchers in the field.

Reviewers provide constructive feedback to the authors, highlighting strengths and weaknesses, and suggest improvements or clarifications where necessary. The goal is to ensure the quality, validity, and integrity of the research being presented.

Determining the Credibility of a Source

Determining the credibility of a source involves considering several factors, many of which are highlighted here. However, scientific articles have more stringent guidelines, requiring that sources derive from the primary literature, and are high quality, recent, relevant, and unbiased. Primary literature refers to original research articles or scholarly papers that present the results of scientific studies or investigations. It is the firsthand account of research conducted by the authors and is typically published in peer-reviewed scientific journals or presented at conferences. In one exception, journals may allow a researcher to cite a personal communication from another researcher about unpublished results with the cited author's permission.

Scientific peer-reviewers evaluate the credibility of sources within primary literature by assessing several factors. Here are some key considerations:

Relevance: Reviewers examine the relevance of the cited sources to the research topic. They assess whether the chosen references are directly related to the research question, methodology, or results presented in the manuscript. Relevant sources strengthen the credibility and support the arguments made in the paper.
Source Quality: Reviewers evaluate the quality of the cited sources. They consider the reputation and credibility of the journals, conferences, or publishing venues where the sources are published. Well-established, peer-reviewed journals with rigorous review processes are generally deemed more credible. Reviewers also assess the expertise and reputation of the authors of the cited sources.
Currency: Reviewers consider the recency of the cited sources. They look for the most up-to-date research and evidence in the field. Reviewers assess whether the author has cited recent studies and if they have appropriately considered any new findings or developments that may impact the research.
Consistency and Reproducibility: Reviewers assess the consistency and reproducibility of the cited sources. They evaluate if the claims or findings made in the manuscript are supported by other studies and if there is a consensus within the scientific community. Reviewers may cross-reference the cited sources to determine if the results have been replicated or if there are any conflicting findings.
Methodological Soundness: Reviewers evaluate the methodology of the cited sources. They assess if the research design, data collection methods, and statistical analyses used in the referenced studies are robust and appropriate. Reviewers consider whether the methods employed are capable of generating reliable and valid results.
Bias and Conflicts of Interest: Reviewers assess potential bias or conflicts of interest in the cited sources. They scrutinize if the authors of the referenced studies have any conflicts of interest that may affect the objectivity or credibility of their work. Reviewers also look for any indication of bias, such as funding sources or affiliations that could influence the research findings.

By considering these factors, peer-reviewers can evaluate the credibility and reliability of the sources cited in the primary literature. Their assessment helps ensure that the research is built upon a strong foundation of credible and relevant scientific evidence.

Module 2 Assignment:

Now that you've learned a little bit more about the peer-review process, you have the chance to apply that knowledge in the Module 1 discussion forum. For this assignment, respond to two peers posts by constructively peer-reviewing their experimental design. When reviewing consider the following:

Did the peer provide enough background information to support why they are testing the hypothesis?
Was the hypothesis clearly written as a statement?
Were the methods appropriately described? Does the scientific method used make sense given the hypothesis? Was there sufficient information included in the methodology for someone to replicate this experiment?
Were the groups and variables correctly identified?
Do the conclusions make sense based on the data presented? What does the evidence show? Does it match what they described? Did the authors overstate their findings? Was the hypothesis supported by the data?
If sources were used, were they credible?
Was the paper written clearly?

Go to the Discussion