article reviewer guidelines

• Reviewer Guidelines
• Peer review model
• Scope & article eligibility
• Reviewer eligibility
• Peer reviewer code of conduct
• Guidelines for reviewing
• How to submit
• The peer-review process
• Benefits for Reviewers

Guidelines for Article Reviewers

• Authors are eligible to publish - authors must be formally affiliated with Wellcome funding.
• Article types – articles are checked whether they meet the criteria and format of specific article types .
• Readability – as we do not copy edit articles, the standard of language and readability must be sufficient for readers to be able to follow the article.
• Plagiarism – articles are checked for plagiarism before publication.
• Methods section – we check that details of methods and resources are provided, so that the work can be assessed (we will ask you as an expert reviewer to comment whether more information would be required for others to reproduce the work).
• Policies – we check that articles publishing research involving humans or animals adhere to our ethical policies .
• Data – we check that the source data underlying the results are made openly available (we will ask you as an expert reviewer to comment whether the source data are appropriate for others to reproduce the work).
• Read the article fully – please read the full text of the article and view all associated figures, tables and data;
• Be thorough – a peer review report should discuss the article in full as well as individual points, and should demonstrate your understanding of the article;
• Be specific – your comments should contain as much detail as possible, with references where appropriate, so the authors are able to fully address the issue;
• Be constructive in your criticism – do not hesitate to include any concerns or criticisms you may have in your review, however, please do so in a constructive and respectful manner;
• Avoid derogatory comments or tone – review as you wish to be reviewed and ensure that your comments focus on the scientific content of the article in question rather than the authors themselves.
• Approved: No or only minor changes are required. For original research, this means that the experimental design, including controls and methods, is adequate; results are presented accurately and the conclusions are justified and supported by the data.
• Approved with Reservations: The reviewer believes the paper has academic merit, but has asked for a number of small changes to the article, or specific, sometimes more significant revisions.
• Not Approved: The article is of very poor quality and there are fundamental flaws in the article that seriously undermine the findings and conclusions.
• Is the work clearly and accurately presented and does it cite the current literature?
• Is the study design appropriate and does the work have academic merit?
• Are sufficient details of methods and analysis provided to allow replication by others?
• If applicable, is the statistical analysis and its interpretation appropriate?
• Are all the source data underlying the results available to ensure full reproducibility?
• Are the conclusions drawn adequately supported by the results?
• Are the rationale for, and objectives of, the Systematic Review clearly stated?
• Are sufficient details of the methods and analysis provided to allow replication by others?
• Is the statistical analysis and its interpretation appropriate?
• Are the conclusions drawn adequately supported by the results presented in the review?
• Is the rationale for developing the new software tool clearly explained?
• Is the description of the software tool technically sound?
• Are sufficient details of the code, methods and analysis (if applicable) provided to allow replication of the software development and its use by others?
• Is sufficient information provided to allow interpretation of the expected output datasets and any results generated using the tool?
• Is the rationale for developing the new method (or application) clearly explained?
• Is the description of the method technically sound?
• Are sufficient details provided to allow replication of the method development and its use by others?
• If any results are presented, are all the source data underlying the results available to ensure full reproducibility?
• Are the conclusions about the method and its performance adequately supported by the findings presented in the article?
• Is the rationale for creating the dataset(s) clearly described?
• Are the protocols appropriate and is the work technically sound?
• Are sufficient details of methods and materials provided to allow replication by others?
• Are the datasets clearly presented in a useable and accessible format?
• Is the rationale for, and objectives of, the study clearly described?
• Is the study design appropriate for the research question?
• Are sufficient details of the methods provided to allow replication by others?
• Is the background of the case’s history and progression described in sufficient detail?
• Are enough details provided of any physical examination and diagnostic tests, treatment given and outcomes?
• Is sufficient discussion included of the importance of the findings and their relevance to future understanding of disease processes, diagnosis or treatment?
• Is the case presented with sufficient detail to be useful for other practitioners?
• Is the background of the cases’ history and progression described in sufficient detail?
• Is the conclusion balanced and justified on the basis of the findings?
• Is the rationale for commenting on the previous publication clearly described?
• Are any opinions stated well-argued, clear and cogent?
• Are arguments sufficiently supported by evidence from the published literature or by new data and results?
• Is the conclusion balanced and justified on the basis of the presented arguments?
• Is the rationale for the Open Letter provided in sufficient detail?
• Does the article adequately reference differing views and opinions?
• Are all factual statements correct, and are statements and arguments made adequately supported by citations?
• Is the Open Letter written in accessible language?
• Where applicable, are recommendations and next steps explained clearly for others to follow?
• Is the study design appropriate for the research question (including statistical power analysis, where appropriate)?
• Have the authors pre-specified sufficient outcome-neutral tests for ensuring that the results obtained can test the stated hypotheses, including positive controls and quality checks?
• Are the data able to test the authors’ proposed hypotheses by satisfying the approved outcome-neutral conditions (such as quality checks, positive controls)?
• Are the introduction, rationale and stated hypotheses the same as the approved Stage 1 submission? (required)
• Did the authors adhere precisely to the registered experimental procedures? If not, has an explanation been provided regarding any change?
• Are any unregistered post hoc analyses added by the authors justified, methodologically sound and informative?

Are you a Wellcome-funded researcher?

If you are a previous or current Wellcome grant holder, sign up for information about developments, publishing and publications from Wellcome Open Research.

We'll keep you updated on any major new updates to Wellcome Open Research

The email address should be the one you originally registered with F1000.

You registered with F1000 via Google, so we cannot reset your password.

To sign in, please click here .

If you still need help with your Google account password, please click here .

You registered with F1000 via Facebook, so we cannot reset your password.

If you still need help with your Facebook account password, please click here .

If your email address is registered with us, we will email you instructions to reset your password.

If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance.

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Guidelines for Reviewers

PLOS ONE  relies on members of the scientific research community to assess the validity of articles under consideration through peer review.

Invitation to Review

PLOS ONE editors select potential reviewers based on their expertise in research areas relevant to the manuscript under consideration. Reviewer invitations are sent by email from the journal's Editorial Manager submission system. Use the links in the invitation email to accept or decline, or check the “New Reviewer Invitations” folder on your Reviewer Main Menu screen in Editorial Manager. Accept an invitation only if you have the knowledge, time and objectivity necessary to provide an unbiased assessment of the research. In agreeing to complete a review, you also give permission to publish your review under a Creative Commons Attribution CC BY license.

Our Peer Review Process

PLOS ONE practices single-anonymized peer review by default, but offers opportunities for authors and reviewers to participate in signed and published peer review.

Our peer review model

Best practices for reviewers, receiving credit.

In choosing to volunteer as a peer reviewer for PLOS, you are helping to support free and open access to rigorous research. We couldn’t be more grateful!

Submit Your Review Now

Learn about peer review, how to submit a peer review in editorial manager.

PLOS ONE uses a structured reviewer form to help reviewers focus on our publication criteria and improve the efficiency of peer review. Preview the form . The form contains two sections:

Comments to the author

Answers to the questions in this section are required and will be included in the decision letter to the author. For questions 1-4 select a response from the drop down (e.g., “Yes,” “No,” “I don't know,” “N/A”) and provide any details you wish. Enter the main text of your peer review in question 5, “Review Comments to the Author.”

Comments to the editor

Use this section to declare any potential or perceived competing interests. You’ll also have the option to list anyone who collaborated with you on the peer review, and indicate whether you think the submission should be highlighted on the PLOS ONE webpage if it goes on to be published. This will not play any role in the editorial decision-making process or be shared with the authors.

PLOS ONE does not allow confidential comments other than the declaration of competing interests. If you have confidential concerns relating to publication or research ethics, please contact us at [email protected] .

What to Assess

To be eligible for publication manuscripts must meet our criteria for publication and comply with our editorial and publishing policies . The following guidance relates to articles reporting the results of original research.

Criteria for publication

Unlike many journals which attempt to use the peer review process to determine whether or not an article reaches the level of 'importance' required by a given journal, PLOS ONE uses peer review to determine whether a paper is technically rigorous and meets the scientific and ethical standard for inclusion in the published scientific record.

Please carefully review our seven editorial criteria for publication as the criteria employed by PLOS ONE are quite different to other journals you may have reviewed for.

To be accepted for publication in PLOS ONE , research articles must satisfy the following criteria:

• The study presents the results of primary scientific research .
• Results reported have not been published elsewhere .
• Experiments, statistics, and other analyses are performed to a high technical standard and are described in sufficient detail.
• Conclusions are presented in an appropriate fashion and are supported by the data .
• The article is presented in an intelligible fashion and is written in standard English .
• The research meets all applicable standards for the ethics of experimentation and research integrity .
• The article adheres to appropriate reporting guidelines and community standards for data availability .

Reviewing Registered Reports

Registered Reports are primary research articles in which the methods and proposed analyses are peer reviewed prior to conducting experiments, data collection or analysis. The PLOS ONE publication criteria apply to Registered Reports as they would to any other research submitted to the journal, but the peer review process is slightly different. Assessment takes place in two stages and, if accepted, results in two linked publications.

Reviewing Lab Protocols

Lab Protocols describe reusable methodologies for experimental and computational techniques.They typically consist of a protocol on the protocols.io platform and a  PLOS ONE  manuscript that contextualizes it, but authors can elect to publish on protocols.io after manuscript submission.

The PLOS ONE  publication criteria apply to Lab Protocols as they would to any other research submitted to the journal, but the peer review process is generally expedited and typically carried out by one internal Academic Editor and one external reviewer.

Lab Protocols are eligible for both signed and published peer review. 

You will review the manuscript, while referencing the protocol on protocols.io or in PDF format as a supplementary information file. 

Peer review aims to ensure that the manuscript complies with the submission guidelines and publication criteria for Lab Protocols.

Consider: 

• Is the protocol of utility to the research community?
• Does it link to a protocol on protocols.io and is this attached in PDF format as a SI file?
• Are the methods and reagents described sufficiently?
• Are the controls and sample sizes appropriate?
• Do the authors demonstrate that the method is validated, either by linking to at least one supporting peer-reviewed publication, or by providing appropriate supporting data?
• If the manuscript contains new data, have the authors made this data fully available?

Reviewing Study Protocols

Study Protocols describe detailed plans and proposals for research projects that have not yet generated results. They consist of a single article on PLOS ONE that can be referenced in future research.

The PLOS ONE publication criteria apply to Study Protocols as they would to any other research submitted to the journal. Study Protocols are eligible for both signed and published peer review.

Study Protocols submitted with proof of ethics approval (if required) and external funding by a funder that has independently peer reviewed the protocol are typically accepted without further external peer review. If, however, the journal staff decide that further review is necessary, reviewers will be invited to ensure that the Study Protocol complies with the submission guidelines and publication criteria

• The protocol prerequisites: the research study should not have generated results, nor should participant recruitment or data collection be complete.
• Are the required ethical standards met?
• Does the manuscript provide valid rationale for the planned or ongoing study, with clearly identified and justified research questions?
• Is the protocol technically sound and planned in a manner that will lead to a meaningful outcome and allow testing of the stated hypotheses?
• Have the authors described where all data underlying the findings will be made available when the study is complete? 
• Does the manuscript title contain the word “Protocol”?
• Is the methodology feasible and does the description provide sufficient methodological detail for the protocol to be reproduced and replicated?
• Are any recommended checklists provided as SI files?
• Have the authors registered on a research platform that is appropriate for the study type, like OSF, for example?
• For clinical trials, is the trial registered and has the registration number been provided? Has the author provided the  required SI files?
• For systematic reviews and meta-analyses, is the PRISMA-P checklist provided and complete? Have the authors opted to register with PROSPERO?

Writing the review

The purpose of the review is to provide the editors with an expert opinion regarding the validity and quality of the manuscript under consideration. The review should also supply authors with explicit feedback on how to improve their papers so that they will be acceptable for publication in PLOS ONE . As you write, consider the following points:

• What are the main claims of the paper and how significant are they for the discipline?
• Are the claims properly placed in the context of the previous literature? Have the authors treated the literature fairly?
• Do the data and analyses fully support the claims? If not, what other evidence is required?
• PLOS ONE encourages authors to publish detailed protocols and algorithms as supporting information online. Do any particular methods used in the manuscript warrant such treatment? If a protocol is already provided, for example for a randomized controlled trial, are there any important deviations from it? If so, have the authors explained adequately why the deviations occurred?
• If the paper is considered unsuitable for publication in its present form, does the study itself show sufficient potential that the authors should be encouraged to resubmit a revised version?
• Are original data deposited in appropriate repositories and accession/version numbers provided for genes, proteins, mutants, diseases, etc.?
• Does the study conform to any relevant guidelines such as CONSORT, MIAME, QUORUM, STROBE, and the Fort Lauderdale agreement ?
• Are details of the methodology sufficient to allow the experiments to be reproduced?
• Is any software created by the authors freely available?
• Is the manuscript well organized and written clearly enough to be accessible to non-specialists?
• Is it your opinion that this manuscript contains an NIH-defined experiment of Dual Use concern ?

Although confidential comments to the editors are respected, any remarks that might help to strengthen the paper should be directed to the authors themselves.

We often ask the original reviewers to evaluate revised manuscripts and the authors’ response to reviewer comments. We hope that you’ll make yourself available for re-review and questions from the editors.

Editing reviewers’ reports

The editors and PLOS staff will not change any reviewer comments that are intended for authors to read, except with reviewer approval prior to the decision letter being sent. For example, we may request changes if language is deemed inappropriate for professional communication, or if the comments contain information considered confidential, such as competing interest declarations.

The Editorial Process

Decision process.

The editors make the final decision on whether to publish each submission based on the reviewers’ comments, the PLOS ONE   criteria for publication , and their own assessment of the manuscript.

Conflicting reviews

If reviewers appear to disagree fundamentally, the editors may choose to share all the reviews with each of the reviewers and request additional comments that may help the editors to reach a decision. Decisions are not necessarily made according to majority rule. Experts may disagree, and it is the job of the Editor to make a decision. Editors evaluate reviewer recommendations and comments alongside comments by the authors and material that may not have been made available to reviewers. Please know that your recommendation has been duly considered and your service is appreciated, even if the final decision does not accord with your review.

More on our Editorial Process .

For more tips on peer review go to the Peer Review Center , and sign up to the Peer Review Toolbox .

If you have questions or concerns about the manuscript you are reviewing, or if you need assistance submitting the review, please email us [email protected] .

Peer Review General and Ethical Guidelines

The fundamentals of ethical reviewing.

Take advantage of the industry's most comprehensive publishing ethics guidelines.

We believe that ethical publishing leads to a better research community, where everyone is valued and everyone is responsible for the work they do. Wiley's Best Practice Guidelines on Publishing Ethics: A Publisher's Perspective, Second Edition , is widely acknowledged as the industry's most comprehensive publishing ethics guidance.

Read the Wiley Guidelines on editorial standards and processes, including peer review.

COPE guidelines for peer reviewers

Wiley provides membership of the Committee on Publication Ethics (COPE) as an option for our journal editors. COPE serves more than 8,500 members around the world with practical tools, e-learning, seminars, and much more. COPE has developed Ethical Guidelines for Peer Reviewers, to which editors and their editorial boards can refer for guidance. Read the COPE guidelines on their webpage, Ethical Guidelines for Peer Reviewers .

Additional Resources

Visit our peer review training resources for expert advice on peer review.

• Technical Support
• Find My Rep

You are here

How to review articles.

Become a reviewer | Things to consider | What to include  | Your recommendation | Webinar | Web of Science Academy

Peer review is essential for filtering out poor quality articles by assessing the validity and integrity of the research. We value the work done by peer reviewers in the academic community, who facilitate the process of publication and drive research within their fields of expertise. Please visit the Reviewer Rewards page to learn more about discounts and free journal access offered to reviewers of articles for Sage journals.

If you are an inexperienced or first-time reviewer, the peer review process may seem daunting. In fact, peer review can be a very rewarding process that allows you to contribute to the development of your field and hone your own research and writing skills. The resources below will explain what peer review involves and help you to write useful reviews.

How to become a reviewer

There are three ways to register as a reviewer.

1. Create a journal-specific reviewer account on Sage Track. Search for the journal’s name here and then click the ‘Submit paper’ link. This will take you to the peer review system where you can create an account.

Why create an account on a specific journal Sage Track site?  

• You will be part of the journal’s reviewer database.
• You can make your profile more attractive by adding keywords related to your areas of expertise to boost your chances of being invited to review.
• Editors can rate your reviews which may increase your chances of being invited to review again.

2. Contact the journal editor or editorial office directly. In your communication, express your interest, summarize your expertise, and present yourself as a valuable reviewer.

Why contact the editor or editorial office directly?   

• If you’re unsure if your area of expertise fits the journal's scope you can double-check with the editor or editorial office directly before registering your details on the Web of Science or the journal’s Sage Track site.  
• Editors get a firsthand look at your experience and expertise.  
• This is an opportunity for you to introduce yourself and your interests to the editor to increase your chances of being invited to review.  

3. Sign up to the Web of Science Reviewer Recognition Service . Indicate your interest in reviewing for a journal by clicking on the journal’s Reviewer Recognition page. Editors use Reviewer Recognition to find suitable reviewers for their journals and may contact you directly via the Reviewer Recognition site.

Why sign up to WoS Reviewer Recognition?  

• Editors can clearly see your interest in reviewing for their journal.  
• Editors can look into your reviewer insights, including which other journals you are reviewing for.
• Editors can rate reviewers as ‘excellent’.
• You can review your satisfaction of Reviewer Recognition.
• You can receive weekly email updates summarizing your reviewer activity on the site. 
• For Sage Journals, a claimed review will result in the individual’s name being listed as a reviewer for that journal. 
• Co-reviewers can also receive credit on Reviewer Recognition.

Once you are registered as a reviewer, the editors will send you an invitation to review if a manuscript in your area of expertise is submitted.

We understand that our reviewers are busy, so it may not always be possible for you to accept an invitation to review. To avoid delays, please inform the editor as soon as possible if you are unable to accept an invitation to review or encounter any issues after accepting. If you cannot review a manuscript, we appreciate if you can suggest an alternative reviewer.

Tip: To further increase your chances of receiving review invitations, see our comprehensive blog post on  Steps to Strengthen Your Reviewer Profile .

Watch the Video Tutorial

Things to consider before you begin a review

• Timing Inform the editor immediately if you will not be able to meet the deadline and keep your availability updated in Sage Track to avoid receiving invitations to review when you are unavailable.
• Suitability Do you have any reason why you should not review the submission? If in doubt, check with the journal’s editor. Learn more about your ethical responsibilities as a reviewer.
• Individual Journal Reviewer Guidelines Some journals ask reviewers to answer specific questions, so check what is expected before beginning your review.
• Confidentiality You must not share the content of a paper you have been invited to review, unless you have permission from the journal’s editor. If you suspect that author misconduct has taken place, only discuss this with the editor.
• Co-reviewing Inform the journal editor if you wish to collaborate on a review with a colleague or student. See the Ethics and Responsibility page for further instructions on this.

You may also wish to refer to COPE’s guide on what to consider when asked to peer review a manuscript before beginning a review .

What to include in a review

Watch our short video, How to Conduct a Peer Review , for a step-by-step walkthrough of the review process. Alternatively, you can download our Reviewer's Guide for written instructions on how to assess a manuscript and what to include in a review.

Watch the video tutorial

DOWNLOAD REVIEWER'S GUIDE PDF

Making your recommendation

In addition to your review comments, you will likely be expected to select an overall recommendation to the editor. Sage’s most common recommendation types are:

Accept:  No further revision required. The manuscript is publishable in its current form.

The majority of articles require revision before reaching this stage.

Minor Revision:  The paper is mostly sound but will be sent back to the authors for minor corrections and clarifications such as the addition of minor citations or the tweaking of arguments.

These revisions should not involve any major changes. However, changes should be clearly marked for the attention of the previous reviewers. The paper may be subject to re-review.

Major Revision:  The principle of the article is sound and it has a chance of being accepted but requires substantial change to be made. This may include further experiments or analysis, the inclusion of additional literature or theory, or an improvement of arguments and conclusions. The authors are required to submit a point-by-point response to the reviewers and the paper will be subject to a re-review. 

If issues of quality, novelty and/or contribution* cannot be addressed through revision, the reviewer should recommend rejection rather than revision. Editors withhold the right to reject the paper should revisions be insufficient.

Reject:  The manuscript is of insufficient quality, novelty or significance to warrant publication.

Even when recommending rejection, the reviewer is encouraged to share their suggestions for improvement in the Comments to the Authors field.

If you would like to give us feedback on your experience of reviewing for a Sage journal to help us to improve our systems, please contact [email protected] .

*Check the journal Aims and Scope for any specific requirements with regard to levels of novelty and/or contribution.  

How to Be a Peer Reviewer webinar

Considering becoming a reviewer or getting more involved with peer review? Our free webinar will guide you through the process of conducting peer review, including how to get started, basic principles of reviewing articles, what journal editors expect from reviewers, and important considerations such as research ethics and reviewer responsibilities. Learn more here .   

Learn more with the Web of Science Academy

The Web of Science Academy offers free-of-charge short courses providing researchers with the skills and experience required to become an expert peer reviewer. Courses cover:

• What's expected of you as a reviewer
• What to look for in a manuscript
• How to write a review
• Co-reviewing with a mentor

A certificate is awarded on completion of the course.

• Journal Author Gateway
• Journal Editor Gateway
• What is Peer Review?
• How to Be a Peer Reviewer Webinar
• How to Review Plain Language Summaries
• How to Review Registered Reports
• Using Sage Track
• Peer Review Ethics
• Resources for Reviewers
• Reviewer Rewards
• Ethics & Responsibility
• Sage Editorial Policies
• Publication Ethics Policies
• Sage Chinese Author Gateway 中国作者资源
• Open Resources & Current Initiatives
• Discipline Hubs

Get Started

Take the first step and invest in your future.

Online Programs

Offering flexibility & convenience in 51 online degrees & programs.

Prairie Stars

Featuring 15 intercollegiate NCAA Div II athletic teams.

Find your Fit

UIS has over 85 student and 10 greek life organizations, and many volunteer opportunities.

Arts & Culture

Celebrating the arts to create rich cultural experiences on campus.

Give Like a Star

Your generosity helps fuel fundraising for scholarships, programs and new initiatives.

Bragging Rights

UIS was listed No. 1 in Illinois and No. 3 in the Midwest in 2023 rankings.

• Quick links Applicants & Students Important Apps & Links Alumni Faculty and Staff Community Admissions How to Apply Cost & Aid Tuition Calculator Registrar Orientation Visit Campus Academics Register for Class Programs of Study Online Degrees & Programs Graduate Education International Student Services Study Away Student Support Bookstore UIS Life Dining Diversity & Inclusion Get Involved Health & Wellness COVID-19 United in Safety Residence Life Student Life Programs UIS Connection Important Apps UIS Mobile App Advise U Canvas myUIS i-card Balance Pay My Bill - UIS Bursar Self-Service Email Resources Bookstore Box Information Technology Services Library Orbit Policies Webtools Get Connected Area Information Calendar Campus Recreation Departments & Programs (A-Z) Parking UIS Newsroom The Observer Connect & Get Involved Update your Info Alumni Events Alumni Networks & Groups Volunteer Opportunities Alumni Board News & Publications Featured Alumni Alumni News UIS Alumni Magazine Resources Order your Transcripts Give Back Alumni Programs Career Development Services & Support Accessibility Services Campus Services Campus Police Facilities & Services Registrar Faculty & Staff Resources Website Project Request Web Services Training & Tools Academic Impressions Career Connect CSA Reporting Cybersecurity Training Faculty Research FERPA Training Website Login Campus Resources Newsroom Campus Calendar Campus Maps i-Card Human Resources Public Relations Webtools Arts & Events UIS Performing Arts Center Visual Arts Gallery Event Calendar Sangamon Experience Center for Lincoln Studies ECCE Speaker Series Community Engagement Center for State Policy and Leadership Illinois Innocence Project Innovate Springfield Central IL Nonprofit Resource Center NPR Illinois Community Resources Child Protection Training Academy Office of Electronic Media University Archives/IRAD Institute for Illinois Public Finance

Request Info

How to Review a Journal Article

• Request Info Request info for....     Undergraduate/Graduate     Online     Study Away     Continuing & Professional Education     International Student Services     General Inquiries

For many kinds of assignments, like a  literature review , you may be asked to offer a critique or review of a journal article. This is an opportunity for you as a scholar to offer your  qualified opinion  and  evaluation  of how another scholar has composed their article, argument, and research. That means you will be expected to go beyond a simple  summary  of the article and evaluate it on a deeper level. As a college student, this might sound intimidating. However, as you engage with the research process, you are becoming immersed in a particular topic, and your insights about the way that topic is presented are valuable and can contribute to the overall conversation surrounding your topic.

IMPORTANT NOTE!!

Some disciplines, like Criminal Justice, may only want you to summarize the article without including your opinion or evaluation. If your assignment is to summarize the article only, please see our literature review handout.

Before getting started on the critique, it is important to review the article thoroughly and critically. To do this, we recommend take notes,  annotating , and reading the article several times before critiquing. As you read, be sure to note important items like the thesis, purpose, research questions, hypotheses, methods, evidence, key findings, major conclusions, tone, and publication information. Depending on your writing context, some of these items may not be applicable.

Questions to Consider

To evaluate a source, consider some of the following questions. They are broken down into different categories, but answering these questions will help you consider what areas to examine. With each category, we recommend identifying the strengths and weaknesses in each since that is a critical part of evaluation.

Evaluating Purpose and Argument

• How well is the purpose made clear in the introduction through background/context and thesis?
• How well does the abstract represent and summarize the article’s major points and argument?
• How well does the objective of the experiment or of the observation fill a need for the field?
• How well is the argument/purpose articulated and discussed throughout the body of the text?
• How well does the discussion maintain cohesion?

Evaluating the Presentation/Organization of Information

• How appropriate and clear is the title of the article?
• Where could the author have benefited from expanding, condensing, or omitting ideas?
• How clear are the author’s statements? Challenge ambiguous statements.
• What underlying assumptions does the author have, and how does this affect the credibility or clarity of their article?
• How objective is the author in his or her discussion of the topic?
• How well does the organization fit the article’s purpose and articulate key goals?

Evaluating Methods

• How appropriate are the study design and methods for the purposes of the study?
• How detailed are the methods being described? Is the author leaving out important steps or considerations?
• Have the procedures been presented in enough detail to enable the reader to duplicate them?

Evaluating Data

• Scan and spot-check calculations. Are the statistical methods appropriate?
• Do you find any content repeated or duplicated?
• How many errors of fact and interpretation does the author include? (You can check on this by looking up the references the author cites).
• What pertinent literature has the author cited, and have they used this literature appropriately?

Following, we have an example of a summary and an evaluation of a research article. Note that in most literature review contexts, the summary and evaluation would be much shorter. This extended example shows the different ways a student can critique and write about an article.

Chik, A. (2012). Digital gameplay for autonomous foreign language learning: Gamers’ and language teachers’ perspectives. In H. Reinders (ed.),  Digital games in language learning and teaching  (pp. 95-114). Eastbourne, UK: Palgrave Macmillan.

Be sure to include the full citation either in a reference page or near your evaluation if writing an  annotated bibliography .

In Chik’s article “Digital Gameplay for Autonomous Foreign Language Learning: Gamers’ and Teachers’ Perspectives”, she explores the ways in which “digital gamers manage gaming and gaming-related activities to assume autonomy in their foreign language learning,” (96) which is presented in contrast to how teachers view the “pedagogical potential” of gaming. The research was described as an “umbrella project” consisting of two parts. The first part examined 34 language teachers’ perspectives who had limited experience with gaming (only five stated they played games regularly) (99). Their data was recorded through a survey, class discussion, and a seven-day gaming trial done by six teachers who recorded their reflections through personal blog posts. The second part explored undergraduate gaming habits of ten Hong Kong students who were regular gamers. Their habits were recorded through language learning histories, videotaped gaming sessions, blog entries of gaming practices, group discussion sessions, stimulated recall sessions on gaming videos, interviews with other gamers, and posts from online discussion forums. The research shows that while students recognize the educational potential of games and have seen benefits of it in their lives, the instructors overall do not see the positive impacts of gaming on foreign language learning.

The summary includes the article’s purpose, methods, results, discussion, and citations when necessary.

This article did a good job representing the undergraduate gamers’ voices through extended quotes and stories. Particularly for the data collection of the undergraduate gamers, there were many opportunities for an in-depth examination of their gaming practices and histories. However, the representation of the teachers in this study was very uneven when compared to the students. Not only were teachers labeled as numbers while the students picked out their own pseudonyms, but also when viewing the data collection, the undergraduate students were more closely examined in comparison to the teachers in the study. While the students have fifteen extended quotes describing their experiences in their research section, the teachers only have two of these instances in their section, which shows just how imbalanced the study is when presenting instructor voices.

Some research methods, like the recorded gaming sessions, were only used with students whereas teachers were only asked to blog about their gaming experiences. This creates a richer narrative for the students while also failing to give instructors the chance to have more nuanced perspectives. This lack of nuance also stems from the emphasis of the non-gamer teachers over the gamer teachers. The non-gamer teachers’ perspectives provide a stark contrast to the undergraduate gamer experiences and fits neatly with the narrative of teachers not valuing gaming as an educational tool. However, the study mentioned five teachers that were regular gamers whose perspectives are left to a short section at the end of the presentation of the teachers’ results. This was an opportunity to give the teacher group a more complex story, and the opportunity was entirely missed.

Additionally, the context of this study was not entirely clear. The instructors were recruited through a master’s level course, but the content of the course and the institution’s background is not discussed. Understanding this context helps us understand the course’s purpose(s) and how those purposes may have influenced the ways in which these teachers interpreted and saw games. It was also unclear how Chik was connected to this masters’ class and to the students. Why these particular teachers and students were recruited was not explicitly defined and also has the potential to skew results in a particular direction.

Overall, I was inclined to agree with the idea that students can benefit from language acquisition through gaming while instructors may not see the instructional value, but I believe the way the research was conducted and portrayed in this article made it very difficult to support Chik’s specific findings.

Some professors like you to begin an evaluation with something positive but isn’t always necessary.

The evaluation is clearly organized and uses transitional phrases when moving to a new topic.

This evaluation includes a summative statement that gives the overall impression of the article at the end, but this can also be placed at the beginning of the evaluation.

This evaluation mainly discusses the representation of data and methods. However, other areas, like organization, are open to critique.

• PRO Courses Guides New Tech Help Pro Expert Videos About wikiHow Pro Upgrade Sign In
• EDIT Edit this Article
• EXPLORE Tech Help Pro About Us Random Article Quizzes Request a New Article Community Dashboard This Or That Game Popular Categories Arts and Entertainment Artwork Books Movies Computers and Electronics Computers Phone Skills Technology Hacks Health Men's Health Mental Health Women's Health Relationships Dating Love Relationship Issues Hobbies and Crafts Crafts Drawing Games Education & Communication Communication Skills Personal Development Studying Personal Care and Style Fashion Hair Care Personal Hygiene Youth Personal Care School Stuff Dating All Categories Arts and Entertainment Finance and Business Home and Garden Relationship Quizzes Cars & Other Vehicles Food and Entertaining Personal Care and Style Sports and Fitness Computers and Electronics Health Pets and Animals Travel Education & Communication Hobbies and Crafts Philosophy and Religion Work World Family Life Holidays and Traditions Relationships Youth
• Browse Articles
• Learn Something New
• Quizzes Hot
• This Or That Game
• Train Your Brain
• Explore More
• Support wikiHow
• About wikiHow
• Log in / Sign up
• Education and Communications
• Critical Reviews

How to Write an Article Review (With Examples)

Last Updated: April 24, 2024 Fact Checked

Preparing to Write Your Review

Writing the article review, sample article reviews, expert q&a.

This article was co-authored by Jake Adams . Jake Adams is an academic tutor and the owner of Simplifi EDU, a Santa Monica, California based online tutoring business offering learning resources and online tutors for academic subjects K-College, SAT & ACT prep, and college admissions applications. With over 14 years of professional tutoring experience, Jake is dedicated to providing his clients the very best online tutoring experience and access to a network of excellent undergraduate and graduate-level tutors from top colleges all over the nation. Jake holds a BS in International Business and Marketing from Pepperdine University. There are 12 references cited in this article, which can be found at the bottom of the page. This article has been fact-checked, ensuring the accuracy of any cited facts and confirming the authority of its sources. This article has been viewed 3,100,459 times.

An article review is both a summary and an evaluation of another writer's article. Teachers often assign article reviews to introduce students to the work of experts in the field. Experts also are often asked to review the work of other professionals. Understanding the main points and arguments of the article is essential for an accurate summation. Logical evaluation of the article's main theme, supporting arguments, and implications for further research is an important element of a review . Here are a few guidelines for writing an article review.

Education specialist Alexander Peterman recommends: "In the case of a review, your objective should be to reflect on the effectiveness of what has already been written, rather than writing to inform your audience about a subject."

Article Review 101

• Read the article very closely, and then take time to reflect on your evaluation. Consider whether the article effectively achieves what it set out to.
• Write out a full article review by completing your intro, summary, evaluation, and conclusion. Don't forget to add a title, too!
• Proofread your review for mistakes (like grammar and usage), while also cutting down on needless information.

• Article reviews present more than just an opinion. You will engage with the text to create a response to the scholarly writer's ideas. You will respond to and use ideas, theories, and research from your studies. Your critique of the article will be based on proof and your own thoughtful reasoning.
• An article review only responds to the author's research. It typically does not provide any new research. However, if you are correcting misleading or otherwise incorrect points, some new data may be presented.
• An article review both summarizes and evaluates the article.

• Summarize the article. Focus on the important points, claims, and information.
• Discuss the positive aspects of the article. Think about what the author does well, good points she makes, and insightful observations.
• Identify contradictions, gaps, and inconsistencies in the text. Determine if there is enough data or research included to support the author's claims. Find any unanswered questions left in the article.

• Make note of words or issues you don't understand and questions you have.
• Look up terms or concepts you are unfamiliar with, so you can fully understand the article. Read about concepts in-depth to make sure you understand their full context.

• Pay careful attention to the meaning of the article. Make sure you fully understand the article. The only way to write a good article review is to understand the article.

• With either method, make an outline of the main points made in the article and the supporting research or arguments. It is strictly a restatement of the main points of the article and does not include your opinions.
• After putting the article in your own words, decide which parts of the article you want to discuss in your review. You can focus on the theoretical approach, the content, the presentation or interpretation of evidence, or the style. You will always discuss the main issues of the article, but you can sometimes also focus on certain aspects. This comes in handy if you want to focus the review towards the content of a course.
• Review the summary outline to eliminate unnecessary items. Erase or cross out the less important arguments or supplemental information. Your revised summary can serve as the basis for the summary you provide at the beginning of your review.

• What does the article set out to do?
• What is the theoretical framework or assumptions?
• Are the central concepts clearly defined?
• How adequate is the evidence?
• How does the article fit into the literature and field?
• Does it advance the knowledge of the subject?
• How clear is the author's writing? Don't: include superficial opinions or your personal reaction. Do: pay attention to your biases, so you can overcome them.

• For example, in MLA , a citation may look like: Duvall, John N. "The (Super)Marketplace of Images: Television as Unmediated Mediation in DeLillo's White Noise ." Arizona Quarterly 50.3 (1994): 127-53. Print. [9] X Trustworthy Source Purdue Online Writing Lab Trusted resource for writing and citation guidelines Go to source

• For example: The article, "Condom use will increase the spread of AIDS," was written by Anthony Zimmerman, a Catholic priest.

• Your introduction should only be 10-25% of your review.
• End the introduction with your thesis. Your thesis should address the above issues. For example: Although the author has some good points, his article is biased and contains some misinterpretation of data from others’ analysis of the effectiveness of the condom.

• Use direct quotes from the author sparingly.
• Review the summary you have written. Read over your summary many times to ensure that your words are an accurate description of the author's article.

• Support your critique with evidence from the article or other texts.
• The summary portion is very important for your critique. You must make the author's argument clear in the summary section for your evaluation to make sense.
• Remember, this is not where you say if you liked the article or not. You are assessing the significance and relevance of the article.
• Use a topic sentence and supportive arguments for each opinion. For example, you might address a particular strength in the first sentence of the opinion section, followed by several sentences elaborating on the significance of the point.

• This should only be about 10% of your overall essay.
• For example: This critical review has evaluated the article "Condom use will increase the spread of AIDS" by Anthony Zimmerman. The arguments in the article show the presence of bias, prejudice, argumentative writing without supporting details, and misinformation. These points weaken the author’s arguments and reduce his credibility.

• Make sure you have identified and discussed the 3-4 key issues in the article.

You Might Also Like

• ↑ https://libguides.cmich.edu/writinghelp/articlereview
• ↑ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4548566/
• ↑ Jake Adams. Academic Tutor & Test Prep Specialist. Expert Interview. 24 July 2020.
• ↑ https://guides.library.queensu.ca/introduction-research/writing/critical
• ↑ https://www.iup.edu/writingcenter/writing-resources/organization-and-structure/creating-an-outline.html
• ↑ https://writing.umn.edu/sws/assets/pdf/quicktips/titles.pdf
• ↑ https://owl.purdue.edu/owl/research_and_citation/mla_style/mla_formatting_and_style_guide/mla_works_cited_periodicals.html
• ↑ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4548565/
• ↑ https://writingcenter.uconn.edu/wp-content/uploads/sites/593/2014/06/How_to_Summarize_a_Research_Article1.pdf
• ↑ https://www.uis.edu/learning-hub/writing-resources/handouts/learning-hub/how-to-review-a-journal-article
• ↑ https://writingcenter.unc.edu/tips-and-tools/editing-and-proofreading/

About This Article

If you have to write an article review, read through the original article closely, taking notes and highlighting important sections as you read. Next, rewrite the article in your own words, either in a long paragraph or as an outline. Open your article review by citing the article, then write an introduction which states the article’s thesis. Next, summarize the article, followed by your opinion about whether the article was clear, thorough, and useful. Finish with a paragraph that summarizes the main points of the article and your opinions. To learn more about what to include in your personal critique of the article, keep reading the article! Did this summary help you? Yes No

• Send fan mail to authors

Reader Success Stories

Prince Asiedu-Gyan

Apr 22, 2022

Did this article help you?

Sammy James

Sep 12, 2017

Juabin Matey

Aug 30, 2017

Vanita Meghrajani

Jul 21, 2016

Nov 27, 2018

Featured Articles

Trending Articles

Watch Articles

• Terms of Use
• Privacy Policy
• Do Not Sell or Share My Info
• Not Selling Info

Don’t miss out! Sign up for

wikiHow’s newsletter

• Submit your Research
• My Submissions
• Article Guidelines
• Article Guidelines (New Versions)
• Data Guidelines
• Posters and Slides Guidelines
• Document Guidelines
• Article Processing Charges
• Peer Review
• The Peer Review Process
• The Editorial Team’s Role
• Understanding Peer Review Reports and Statuses
• Revising and Responding to Reviewers

Finding Article Reviewers

• Reviewer Criteria
• Hints and Tips for Finding Reviewers
• Dos and Don’ts for Suggesting Reviewers
• Identity transparency: All identities visible
• Reviewer interacts with: Editor, other reviewers, authors
• Review information published: Review reports, submitted manuscript, reviewer identities
• Post publication commenting: Open
• Qualified: Reviewers should typically hold a doctorate (PhD/MD/MBBS or equivalent). Exceptions will be made for scholarly disciplines where doctorates are not necessary (e.g. Education, Library Science), or when an individual has a demonstrable public record of expertise. If possible, when a reviewer suggestion is rejected due to lack of qualifications, the editorial team will suggest that their Principal Investigator/Supervisor is invited instead, and the original person could then take the role of co-reviewer.
• Expert: Reviewers should have published at least three articles as lead author in a relevant topic, with at least one article having been published in the last five years . In fields where a reviewer’s expertise is not typically measured by their publication record or if the suggested reviewer’s expertise is demonstrable in ways other than their publication record, please provide an explanation of their suitability.
• have co-authored with any of the lead authors in the three years preceding publication of Version 1;
• have co-authored with any of the lead authors since the publication of Version 1;
• currently work at the same institution as the authors;
• be a close collaborator with an author.
• Global: For any given article, we require that reviewers are from different institutions . (This does not apply to large, multi-site institutions, such as Max-Planck Institutes or University of California). We also strongly encourage that geographically-diverse reviewers are invited to review, to gain an international perspective on the article. In cases where we receive multiple reviewer suggestions from the same country, the editorial team can only invite one of these reviewers at a time. Providing a geographically-diverse set of reviewers will help to prevent delays to the peer review process.
• Use the authors of the references cited in your article as a starting point for finding reviewers working in your specific field.
• Search abstracting and indexing databases such as Google Scholar , PubMed , Web of Science * and Scopus * (or other subject-specific literature databases) for recent articles with specific keywords can help you to identify authors currently working in the same field as yourself, and who may be suitable to review your article. As an expert in your area of research, you will likely be aware of prominent laboratories whose staff may be suitable to review your articles - try searching their website for potential reviewers. You can also search for specific experts with whom you have no recent collaborations, as they or their postdocs may be suitable to review.
• Try the Journal/Author Name Estimator and other similar tools can help to identify authors who have published related articles.
• Use our Reviewer Finder Tool. This tool analyzes the submission and provides a ranked list of reviewer candidates based on leading authors of related published studies. Authors can access this tool via the 'Suggest Reviewers' link next to submitted and published articles in the Submissions section in My Research . As this is an automatically generated list of potential reviewers, authors must use their own judgement to determine if the suggested reviewers have the appropriate expertise to review the article.
• Make sure suggested reviewers are experts in the relevant subject area F1000Research will only invite reviewers who have expertise in the field of research covered by the article. Not only does this ensure thorough peer review, but also reviewers are more likely to agree to provide a report when the subject matter is close to their own area of expertise.
• Try and ensure a global spread of reviewers For any given article, the reviewers must come from separate institutions and should not be affiliated with the authors’ institutions. We also strongly encourage that reviewers from around the world are invited to review where possible so that a global perspective can be gained for the article, and to ensure that all aspects of the work are reviewed.
• Ensure reviewers from the algorithm are suitable before approving them To provide authors who wish to suggest reviewers with a shortlist, each article is scanned by our Reviewer Finder Tool, which automatically provides a list of researchers who have published related articles. Authors can suggest appropriate reviewers from this automatically generated list (which can be accessed via the ‘Suggest Reviewers’ link next to your submitted or published article in My Research ), but must use their own judgement to determine if the suggested reviewers have the appropriate expertise to review the article.
• Discuss with your co-authors It may be that your co-authors would like to suggest reviewers – only the submitting author is able to provide these, however we welcome suggestions on your other authors’ behalfs. We would also be happy to change the submitting author so that a co-author can submit reviewers directly, please contact the editorial team if you wish to do this.
• Contact us if you have any questions If we have rejected a reviewer who you believe to be suitable, or if you have any questions or concerns about our reviewer criteria, we are always happy to discuss. Please email us so that we can explore possible options.
• Suggest reviewers who have recently closely collaborated with you or your co-authors We consider this to be a potential conflict of interest. A reviewer should not be based at the same institution as any author, be a close collaborator, or have co-authored with any of the lead authors for three years before the publication of the article. Please note that we make exceptions for cases where researchers have published together on the findings or consensus from a large consortia.
• Suggest reviewers who do not have the right expertise The editorial team will not invite reviewers who do not appear to have appropriate expertise. Suggesting inappropriate reviewers can cause significant delays to the peer review process.
• Contact the reviewers directly To ensure a fair peer review process is maintained, the editorial team acts as the intermediary between authors and reviewers. By directly contacting the reviewers, authors could not only influence their assessment of the article, but could also dissuade them from reviewing. Please be aware that if evidence of an author coercing reviewers is brought to our attention, we will investigate and take appropriate action. Authors can respond to a peer review report by posting a comment under the report.

The email address should be the one you originally registered with F1000.

You registered with F1000 via Google, so we cannot reset your password.

To sign in, please click here .

If you still need help with your Google account password, please click here .

You registered with F1000 via Facebook, so we cannot reset your password.

If you still need help with your Facebook account password, please click here .

If your email address is registered with us, we will email you instructions to reset your password.

If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance.

Browse Econ Literature

• Working papers
• Software components
• Book chapters
• JEL classification

More features

• Subscribe to new research

RePEc Biblio

Author registration.

• Economics Virtual Seminar Calendar NEW!

How to Undertake an Impactful Literature Review: Understanding Review Approaches and Guidelines for High-impact Systematic Literature Reviews

• Author & abstract
• Related works & more

Corrections

• Amrita Chakraborty
• Arpan Kumar Kar

Suggested Citation

Download full text from publisher.

Follow serials, authors, keywords & more

Public profiles for Economics researchers

Various research rankings in Economics

RePEc Genealogy

Who was a student of whom, using RePEc

Curated articles & papers on economics topics

Upload your paper to be listed on RePEc and IDEAS

New papers by email

Subscribe to new additions to RePEc

EconAcademics

Blog aggregator for economics research

Cases of plagiarism in Economics

About RePEc

Initiative for open bibliographies in Economics

News about RePEc

Questions about IDEAS and RePEc

RePEc volunteers

Participating archives

Publishers indexing in RePEc

Privacy statement

Found an error or omission?

Opportunities to help RePEc

Get papers listed

Have your research listed on RePEc

Open a RePEc archive

Have your institution's/publisher's output listed on RePEc

Get RePEc data

Use data assembled by RePEc

This paper is in the following e-collection/theme issue:

Published on 10.5.2024 in Vol 10 (2024)

Community Engagement in Vaccination Promotion: Systematic Review and Meta-Analysis

Authors of this article:

• Yao Jie Xie 1, 2 * , PhD   ; 
• Xiaoli Liao 1 * , PhD   ; 
• Meijuan Lin 1 , MM   ; 
• Lin Yang 1 , PhD   ; 
• Kin Cheung 1 , PhD   ; 
• Qingpeng Zhang 3, 4 , PhD   ; 
• Yan Li 1 , PhD   ; 
• Chun Hao 5 , PhD   ; 
• Harry HX Wang 5, 6 , PhD   ; 
• Yang Gao 7 , PhD   ; 
• Dexing Zhang 8 , PhD   ; 
• Alex Molassiotis 9 , PhD   ; 
• Gilman Kit Hang Siu 10 , PhD   ; 
• Angela Yee Man Leung 1, 11 , PhD  

1 School of Nursing, The Hong Kong Polytechnic University, Hong Kong, China (Hong Kong)

2 Research Centre for Chinese Medicine Innovation, The Hong Kong Polytechnic University, Hong Kong, China (Hong Kong)

3 Musketeers Foundation Institute of Data Science, The University of Hong Kong, Hong Kong, China (Hong Kong)

4 Department of Pharmacology and Pharmacy, LKS Faculty of Medicine, The University of Hong Kong, Hong Kong, China (Hong Kong)

5 School of Public Health, Sun Yat-sen University, Guangzhou, China

6 Usher Institute, Deanery of Molecular, Genetic & Population Health Sciences, The University of Edinburgh, Edinburgh, United Kingdom

7 Department of Sport, Physical Education and Health, Hong Kong Baptist University, Hong Kong, China (Hong Kong)

8 JC School of Public Health and Primary Care, The Chinese University of Hong Kong, Hong Kong, China (Hong Kong)

9 Health and Social Care Research Centre, University of Derby, Derby, United Kingdom

10 Department of Health Technology and Informatics, Faculty of Health and Social Sciences, The Hong Kong Polytechnic University, Hong Kong, China (Hong Kong)

11 Research Institute on Smart Aging (RISA), The Hong Kong Polytechnic University, Hong Kong, China (Hong Kong)

*these authors contributed equally

Corresponding Author:

Yao Jie Xie, PhD

School of Nursing

The Hong Kong Polytechnic University

11 Yuk Choi Road

Hung Hom, Kowloon

China (Hong Kong)

Phone: 852 34003798

Fax:852 23649663

Email: [email protected]

Background: Community engagement plays a vital role in global immunization strategies, offering the potential to overcome vaccination hesitancy and enhance vaccination confidence. Although there is significant backing for community engagement in health promotion, the evidence supporting its effectiveness in vaccination promotion is fragmented and of uncertain quality.

Objective: This review aims to systematically examine the effectiveness of different contents and extent of community engagement for promoting vaccination rates.

Methods: This study was performed in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. A comprehensive and exhaustive literature search was performed in 4 English databases (PubMed, Embase, Web of Science, and Cochrane Library) and 2 Chinese databases (CNKI and Wan Fang) to identify all possible articles. Original research articles applying an experimental study design that investigated the effectiveness of community engagement in vaccination promotion were eligible for inclusion. Two reviewers independently performed the literature search, study selection, quality assessment, and data extraction. Discrepancies were resolved through discussion, with the arbitration of a third reviewer where necessary.

Results: A total of 20 articles out of 11,404 records from 2006 to 2021 were retrieved. The studies used various designs: 12 applied single-group pre-post study designs, 5 were cluster randomized controlled trials (RCTs), and 3 were non-RCTs. These studies targeted multiple vaccines, with 8 focusing on children’s immunization, 8 on human papillomavirus vaccine, 3 on hepatitis B virus vaccine, and 1 on COVID-19 vaccine. The meta-analysis revealed significant increases in vaccination rates both in pre-post comparison (rate difference [RD] 0.34, 95% CI 0.21-0.47, I 2 =99.9%, P <.001) and between-group comparison (RD 0.18, 95% CI 0.07-0.29, I 2 =98.4%, P <.001). The meta-analysis revealed that participant recruitment had the largest effect size (RD 0.51, 95% CI 0.36-0.67, I 2 =99.9%, P <.001), followed by intervention development (RD 0.36, 95% CI 0.23-0.50, I 2 =100.0%, P <.001), intervention implementation (RD 0.35, 95% CI 0.22-0.47, I 2 =99.8%, P <.001), and data collection (RD 0.34, 95% CI 0.19-0.50, I 2 =99.8%, P <.001). The meta-analysis indicated that high community engagement extent yielded the largest effect size (RD 0.49, 95% CI 0.17-0.82, I 2 =100.0%, P <.001), followed by moderate community engagement extent (RD 0.45, 95% CI 0.33-0.58, I 2 =99.6%, P <.001) and low community engagement extent (RD 0.15, 95% CI 0.05-0.25, I 2 =99.2%, P <.001). The meta-analysis revealed that “health service support” demonstrated the largest effect sizes (RD 0.45, 95% CI 0.25-0.65, I 2 =99.9%, P <.001), followed by “health education and discussion” (RD 0.39, 95% CI 0.20-0.58, I 2 =99.7%, P <.001), “follow-up and reminder” (RD 0.33, 95% CI 0.23-0.42, I 2 =99.3%, P <.001), and “social marketing campaigns and community mobilization” (RD 0.24, 95% CI 0.06-0.41, I 2 =99.9%, P <.001).

Conclusions: The results of this meta-analysis supported the effectiveness of community engagement in vaccination promotion with variations in terms of engagement contents and extent. Community engagement required a “fit-for-purpose” approach rather than a “one-size-fits-all” approach to maximize the effectiveness of vaccine promotion.

Trial Registration: PROSPERO CRD42022339081; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=339081

Introduction

Vaccination stands as one of the top 10 great public health achievements of the last century. It has made significant strides in eliminating and controlling various vaccine-preventable diseases, as evidenced by the reduction in morbidity, mortality, and disability caused by these diseases [ 1 , 2 ]. A notable illustration is the use of vaccines as a crucial measure to mitigate the COVID-19 pandemic in the past 3 years [ 3 , 4 ]. A previous study analyzed the economic advantages of vaccination against 10 diseases across 73 countries from 2001 to 2020. It reported that vaccinations have prevented over 20 million deaths and saved approximately US $350 billion in disease costs [ 5 ]. A modeling study examined the health implications of vaccination against 10 pathogens across 98 countries from 2000 to 2030. It revealed that vaccinations have prevented 69 million deaths [ 6 ].

Both the Global Vaccine Action Plan 2011-2020 and Immunization Agenda 2030 have established strategic objectives to immunize every eligible individual with appropriate vaccines and to ensure equitable coverage of immunization benefits for all. However, the immunization coverage of many vaccines has yet to reach the expected level. For instance, between 2006 and 2014, only 47 million women across 80 countries and territories received the full course of human papillomavirus (HPV) vaccines, representing a mere 1.4% coverage of the total female population [ 7 ]. In addition, a study assessing the coverage of childhood vaccines across 1366 administrative regions in 43 countries revealed that only one-third of children in 4 countries had fully received routine childhood vaccines [ 8 ]. In terms of adult vaccination, only 11 out of 204 countries achieved the World Health Organization (WHO) target of 90% coverage for 11 routine vaccines by 2019 [ 9 ]. Various reasons and barriers contribute to the lack of vaccination, with a significant obstacle being vaccine hesitancy. Vaccine hesitancy has been steadily rising worldwide over the past decade [ 10 , 11 ], emerging as one of the top 10 threats to global health listed by the WHO in 2019.

Community engagement is a process that involves engaging and motivating diverse partners to collaborate in harnessing community potential and enhancing community health [ 12 , 13 ]. It first gained prominence in the public health sphere with the Declaration of Alma-Ata and has since become increasingly prominent, particularly with the introduction of the new Sustainable Development Goals [ 14 ]. The WHO defines community engagement as “a process of developing relationship which enables stakeholders work together to address health issues” [ 15 ]. The United Nations Children’s Fund (UNICEF) defines community engagement as “an action for working with community stakeholders to improve community health” [ 13 ]. The definition of community engagement often intersects, competes with, and contradicts definitions of other terms such as community participation and community involvement, among others. Despite the extensive literature on community engagement, there is a lack of comprehensive guidelines to clarify the content and scope of community engagement, including what constitutes community engagement and the extent of its involvement. The levels of community engagement are structured along a continuum that spans from informing and consulting to involving, collaborating, and empowering [ 16 , 17 ]. The elements of community engagement manifest across a spectrum of initiatives, encompassing participant recruitment, intervention development, intervention implementation, and data collection [ 18 , 19 ]. Community engagement is characterized as a dynamic process rather than a singular intervention, operating within diverse contexts to address various issues through multiple mechanisms involving different actors.

A meta-analysis, incorporating 131 individual studies, supported the positive impact of community engagement on health and psychosocial outcomes for disadvantaged groups across various conditions [ 20 ]. It plays a prominent role in global immunization strategies, as it has the capacity to alleviate vaccination hesitancy and enhance vaccination confidence. A systematic review, which included 14 studies, examined the effectiveness of community interventions on HPV vaccine coverage. Of these, 12 studies reported that community interventions led to an increase in the uptake of the HPV vaccine [ 21 ]. Another review, spanning across 19 countries, assessed studies indicating that community engagement enhanced the timeliness and coverage of routine childhood immunization vaccines [ 22 ]. Despite robust evidence supporting the role of community engagement in promoting health within diverse populations, the evidence for community engagement specifically in vaccination promotion remains fragmented. Thus, we conducted a systematic review and meta-analysis to investigate the effectiveness of various aspects and levels of community engagement in enhancing vaccination rates.

This study was conducted following the guidelines outlined in the Cochrane Handbook for Systematic Reviews of Interventions [ 23 ], and the results were reported following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines [ 24 ]. The review protocol was registered in the PROSPERO database (CRD42022339081). Two reviewers (ML and YJX) conducted the literature search, study selection, quality assessment, and data extraction independently. Any discrepancies were resolved through discussion, and a third reviewer (LY) was consulted for arbitration when necessary.

Ethics Approval

This review paper was a secondary analysis of existing data from original studies published before, rather than a direct collection of new data, and thus, does not require ethical approval.

Search Strategies

A comprehensive and exhaustive literature search was conducted across 4 English databases (PubMed, Embase, Web of Science, and Cochrane Library), as well as 2 Chinese databases (CNKI and Wan Fang).

The search strategy involved combining terms related to “community engagement” and “vaccination” using specific vocabulary terms (MeSH and Emtree) and their corresponding free-text terms [ 25 , 26 ]. These terms were identified based on key publications in relevant fields, and the search strategy was adjusted to suit each database. Boolean operators, specifically “OR” between terms and “AND” between concepts, were used to combine search terms effectively.

No restrictions were placed on language, study design, country of origin, or publication date. Studies were searched in the selected databases from their inception to April 30, 2023. The initial literature searches were performed in June 2022, with an updated search conducted in April 2023. In addition, the reference lists of relevant articles and previous reviews were manually reviewed to identify any additional relevant studies. The ProQuest Dissertations & Theses Database was consulted to identify unpublished dissertations and theses. Furthermore, Google and Google Scholar were searched to identify gray literature for potential inclusion. Clinical trial registries, including ClinicalTrials.gov and the WHO International Clinical Trials Registry, were also searched to identify trials with outcomes that had not yet been published.

Details of the full search strategy for each database are listed in Table S1 in Multimedia Appendix 1 .

Selection Criteria

The inclusion and exclusion criteria were established based on the participants, interventions, comparisons, outcomes, and study design (PICOS) strategy [ 27 ]. Initially, these criteria were applied to titles and abstracts, and subsequently to full-text articles, to determine their final inclusion status ( Table 1 ).

All records retrieved from the literature search were imported into the bibliographic database EndNote (Clarivate), which was used to manage records and eliminate duplicates. Two reviewers (ML and XL) independently screened the records based on the eligibility criteria. Any discrepancies between the 2 reviewers were resolved through discussion, and a third reviewer (YJX) was consulted if consensus could not be reached. The search terms and selection criteria were designed to provide inclusive flexibility and discretion, considering the various permutations of community engagement.

Data Extraction

A data extraction form was developed and piloted on 6 randomly selected sample studies to establish consensus on the data abstraction procedures. Subsequently, 2 independent investigators (ML and XL) extracted information including the first author, publication year, study design, country, participant number, intervention details, control condition, vaccine rates, and effect size of the intervention, where reported. In cases where a study provided data for both vaccine series initiation and completion, only the latter was included in the summary table. If a study evaluated multiple vaccine types and reported a combined vaccination rate, that result was selected; otherwise data for the primary vaccine under focus were presented. In instances where a study reported incomplete data, the authors were contacted via email to obtain the required information.

Assessment of the Risk of Bias

The revised Cochrane Tool for Risk of Bias 2.0 (RoB2) was used to assess the risk of bias in randomized controlled trials (RCTs) [ 33 ]. For nonrandomized trials and controlled pre-post studies, the Risk of Bias in Non-randomized Studies-of Interventions (ROBINS-I) tool was used to evaluate the risk of bias [ 34 ].

Each study was assessed and categorized as having low, moderate, or high risk of bias for each domain. Studies with low risk in 3 or more domains and moderate risk in any remaining domain(s) were classified as having an overall low risk of bias. Studies with moderate risk in 3 or more domains and low or unclear risk in any remaining domain(s) were classified as having an overall moderate risk of bias. Studies with high risk in 3 or more domains and moderate risk in any remaining domain(s) were classified as having an overall high risk of bias. Studies with moderate risk in 3 or more domains and high risk in any remaining domain(s) were also classified as having an overall high risk of bias.

Data Synthesis

Descriptive statistics were used to summarize the key variables of the included studies. Meta-analysis was conducted using Stata version 15.1 (StataCorp LLC) to investigate the effectiveness of community engagement in promoting vaccination.

Vaccination rates were computed as the proportion of vaccinated individuals to the total targeted population. Effect sizes were represented as the rate difference (RD) of vaccination rates, along with 95% CIs [ 35 , 36 ]. Random effects models were used to calculate pooled effect sizes, considering the expected heterogeneity among studies. Standard errors were adjusted for clustering effects when trials used a cluster randomized controlled design.

Forest plots were used to display individual and pooled vaccination rates. Heterogeneity was assessed using the Cochrane Q test ( P CQ <.10) and the I 2 statistics. Subgroup analyses were conducted based on age groups, vaccine types, and immunization. A meta-regression analysis was performed to explore the effects of study design or quality on the pooled effect size [ 37 , 38 ]. Sensitivity analysis using a single-study knockout approach was performed to determine the contribution of each study to the pooled effect size. Publication bias was evaluated through visual inspection of the funnel plot, and the asymmetry of the funnel plot was further assessed using the Egger test [ 39 ]. The Egger tests required a minimum of 10 publications to examine the association between SE and effect size in the funnel plot [ 37 ]. We classified the evidence quality into different levels according to the recommendations from van Tulder et al [ 40 ].

Study Identification and Selection

The flowchart depicting the study selection process is presented in Figure 1 . The literature search was conducted across 6 electronic databases from July 5, 2022, to July 12, 2022, yielding a total of 11,404 records. After removing duplicates, 9512 articles remained. Following the preliminary review of titles and abstracts, 83 articles were retained for full-text assessment. Subsequently, after reviewing the full texts, the final selection of 19 eligible articles was made. An additional article was identified through a manual search of reference lists. Therefore, a total of 20 eligible articles published in English were identified that met all inclusion criteria.

Characteristics of the Included Studies

The characteristics of the included studies are summarized in Table S2 in Multimedia Appendix 1 . This review did not restrict the timeframe of the literature search to provide a broad temporal perspective. The included studies were published between 2006 and 2021, with the majority (n=8) in the last 5 years. These studies were conducted in various countries, with the highest number (n=13) conducted in the United States [ 41 - 53 ], followed by Nigeria (n=3) [ 54 - 56 ] and Peru (n=2) [ 57 , 58 ], and 1 each in Pakistan [ 29 ] and India [ 59 ]. The studies used various designs, with over one-half (n=12) adopting single-group pre-post study designs [ 41 - 44 , 46 - 48 , 53 , 54 , 56 - 58 ], while the rest used cluster RCTs (n=5) [ 29 , 45 , 52 , 55 , 59 ] and non-RCTs (n=3) [ 49 - 51 ]. The studies recruited participants across all age groups, spanning from children (n=8) [ 29 , 43 , 44 , 46 , 54 - 56 , 59 ], to adolescents (n=7) [ 41 , 49 - 52 , 57 , 58 ], and to adults (n=5) [ 42 , 45 , 47 , 48 , 53 ]. The sample sizes of pre-post studies ranged from 30 to 12,103, with a median of 323, while the sample sizes of RCTs ranged from 337 to 2598, with a median of 349. These included studies targeted multiple vaccines, with 8 studies focusing on children’s immunization [ 29 , 43 , 44 , 46 , 54 - 56 , 59 ], 8 studies on HPV vaccine [ 41 , 47 , 49 - 52 , 57 , 58 ], 3 studies on hepatitis B virus (HBV) vaccine [ 42 , 45 , 53 ], and 1 study on COVID-19 vaccine [ 48 ]. Vaccination coverage was calculated using either individual-reported or officially recorded data.

Conceptualization of Community Engagement

Community engagement does not neatly fit into predefined typologies, as it encompasses a variety of contexts, extents, and outcomes [ 60 , 61 ]. To address this complexity, a conceptual framework of community engagement was developed. This framework aims to delineate the different contents and extent of community engagement, drawing from the WHO definition of community engagement [ 62 ] and the utilitarian perspective of community engagement [ 63 ]. The contents of community engagement were delineated into 4 main categories: participant recruitment, intervention development, intervention implementation, and data collection. The extents of community engagement were categorized as low, moderate, and high [ 64 ]. Specifically, a low extent of community engagement indicated that studies fulfilled 1 or 2 contents of community engagement; a moderate extent of community engagement indicated that studies fulfilled 3 contents of community engagement; and a high extent of community engagement indicated that studies fulfilled all 4 contents of community engagement [ 64 ].

Most studies incorporated 2 engagement contents, with the majority engaged in intervention implementation (19/20, 95%) [ 29 , 41 - 46 , 48 - 59 ] and intervention development (13/20, 65%) [ 41 - 43 , 45 - 50 , 52 - 54 , 56 ], followed by participant recruitment (12/20,60%) [ 41 , 43 - 49 , 51 , 56 - 58 ] and outcome evaluation (11/20, 55%) [ 29 , 42 - 44 , 46 , 48 , 51 , 53 - 55 , 57 ] ( Table 2 ). Furthermore, most studies fell into the moderate engagement extent category (n=10) [ 41 , 42 , 44 , 45 , 49 , 51 , 53 , 54 , 56 , 57 ], followed by low engagement extent (n=7) [ 29 , 47 , 50 , 52 , 55 , 58 , 59 ] and high engagement extent (n=3) [ 43 , 46 , 48 ] ( Table 2 ).

Community engagement in these studies took various forms of intervention strategies, including social marketing campaigns, community mobilization, health education and discussions, health service support, and follow-up and reminders. These interventions were often combined into intervention packages, which included combinations such as health education and discussion with follow-up and reminders, health education and discussion with health service support, health education and discussion with health service support and follow-up reminders, social marketing campaigns and community mobilization with health service support and follow-up reminders, and social marketing campaigns and community mobilization with health education and discussion as well as follow-up reminders.

Community engagement varied in geographical coverage, ranging from localized sites in 1 village or city to broader areas encompassing 1 district or more. However, many interventions failed to consider implementation constraints and practicalities on the ground, which in turn limited the fidelity of community engagement and the efficient utilization of community resources.

Risk-of-Bias Assessment

These studies exhibited variable quality across different study designs, with none meeting all the elements of a good quality design. Individual domain ratings and overall bias risk ratings for each study are presented in Tables S3 and S4 in Multimedia Appendix 1 .

Two cluster RCT studies [ 29 , 59 ] were identified as having a low risk of bias, 1 [ 55 ] as a moderate risk of bias, and 2 [ 45 , 52 ] as a high risk of bias. This variability in the risk of bias may result from incorrect randomization procedures, deviations in intervention implementation, and incomplete outcome reporting. In addition, 6 quasi-experimental studies [ 44 , 48 , 49 , 53 , 54 , 57 ] were rated as having a low risk of bias, 4 [ 42 , 46 , 47 , 56 ] were rated as having a moderate risk of bias, and 5 [ 41 , 43 , 50 , 51 , 58 ] were rated as having a high risk of bias. The sources of bias in these studies may include confounding effects, missing outcome data, and selective reporting of results.

Overall Meta-Analysis of Community Engagement on Vaccination Rates

The pooled meta-analysis incorporated usable data from 21 intervention groups across 20 studies. The random effects meta-analysis of pre-post intervention effects revealed a moderate positive effect size of community engagement on promoting vaccination rates (RD 0.34, 95% CI 0.21-0.47, I 2 =99.9%, P CQ <.001; Figure 2 , see also [ 29 , 41 - 59 ]). Similarly, the random effects meta-analysis of between-group intervention effects showed a small positive effect size of community engagement on promoting vaccination rates (RD 0.18, 95% CI 0.07-0.29, I 2 =98.4%, P CQ <.001; Figure 2 ).

Meta-Analysis of Community Engagement Contents and Extent on Vaccination Rates

Regarding the contents of community engagement, the random effects meta-analysis revealed that participant recruitment yielded the largest effect size (RD 0.51, 95% CI 0.36-0.67, I 2 =99.9%, P CQ <.001), followed by intervention development (RD 0.36, 95% CI 0.23-0.50, I 2 =100.0%, P CQ <.001), intervention implementation (RD 0.35, 95% CI 0.22-0.47, I 2 =99.8%, P CQ <.001), and data collection (RD 0.34, 95% CI 0.19-0.50, I 2 =99.8%, P CQ <.001; Figure 3 , see also [ 29 , 41 - 59 ]).

With regard to the extent of community engagement, the random effects meta-analysis found that high community engagement extent yielded the largest effect size (RD 0.49, 95% CI 0.17-0.82, I 2 =100.0%, P CQ <.001), followed by moderate community engagement extent (RD 0.45, 95% CI 0.33-0.58, I 2 =99.6%, P CQ <.001) and low community engagement extent (RD 0.15, 95% CI 0.05-0.25, I 2 =99.2%, P CQ <.001; Figure 4 , see also [ 29 , 41 - 59 ]).

Meta-Analysis of Intervention Strategies on Vaccination Rates

With regard to single types of intervention strategies, the meta-analysis of 4 intervention strategies found that “health service support” yielded the largest effect sizes (RD 0.45, 95% CI 0.25-0.65, I 2 =99.9%, P CQ <.001), followed by “health education and discussion” (RD 0.39, 95% CI 0.20-0.58, I 2 =99.7%, P CQ <.001), “follow-up and reminder” (RD 0.33, 95% CI 0.23-0.42, I 2 =99.3%, P CQ <.001), and “social marketing campaigns and community mobilization” (RD 0.24, 95% CI 0.06-0.41, I 2 =99.9%, P CQ <.001; Figure 5 , see also [ 29 , 41 - 59 ]).

With regard to combined types of intervention strategies, the meta-analysis of 5 intervention strategy packages found that intervention package 2 yielded the largest increase in vaccination rates (RD 0.64, 95% CI 0.31-0.97, I 2 =99.3%, P CQ <.001), followed by intervention package 3 (RD 0.58, 95% CI 0.05-1.11, I 2 =99.1%, P CQ <.001), intervention package 4 (RD 0.31, 95% CI 0.20-0.41, I 2 =99.2%, P CQ <.001), and intervention package 1 (RD 0.25, 95% CI 0.09-0.41, I 2 =98.6%, P CQ <.001). However, intervention package 5 had no statistically significant impact on vaccination rates (RD 0.07, 95% CI 0.00-0.14, I 2 =72.7%, P CQ =.03; Figure 6 , see also [ 41 , 43 - 45 , 47 , 49 - 57 , 59 ]). Data from 5 studies were not synthesized because of high heterogeneity in their intervention strategies [ 29 , 42 , 46 , 48 , 58 ].

Subgroup Analyses of Age Groups, Vaccine Types, and Immunization Definitions on Vaccination Rates

Subgroup analyses revealed that adults (RD 0.50, 95% CI 0.16-0.85, I 2 =100.0%, P CQ <.001) exhibited a larger effect size compared with adolescents (RD 0.44, 95% CI 0.18-0.70, I 2 =99.3%, P CQ <.001) or children (RD 0.18, 95% CI 0.04-0.33, I 2 =99.7%, P <.001; Figure S1 in Multimedia Appendix 1 ). HPV vaccination (RD 0.44, 95% CI 0.18-0.70, I 2 =99.3%, P CQ <.001) exhibited a larger effect size compared with HBV vaccination (RD 0.42, 95% CI 0.12-0.72, I 2 =99.8%, P CQ <.001) or children immunization (RD 0.18, 95% CI 0.04-0.33, I 2 =99.7%, P CQ <.001; Figure S2 in Multimedia Appendix 1 ). Full immunization (RD 0.41, 95% CI 0.30-0.53, I 2 =99.5%, P CQ <.001) exhibited a larger effect size compared with partial immunization (RD 0.20, 95% CI 0.08-0.33, I 2 =93.3%, P CQ <.001). However, no significant increase was found in the vaccine rate of up-to-date immunization (RD 0.25, 95% CI –0.10 to 0.60, I 2 =100.0%, P CQ <.001; Figure S3 in Multimedia Appendix 1 ).

Sensitivity, Meta-Regression, Publication Bias, and Evidence Level

Sensitivity analysis showed that no significant changes were observed in the effect size of the pre-post intervention effect analysis (Figure S4 in Multimedia Appendix 1 ). However, the pooled effect size decreased dramatically when eliminating the study of Ma et al [ 45 ] in the between-group intervention effect analysis (RD 0.08, 95% CI 0.02-0.20; Figure S5 in Multimedia Appendix 1 ). Meta-regression analyses did not show any association between effect size and study design or study quality for the pre-post intervention effect analysis ( P =.16 or P =.65; Figure S6 in Multimedia Appendix 1 ). As evidenced by the funnel plot and Egger test, no discernible signs of publication bias were detected either in the pre-post or in the between-group intervention effect analyses ( P =.25; Figures S7 and S8 in Multimedia Appendix 1 ). According to recommendations from van Tulder et al [ 40 ], evidence quality in our meta-analysis was graded as moderate in both the pre-post and between-group intervention effect analyses (Table S5 in Multimedia Appendix 1 ).

Principal Findings

Community engagement drives interventions operated in a bottom-up manner rather than the traditional top-down approach. This approach supports stakeholders coming together to achieve global vaccination coverage goals from childhood to adulthood.

This study reported that community engagement strategies resulted in a 34% increase in vaccination rates through a pre-post intervention effect analysis and an 18% increase in vaccination rates through a between-group intervention effect analysis. The random effects meta-analyses indicated that participant recruitment exhibited the largest effect size, followed by intervention development, intervention implementation, and data collection. Consistent with previous evidence [ 64 ], intervention implementation constituted the primary engagement approach of most included studies and yielded a moderate effect size, while participant recruitment represented the engagement approach of over half of the included studies and yielded the largest effect size. Community partners who possess the knowledge and skills to effectively approach the target population and actively engage in participant recruitment hold the most potential to achieve relatively high recruitment and retention rates for participants. This meta-analysis found that the effect size increased with the extent of community engagement, with the highest community engagement extent exhibiting the largest effect size. Similar to previous evidence [ 65 ], a higher extent of community engagement resulted in greater vaccination promotion. Previous systematic reviews, which classify community engagement into different levels from low to high, also reported positive correlations between community engagement extents and intervention effects [ 65 ]. Regarding intervention strategies, the meta-analyses showed that health service support yielded the largest effect size, followed by health education and discussion, follow-up and reminder, and social marketing campaigns and community mobilization. Similar to previous studies, health service support increased routine childhood vaccine coverage [ 66 ]; health education and discussion increased HPV vaccine coverage among adolescents [ 21 , 67 ] and influenza vaccine coverage among older adults [ 68 ]; follow-up and reminder increased HBV vaccine coverage among adults [ 69 ]; and social marketing campaigns and community mobilization increased routine childhood vaccine coverage [ 70 ]. Health service support, whether used alone or in combination with other strategies, demonstrated effectiveness for vaccination promotion. Consistent with our analysis, previous studies have supported the effectiveness of health service support in the form of free vaccination, vaccination outreach or mobile clinic vaccination [ 66 , 71 ], and flexible vaccination schedules [ 72 ]. Intervention packages that combined health service support with the other 3 strategies resulted in a significant boost in vaccine rates. The intervention packages with adaptability and flexibility, which incorporated diverse intervention strategies, could effectively meet the needs of the community population and maximize intervention benefits.

Meta-analyses across a broad range of topics, populations, and interventions often encounter a disjunction between considerable heterogeneity arising from broad questions and the limited statistical methods available for variance analysis. The limited number of included studies precluded the performance of subgroup analyses and meta-regressions to fully address the sources of heterogeneity. The development of a conceptual framework provided homogeneity at the theoretical level despite the unavoidable nature of situational heterogeneity.

The geographic coverage of the included studies spanned across 5 countries, with most studies located in the United States, which could reflect a type of publication bias along with the skewed nature of global health research. These included studies were published between 2006 and 2021, with the majority in the last 5 years, which could reflect increased academic enthusiasm and enhanced policy support in recent years. However, most studies failed to disclose the social characteristics of community participants, which highlights the reality of known social hierarchies within communities.

Many studies proposed operational definitions of community engagement, and some studies suggested empirical models to explain its connotation. However, few articles made references to definitions or frameworks, reflecting a lack of theoretical basis and critical perspective. The lack of common definitions, along with the absence of conceptual frameworks, has led to diversified procedures and contents of engagement across diverse contexts and practices. Despite the wide acceptance of community engagement in theory and practice, considerable challenges remain in identifying the best engagement approach and evaluating engagement effectiveness [ 73 - 75 ]. Community engagement shares similar spirits but varies in practices, as the extent of engagement spans a spectrum from minimal superficial involvement to fully collaborative participation. Operating community engagement is cost-intensive rather than cost-neutral, requiring labor, capital, and time to establish, develop, and sustain fruitful partnerships, thus posing challenges to its successful and sustainable implementation. These included studies failed to report any analysis of costs, which precluded conclusions about the economic case for community engagement. While studies support the value of community engagement, the evaluation of community engagement has largely focused on health outcomes and ignored economic information. Future studies should incorporate economic analysis to explore the potential cost-utility and cost-effectiveness of community engagement in real-world contexts. This will help close the research-practice gap and facilitate evidence-based policy making.

The inclusion of experimental designs allowed the identification of a clear link between community engagement and vaccination promotion. However, none of these included studies were located at the top level of the evidence hierarchy, which limited the direct contribution of community engagement to vaccination promotion. Future studies with more rigorous designs should be performed to draw more definitive linkages about which participant group benefits the most from which engagement type in what community context. Randomized trials followed allocation sequence methods to ensure between-group comparability, but most interventions differed from comparisons in more ways than just community engagement. The comparator for community engagement always involved a completely different multicomponent intervention rather than the conventional health promotion activity without community engagement. The lack of a pure comparator in most community engagement interventions could cloud the interpretation of this meta-analysis. Community engagement often operates in nonlinear pathways synergized between various components and multiple outcomes, thus complicating effect evaluation compared with simple dose-response relationships. Community engagement functions as a dynamic process rather than as a discrete intervention, implying that evaluation should fully account for intrinsic complexities rather than simply focusing on outcome indicators. The primary studies should conduct thorough process evaluations to incorporate a spectrum of outcome measures and complement qualitative evaluations to elucidate the active ingredients of community engagement and the potential unintended effects of community engagement.

The effects of community engagement on vaccination promotion did not occur as a linear progression, but rather consisted of complex processes influenced by facilitators or challenges. These included studies identified individual- and community-level factors that facilitated or challenged community engagement in the context of vaccination promotion. At the individual level, the sense of confidence and ownership, along with the development of leadership skills and knowledge, facilitated community partners to engage with participatory processes. Conversely, the lack of interest and capacity, as well as the ambiguity of role and responsibility, challenged community partners to engage with participatory processes. At the community level, trust facilitated effective community engagement, while mistrust inhibited genuine community engagement. Further work should adopt a broader range of study designs that encompass both quantitative and qualitative methodologies to measure these intangible facilitators or challenges in the area of community engagement.

These included studies faced the challenge of measuring the level of community engagement, as engagement levels span a spectrum from more passive involvement to more active participation. This study proposed operationalized extents of community engagement beyond levels of community engagement from a pragmatic perspective. Further studies should be performed to develop tools or standards to measure and evaluate the levels of community engagement effectively.

As most studies narrowly defined community engagement as an intervention program imposed on the community, they framed the effectiveness of community engagement in terms of short-term individual-level outcomes [ 60 ] while neglecting multidimensional community-level outcomes. A narrow definition of community engagement, along with a restricted view of effectiveness, excludes a conceptually coherent and methodologically sound evaluation of community engagement [ 15 ]. Evaluating community engagement raises a unique set of challenges around conceptual, methodological, and practical aspects [ 76 ]. The interaction between the engagement strategy and the community system creates a degree of complexity beyond the detail of intervention implementation [ 77 ]. This complexity grows in concert with the delivery of the engagement strategy, which may, in some instances, reshape the intervention and the community context [ 77 ]. Future work should focus on intervention theories, logic models, and outcome frameworks to clarify the relationship between community engagement and health outcomes.

Community engagement can function independently or in conjunction with other initiatives. However, when combined with other initiatives, it becomes challenging to isolate the specific contribution of community engagement to health outcomes [ 78 ]. On the other hand, some studies treated community engagement as a discrete intervention rather than a dynamic process. This oversight has resulted in a lack of alternative process evaluations to explore how community engagement contributes to vaccination promotion [ 79 - 81 ]. Despite the widespread use and recognized significance of community engagement [ 82 ], there are still gaps in measuring and evaluating its implementation. While there is a vast body of literature on community engagement spanning various disciplines, comprehensive guidelines and frameworks for community engagement are lacking. The adoption of consistent guidelines and frameworks can formalize the implementation and evaluation of community engagement efforts.

Limitations

This study faces some challenges and limitations that warrant consideration and point toward future directions. The first challenge was the range of different definitions and terminology referring to engagement versus involvement and participation. The second challenge was the disjunction between the conceptual heterogeneity inherent in such broad questions and the limited statistical methods available to analyze variance. The third limitation was the possibility of study omission due to search deficiencies or publication bias, despite the extensive and rigorous literature search conducted.

Conclusions

The findings of this meta-analysis support the effectiveness of community engagement in promoting vaccination, with variations observed in terms of the contents and extent of engagement. Experimental studies often involve differences between the intervention and comparison groups beyond just community engagement. Studies designed to specifically isolate community engagement as the only differing factor between the intervention and comparison groups are suggested, which allows for a clearer understanding of its added value in vaccination promotion. Comprehensive process evaluations and qualitative evaluations should be used, to provide insights into the active ingredients of community engagement and uncover any unintended effects it may have. A further scientific agenda on community engagement should focus on theory development, framework construction, and effectiveness evaluation. Future studies will benefit from the adoption of standard guidelines and frameworks to enable cross-study or cross-country comparisons of community engagement, promoting effective, sustainable, and appropriate community initiatives.

Acknowledgments

This study is funded by the Health and Medical Research Fund (HMRF)–Commissioned Research on the Novel Coronavirus Disease (COVID-19; reference number COVID1903006-A). The funding body has (and will continue to have) no role in the study design, the collection, analysis, and interpretation of any data, or in the decision to submit the paper for publication.

Data Availability

All data generated or analyzed during this study are included in this published article and its supplementary information files.

Disclosure of AI Use

The generative artificial intelligence was not used in any portion of the manuscript writing.

Authors' Contributions

YJX conceptualized the study. XL and ML drafted the initial manuscript. YJX checked and revised the manuscript. All other authors contributed to article revisions for important intellectual content and approved the final draft.

Conflicts of Interest

None declared.

Literature search results, characteristics of included studies, risk-of-bias assessment, and subgroup and sensitivity analyses.

• Andre FE, Booy R, Bock HL, Clemens J, Datta SK, John TJ, et al. Vaccination greatly reduces disease, disability, death and inequity worldwide. Bull World Health Organ. Feb 2008;86(2):140-146. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Chang YC, Tung HJ, Huang YT, Lu CT, Ernawaty E, Wu SY. Effect of influenza vaccination on mortality and risk of hospitalization in elderly individuals with and without disabilities: a nationwide, population-based cohort study. Vaccines (Basel). Mar 02, 2020;8(1):112. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Jeyanathan M, Afkhami S, Smaill F, Miller MS, Lichty BD, Xing Z. Immunological considerations for COVID-19 vaccine strategies. Nat Rev Immunol. Oct 04, 2020;20(10):615-632. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Thomas SJ, Moreira ED, Kitchin N, Absalon J, Gurtman A, Lockhart S, et al. C4591001 Clinical Trial Group. Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine through 6 months. N Engl J Med. Nov 04, 2021;385(19):1761-1773. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ozawa S, Clark S, Portnoy A, Grewal S, Stack ML, Sinha A, et al. Estimated economic impact of vaccinations in 73 low- and middle-income countries, 2001-2020. Bull World Health Organ. Sep 01, 2017;95(9):629-638. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Li X, Mukandavire C, Cucunubá ZM, Echeverria Londono S, Abbas K, Clapham HE, et al. Vaccine Impact Modelling Consortium. Estimating the health impact of vaccination against ten pathogens in 98 low-income and middle-income countries from 2000 to 2030: a modelling study. Lancet. Jan 30, 2021;397(10272):398-408. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bruni L, Diaz M, Barrionuevo-Rosas L, Herrero R, Bray F, Bosch FX, et al. Global estimates of human papillomavirus vaccination coverage by region and income level: a pooled analysis. The Lancet Global Health. Jul 2016;4(7):e453-e463. [ CrossRef ]
• Dimitrova A, Carrasco-Escobar G, Richardson R, Benmarhnia T. Essential childhood immunization in 43 low- and middle-income countries: analysis of spatial trends and socioeconomic inequalities in vaccine coverage. PLoS Med. Jan 2023;20(1):e1004166. [ FREE Full text ] [ CrossRef ] [ Medline ]
• GBD 2020‚ Release 1‚ Vaccine Coverage Collaborators. Measuring routine childhood vaccination coverage in 204 countries and territories, 1980-2019: a systematic analysis for the Global Burden of Disease Study 2020, Release 1. Lancet. Aug 07, 2021;398(10299):503-521. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lane S, MacDonald NE, Marti M, Dumolard L. Vaccine hesitancy around the globe: analysis of three years of WHO/UNICEF Joint Reporting Form data-2015-2017. Vaccine. Jun 18, 2018;36(26):3861-3867. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lazarus JV, Wyka K, White TM, Picchio CA, Rabin K, Ratzan SC, et al. Revisiting COVID-19 vaccine hesitancy around the world using data from 23 countries in 2021. Nat Commun. Jul 01, 2022;13(1):3801. [ FREE Full text ] [ CrossRef ] [ Medline ]
• WHO TEAM: Enhanced Well-being (WEL), Health Promotion (HPR). Community engagement: a health promotion guide for universal health coverage in the hands of the people. World Health Organization. Geneva, Switzerland. World Health Organization; Oct 05, 2020. URL: https://www.who.int/publications/i/item/9789240010529 [accessed 2024-04-13]
• Minimum quality standards and indicators in community engagement. A guidance towards high quality, evidence-based community engagement in development and humanitarian contexts. UNICEF. New York, NY. UNICEF; Mar 1, 2020. URL: https://www.unicef.org/mena/reports/community-engagement-standards [accessed 2024-04-13]
• Marston C, Hinton R, Kean S, Baral S, Ahuja A, Costello A, et al. Community participation for transformative action on women’s, children’s and adolescents’ health. Bull World Health Organ. May 01, 2016;94(5):376-382. [ CrossRef ]
• World Health Organization (WHO). Tool 16: Community engagement: refugee and migrant health toolkit. WHO. URL: https://www.who.int/tools/refugee-and-migrant-health-toolkit/module-5/tool-16 [accessed 2024-04-13]
• Dayna B, Antonio D. Rhetoric or reality? Putting affected people at the centre of humanitarian action. ReliefWeb. 2014. URL: https:/​/reliefweb.​int/​report/​world/​rhetoric-or-reality-putting-affected-people-centre-humanitarian-action [accessed 2024-04-21]
• IAP2 Australasia. Public participation spectrum. International Association for Public Participation. URL: https://iap2.org.au/resources/spectrum/ [accessed 2024-04-21]
• Rifkin SB, Muller F, Bichmann W. Primary health care: on measuring participation. Soc Sci Med. 1988;26(9):931-940. [ CrossRef ] [ Medline ]
• Israel BA, Schulz AJ, Parker EA, Becker AB. Review of community-based research: assessing partnership approaches to improve public health. Annu Rev Public Health. 1998;19:173-202. [ CrossRef ] [ Medline ]
• O'Mara-Eves A, Brunton G, Oliver S, Kavanagh J, Jamal F, Thomas J. The effectiveness of community engagement in public health interventions for disadvantaged groups: a meta-analysis. BMC Public Health. Feb 12, 2015;15(1):129. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Niccolai LM, Hansen CE. Practice- and community-based interventions to increase human papillomavirus vaccine coverage: a systematic review. JAMA Pediatr. Jul 2015;169(7):686-692. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Jain M, Shisler S, Lane C, Bagai A, Brown E, Engelbert M. Use of community engagement interventions to improve child immunisation in low-income and middle-income countries: a systematic review and meta-analysis. BMJ Open. Nov 08, 2022;12(11):e061568. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Cumpston MS, McKenzie J, Welch V, Brennan SE. Strengthening systematic reviews in public health: guidance in the Cochrane Handbook for Systematic Reviews of Interventions, 2nd edition. J Public Health (Oxf). Dec 01, 2022;44(4):e588-e592. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. Mar 29, 2021;372:n71. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Israel BA, Eng E, Schulz AJ, Parker EA. Methods for Community-Based Participatory Research for Health, 2nd Edition. San Francisco, CA. Jossey-Bass; Oct 01, 2012.
• Osborne J, Paget J, Giles-Vernick T, Kutalek R, Napier D, Baliatsas C, et al. Community engagement and vulnerability in infectious diseases: a systematic review and qualitative analysis of the literature. Soc Sci Med. Sep 2021;284:114246. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Booth A, Sutton A, Clowes M, James MMS. Systematic Approaches to a Successful Literature Review (3rd Edition). London, UK. SAGE Publications Ltd; Nov 01, 2021.
• Sarrami-Foroushani P, Travaglia J, Debono D, Braithwaite J. Key concepts in consumer and community engagement: a scoping meta-review. BMC Health Serv Res. Jun 13, 2014;14:250. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Habib MA, Soofi S, Cousens S, Anwar S, Haque NU, Ahmed I, et al. Community engagement and integrated health and polio immunisation campaigns in conflict-affected areas of Pakistan: a cluster randomised controlled trial. The Lancet Global Health. Jun 2017;5(6):e593-e603. [ CrossRef ]
• Bondy JN, Thind A, Koval JJ, Speechley KN. Identifying the determinants of childhood immunization in the Philippines. Vaccine. Jan 01, 2009;27(1):169-175. [ CrossRef ] [ Medline ]
• O'Donnell S, Dubé E, Tapiero B, Gagneur A, Doll MK, Quach C. Determinants of under-immunization and cumulative time spent under-immunized in a Quebec cohort. Vaccine. Oct 13, 2017;35(43):5924-5931. [ CrossRef ] [ Medline ]
• Robison SG, Groom H, Young C. Frequency of alternative immunization schedule use in a metropolitan area. Pediatrics. Jul 2012;130(1):32-38. [ CrossRef ] [ Medline ]
• Sterne JAC, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. Aug 28, 2019;366:l4898. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sterne JAC, Hernán MA, McAleenan A, Reeves BC, Higgins JPT. Assessing risk of bias in a non-randomized study. In: Cochrane Handbook for Systematic Reviews of Interventions (2nd Edition). Chichester, UK. John Wiley & Sons; 2019;621-641.
• Sathorn C, Parashos P, Messer H. Antibacterial efficacy of calcium hydroxide intracanal dressing: a systematic review and meta-analysis. Int Endod J. Jan 2007;40(1):2-10. [ CrossRef ] [ Medline ]
• Grossman DC, Garcia CC. Effectiveness of health promotion programs to increase motor vehicle occupant restraint use among young children. Am J Prev Med. Jan 1999;16(1 Suppl):12-22. [ CrossRef ] [ Medline ]
• Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ. In: Welch VA, editor. Cochrane Handbook for Systematic Reviews of Interventions (2nd Edn). Chichester, UK. John Wiley & Sons; 2019.
• Phan K, Tian DH, Cao C, Black D, Yan TD. Systematic review and meta-analysis: techniques and a guide for the academic surgeon. Ann Cardiothorac Surg. Mar 2015;4(2):112-122. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lin L, Chu H. Quantifying publication bias in meta-analysis. Biometrics. Sep 2018;74(3):785-794. [ FREE Full text ] [ CrossRef ] [ Medline ]
• van Tulder M, Furlan A, Bombardier C, Bouter L, Editorial Board of the Cochrane Collaboration Back Review Group. Updated method guidelines for systematic reviews in the Cochrane Collaboration Back Review Group. Spine (Phila Pa 1976). Jun 15, 2003;28(12):1290-1299. [ CrossRef ] [ Medline ]
• Lennon T, Gundacker C, Nugent M, Simpson P, Magallanes NK, West C, et al. Ancillary benefit of increased HPV immunization rates following a CBPR approach to address immunization disparities in younger siblings. J Community Health. Jun 2019;44(3):544-551. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Weir RC, Toyoji M, McKee M, Li V, Wang CC. Assessing the impact of electronic health record interventions on hepatitis B screening and vaccination. J Health Care Poor Underserved. 2018;29(4):1587-1605. [ CrossRef ] [ Medline ]
• Findley SE, Irigoyen M, Sanchez M, Guzman L, Mejia M, Sajous M, et al. Community-based strategies to reduce childhood immunization disparities. Health Promot Pract. Jul 2006;7(3 Suppl):191S-200S. [ CrossRef ] [ Medline ]
• Suryadevara M, Bonville CA, Ferraioli F, Domachowske JB. Community-centered education improves vaccination rates in children from low-income households. Pediatrics. Aug 2013;132(2):319-325. [ CrossRef ] [ Medline ]
• Ma GX, Lee MM, Tan Y, Hanlon AL, Feng Z, Shireman TI, et al. Efficacy of a community-based participatory and multilevel intervention to enhance hepatitis B virus screening and vaccination in underserved Korean Americans. Cancer. Mar 01, 2018;124(5):973-982. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Willis E, Sabnis S, Hamilton C, Xiong F, Coleman K, Dellinger M, et al. Improving immunization rates through community-based participatory research: community health improvement for Milwaukee's children program. Prog Community Health Partnersh. 2016;10(1):19-30. [ CrossRef ] [ Medline ]
• Lee HY, Koopmeiners JS, McHugh J, Raveis VH, Ahluwalia JS. mHealth pilot study: text messaging intervention to promote HPV vaccination. Am J Hlth Behav. Jan 01, 2016;40(1):67-76. [ CrossRef ]
• Marquez C, Kerkhoff AD, Naso J, Contreras MG, Castellanos Diaz E, Rojas S, et al. A multi-component, community-based strategy to facilitate COVID-19 vaccine uptake among Latinx populations: from theory to practice. PLoS One. 2021;16(9):e0257111. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ma GX, Zhu L, Tan Y, Zhai S, Lin TR, Zambrano C, et al. A multilevel intervention to increase HPV vaccination among Asian American adolescents. J Community Health. Feb 2022;47(1):9-16. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sanderson M, Canedo JR, Khabele D, Fadden MK, Harris C, Beard K, et al. Pragmatic trial of an intervention to increase human papillomavirus vaccination in safety-net clinics. BMC Public Health. Feb 02, 2017;17(1):158. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Parra-Medina D, Morales-Campos DY, Mojica C, Ramirez AG. Promotora outreach, education and navigation support for HPV vaccination to Hispanic women with unvaccinated daughters. J Cancer Educ. Jun 2015;30(2):353-359. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Paskett ED, Krok-Schoen J, Pennell M, Tatum CM, Reiter PL, Peng J, et al. Results of a multilevel intervention trial to increase human papillomavirus (HPV) vaccine uptake among adolescent girls. Cancer Epidemiol Biomarkers Prev. Apr 2016;25(4):593-602. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bailey MB, Shiau R, Zola J, Fernyak SE, Fang T, So SKS, et al. San Francisco hep B free: a grassroots community coalition to prevent hepatitis B and liver cancer. J Community Health. Aug 2011;36(4):538-551. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bawa S, Shuaib F, Saidu M, Ningi A, Abdullahi S, Abba B, et al. Conduct of vaccination in hard-to-reach areas to address potential polio reservoir areas, 2014-2015. BMC Public Health. Dec 13, 2018;18(Suppl 4):1312. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Oyo-Ita A, Bosch-Capblanch X, Ross A, Oku A, Esu E, Ameh S, et al. Effects of engaging communities in decision-making and action through traditional and religious leaders on vaccination coverage in Cross River State, Nigeria: a cluster-randomised control trial. PLoS One. 2021;16(4):e0248236. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Akwataghibe NN, Ogunsola EA, Popoola OA, Agbo AI, Dieleman MA. Using participatory action research to improve immunization utilization in areas with pockets of unimmunized children in Nigeria. Health Res Policy Syst. Aug 11, 2021;19(Suppl 2):88. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Levinson KL, Abuelo C, Chyung E, Salmeron J, Belinson SE, Sologuren CV, et al. The Peru Cervical Cancer Prevention Study (PERCAPS): community-based participatory research in Manchay, Peru. Int J Gynecol Cancer. Jan 01, 2013;23(1):141-147. [ CrossRef ]
• Abuelo CE, Levinson KL, Salmeron J, Sologuren CV, Fernandez MJV, Belinson JL. The Peru Cervical Cancer Screening Study (PERCAPS): the design and implementation of a mother/daughter screen, treat, and vaccinate program in the Peruvian jungle. J Community Health. Jun 2014;39(3):409-415. [ FREE Full text ] [ CrossRef ] [ Medline ]
• More NS, Das S, Bapat U, Alcock G, Manjrekar S, Kamble V, et al. Community resource centres to improve the health of women and children in informal settlements in Mumbai: a cluster-randomised, controlled trial. The Lancet Global Health. Mar 2017;5(3):e335-e349. [ FREE Full text ] [ CrossRef ]
• Attree P, French B, Milton B, Povall S, Whitehead M, Popay J. The experience of community engagement for individuals: a rapid review of evidence. Health Soc Care Community. May 2011;19(3):250-260. [ CrossRef ] [ Medline ]
• Kenny A, Hyett N, Sawtell J, Dickson-Swift V, Farmer J, O'Meara P. Community participation in rural health: a scoping review. BMC Health Serv Res. Feb 18, 2013;13:64. [ FREE Full text ] [ CrossRef ] [ Medline ]
• World Health Organization (WHO). Community Engagement: A Health Promotion Guide for Universal Health Coverage in the Hands of the People. Geneva, Switzerland. World Health Organization; 2020.
• Brunton G, Thomas J, O'Mara-Eves A, Jamal F, Oliver S, Kavanagh J. Narratives of community engagement: a systematic review-derived conceptual framework for public health interventions. BMC Public Health. Dec 11, 2017;17(1):944. [ FREE Full text ] [ CrossRef ] [ Medline ]
• De las Nueces D, Hacker K, DiGirolamo A, Hicks LS. A systematic review of community-based participatory research to enhance clinical trials in racial and ethnic minority groups. Health Serv Res. Jun 2012;47(3 Pt 2):1363-1386. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Campbell JA, Yan A, Egede LE. Community-based participatory research interventions to improve diabetes outcomes: a systematic review. Diabetes Educ. Dec 2020;46(6):527-539. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Machado AA, Edwards SA, Mueller M, Saini V. Effective interventions to increase routine childhood immunization coverage in low socioeconomic status communities in developed countries: a systematic review and critical appraisal of peer-reviewed literature. Vaccine. May 21, 2021;39(22):2938-2964. [ CrossRef ] [ Medline ]
• Abdullahi LH, Kagina BM, Ndze VN, Hussey GD, Wiysonge CS. Improving vaccination uptake among adolescents. Cochrane Database Syst Rev. Jan 17, 2020;1(1):CD011895. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Thomas RE, Lorenzetti DL. Interventions to increase influenza vaccination rates of those 60 years and older in the community. Cochrane Database Syst Rev. May 30, 2018;5(5):CD005188. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Juon HS, Strong C, Kim F, Park E, Lee S. Lay health worker intervention improved compliance with hepatitis B vaccination in Asian Americans: randomized controlled trial. PLoS One. 2016;11(9):e0162683. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Opel DJ, Diekema DS, Lee NR, Marcuse EK. Social marketing as a strategy to increase immunization rates. Arch Pediatr Adolesc Med. May 2009;163(5):432-437. [ CrossRef ] [ Medline ]
• Abraham SAA, Amoah JO, Agyare DF, Sekimpi DK, Bosomtwe-Duker D, Druye AA, et al. Health service factors affecting the COVID-19 vaccination campaign in a Ghanaian metropolis: a qualitative exploratory study. BMJ Open. Dec 20, 2023;13(12):e076184. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Gazmararian JA, Coleman M, Prill M, Hinman AR, Ribner BS, Washington ML, et al. Influenza vaccination of health care workers: policies and practices of hospitals in a community setting. Am J Infect Control. Sep 2007;35(7):441-447. [ CrossRef ] [ Medline ]
• Ahmed SM, Nelson D, Kissack A, Franco Z, Whittle J, Kotchen T, et al. Towards building a bridge between community engagement in research (CEnR) and comparative effectiveness research (CER). Clin Transl Sci. Apr 2015;8(2):160-165. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Domecq JP, Prutsky G, Elraiyah T, Wang Z, Nabhan M, Shippee N, et al. Patient engagement in research: a systematic review. BMC Health Serv Res. Feb 26, 2014;14:89. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Graffigna G, Barello S, Riva G, Corbo M, Damiani G, Iannone P, et al. Italian consensus statement on patient engagement in chronic care: process and outcomes. Int J Environ Res Public Health. Jun 11, 2020;17(11):4167. [ FREE Full text ] [ CrossRef ] [ Medline ]
• George AS, Mehra V, Scott K, Sriram V. Community participation in health systems research: a systematic review assessing the state of research, the nature of interventions involved and the features of engagement with communities. PLoS One. 2015;10(10):e0141091. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mason AR, Carr Hill R, Myers LA, Street AD. Establishing the economics of engaging communities in health promotion: what is desirable, what is feasible? Critical Public Health. Sep 2008;18(3):285-297. [ CrossRef ]
• South J, Phillips G. Evaluating community engagement as part of the public health system. J Epidemiol Community Health. Jul 2014;68(7):692-696. [ CrossRef ] [ Medline ]
• Butterfoss FD. Process evaluation for community participation. Annu Rev Public Health. Dec 11, 2006;27(1):323-340. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Farnsworth SK, Böse K, Fajobi O, Souza PP, Peniston A, Davidson LL, et al. Community engagement to enhance child survival and early development in low- and middle-income countries: an evidence review. J Health Commun. Sep 10, 2014;19 Suppl 1(sup1):67-88. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Rifkin SB. Examining the links between community participation and health outcomes: a review of the literature. Health Policy Plan. Sep 01, 2014;29 Suppl 2(Suppl 2):ii98-i106. [ FREE Full text ] [ CrossRef ] [ Medline ]
• March S, Torres E, Ramos M, Ripoll J, García A, Bulilete O, et al. Adult community health-promoting interventions in primary health care: a systematic review. Prev Med. Jul 2015;76 Suppl:S94-104. [ CrossRef ] [ Medline ]

Abbreviations

Edited by A Mavragani, T Sanchez; submitted 06.06.23; peer-reviewed by X Liu, O Baggio; comments to author 01.09.23; revised version received 27.09.23; accepted 27.02.24; published 10.05.24.

©Yao Jie Xie, Xiaoli Liao, Meijuan Lin, Lin Yang, Kin Cheung, Qingpeng Zhang, Yan Li, Chun Hao, Harry HX Wang, Yang Gao, Dexing Zhang, Alex Molassiotis, Gilman Kit Hang Siu, Angela Yee Man Leung. Originally published in JMIR Public Health and Surveillance (https://publichealth.jmir.org), 10.05.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Public Health and Surveillance, is properly cited. The complete bibliographic information, a link to the original publication on https://publichealth.jmir.org, as well as this copyright and license information must be included.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• NATURE INDEX
• 01 May 2024

Plagiarism in peer-review reports could be the ‘tip of the iceberg’

• Jackson Ryan 0

Jackson Ryan is a freelance science journalist in Sydney, Australia.

You can also search for this author in PubMed   Google Scholar

Time pressures and a lack of confidence could be prompting reviewers to plagiarize text in their reports. Credit: Thomas Reimer/Zoonar via Alamy

Mikołaj Piniewski is a researcher to whom PhD students and collaborators turn when they need to revise or refine a manuscript. The hydrologist, at the Warsaw University of Life Sciences, has a keen eye for problems in text — a skill that came in handy last year when he encountered some suspicious writing in peer-review reports of his own paper.

Last May, when Piniewski was reading the peer-review feedback that he and his co-authors had received for a manuscript they’d submitted to an environmental-science journal, alarm bells started ringing in his head. Comments by two of the three reviewers were vague and lacked substance, so Piniewski decided to run a Google search, looking at specific phrases and quotes the reviewers had used.

To his surprise, he found the comments were identical to those that were already available on the Internet, in multiple open-access review reports from publishers such as MDPI and PLOS. “I was speechless,” says Piniewski. The revelation caused him to go back to another manuscript that he had submitted a few months earlier, and dig out the peer-review reports he received for that. He found more plagiarized text. After e-mailing several collaborators, he assembled a team to dig deeper.

Meet this super-spotter of duplicated images in science papers

The team published the results of its investigation in Scientometrics in February 1 , examining dozens of cases of apparent plagiarism in peer-review reports, identifying the use of identical phrases across reports prepared for 19 journals. The team discovered exact quotes duplicated across 50 publications, saying that the findings are just “the tip of the iceberg” when it comes to misconduct in the peer-review system.

Dorothy Bishop, a former neuroscientist at the University of Oxford, UK, who has turned her attention to investigating research misconduct, was “favourably impressed” by the team’s analysis. “I felt the way they approached it was quite useful and might be a guide for other people trying to pin this stuff down,” she says.

Peer review under review

Piniewski and his colleagues conducted three analyses. First, they uploaded five peer-review reports from the two manuscripts that his laboratory had submitted to a rudimentary online plagiarism-detection tool . The reports had 44–100% similarity to previously published online content. Links were provided to the sources in which duplications were found.

The researchers drilled down further. They broke one of the suspicious peer-review reports down to fragments of one to three sentences each and searched for them on Google. In seconds, the search engine returned a number of hits: the exact phrases appeared in 22 open peer-review reports, published between 2021 and 2023.

The final analysis provided the most worrying results. They took a single quote — 43 words long and featuring multiple language errors, including incorrect capitalization — and pasted it into Google. The search revealed that the quote, or variants of it, had been used in 50 peer-review reports.

Predominantly, these reports were from journals published by MDPI, PLOS and Elsevier, and the team found that the amount of duplication increased year-on-year between 2021 and 2023. Whether this is because of an increase in the number of open-access peer-review reports during this time or an indication of a growing problem is unclear — but Piniewski thinks that it could be a little bit of both.

Why would a peer reviewer use plagiarized text in their report? The team says that some might be attempting to save time , whereas others could be motivated by a lack of confidence in their writing ability, for example, if they aren’t fluent in English.

The team notes that there are instances that might not represent misconduct. “A tolerable rephrasing of your own words from a different review? I think that’s fine,” says Piniewski. “But I imagine that most of these cases we found are actually something else.”

The source of the problem

Duplication and manipulation of peer-review reports is not a new phenomenon. “I think it’s now increasingly recognized that the manipulation of the peer-review process, which was recognized around 2010, was probably an indication of paper mills operating at that point,” says Jennifer Byrne, director of biobanking at New South Wales Health in Sydney, Australia, who also studies research integrity in scientific literature.

Paper mills — organizations that churn out fake research papers and sell authorships to turn a profit — have been known to tamper with reviews to push manuscripts through to publication, says Byrne.

The fight against fake-paper factories that churn out sham science

However, when Bishop looked at Piniewski’s case, she could not find any overt evidence of paper-mill activity. Rather, she suspects that journal editors might be involved in cases of peer-review-report duplication and suggests studying the track records of those who’ve allowed inadequate or plagiarized reports to proliferate.

Piniewski’s team is also concerned about the rise of duplications as generative artificial intelligence (AI) becomes easier to access . Although his team didn’t look for signs of AI use, its ability to quickly ingest and rephrase large swathes of text is seen as an emerging issue.

A preprint posted in March 2 showed evidence of researchers using AI chatbots to assist with peer review, identifying specific adjectives that could be hallmarks of AI-written text in peer-review reports .

Bishop isn’t as concerned as Piniewski about AI-generated reports, saying that it’s easy to distinguish between AI-generated text and legitimate reviewer commentary. “The beautiful thing about peer review,” she says, is that it is “one thing you couldn’t do a credible job with AI”.

Preventing plagiarism

Publishers seem to be taking action. Bethany Baker, a media-relations manager at PLOS, who is based in Cambridge, UK, told Nature Index that the PLOS Publication Ethics team “is investigating the concerns raised in the Scientometrics article about potential plagiarism in peer reviews”.

How big is science’s fake-paper problem?

An Elsevier representative told Nature Index that the publisher “can confirm that this matter has been brought to our attention and we are conducting an investigation”.

In a statement, the MDPI Research Integrity and Publication Ethics Team said that it has been made aware of potential misconduct by reviewers in its journals and is “actively addressing and investigating this issue”. It did not confirm whether this was related to the Scientometrics article.

One proposed solution to the problem is ensuring that all submitted reviews are checked using plagiarism-detection software. In 2022, exploratory work by Adam Day, a data scientist at Sage Publications, based in Thousand Oaks, California, identified duplicated text in peer-review reports that might be suggestive of paper-mill activity. Day offered a similar solution of using anti-plagiarism software , such as Turnitin.

Piniewski expects the problem to get worse in the coming years, but he hasn’t received any unusual peer-review reports since those that originally sparked his research. Still, he says that he’s now even more vigilant. “If something unusual occurs, I will spot it.”

doi: https://doi.org/10.1038/d41586-024-01312-0

Piniewski, M., Jarić, I., Koutsoyiannis, D. & Kundzewicz, Z. W. Scientometrics https://doi.org/10.1007/s11192-024-04960-1 (2024).

Article   Google Scholar  

Liang, W. et al. Preprint at arXiv https://doi.org/10.48550/arXiv.2403.07183 (2024).

Download references

Related Articles

• Peer review
• Research management

Illuminating ‘the ugly side of science’: fresh incentives for reporting negative results

Career Feature 08 MAY 24

Algorithm ranks peer reviewers by reputation — but critics warn of bias

Nature Index 25 APR 24

Researchers want a ‘nutrition label’ for academic-paper facts

Nature Index 17 APR 24

Structure peer review to make it more robust

World View 16 APR 24

Is ChatGPT corrupting peer review? Telltale words hint at AI use

News 10 APR 24

Mount Etna’s spectacular smoke rings and more — April’s best science images

News 03 MAY 24

How reliable is this research? Tool flags papers discussed on PubPeer

News 29 APR 24

Southeast University Future Technology Institute Recruitment Notice

Professor openings in mechanical engineering, control science and engineering, and integrating emerging interdisciplinary majors

Nanjing, Jiangsu (CN)

Southeast University

Staff Scientist

A Staff Scientist position is available in the laboratory of Drs. Elliot and Glassberg to study translational aspects of lung injury, repair and fibro

Maywood, Illinois

Loyola University Chicago - Department of Medicine

W3-Professorship (with tenure) in Inorganic Chemistry

The Institute of Inorganic Chemistry in the Faculty of Mathematics and Natural Sciences at the University of Bonn invites applications for a W3-Pro...

53113, Zentrum (DE)

Rheinische Friedrich-Wilhelms-Universität

Principal Investigator Positions at the Chinese Institutes for Medical Research, Beijing

Studies of mechanisms of human diseases, drug discovery, biomedical engineering, public health and relevant interdisciplinary fields.

Beijing, China

The Chinese Institutes for Medical Research (CIMR), Beijing

Research Associate - Neural Development Disorders

Houston, Texas (US)

Baylor College of Medicine (BCM)

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Featured Clinical Reviews

• Screening for Atrial Fibrillation: US Preventive Services Task Force Recommendation Statement JAMA Recommendation Statement January 25, 2022
• Evaluating the Patient With a Pulmonary Nodule: A Review JAMA Review January 18, 2022
• Download PDF
• CME & MOC
• Share X Facebook Email LinkedIn
• Permissions

The Women’s Health Initiative Randomized Trials and Clinical Practice : A Review

• 1 Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts
• 2 Department of Medicine, David Geffen School of Medicine, University of California, Los Angeles
• 3 National Heart, Lung, and Blood Institute, National Institutes of Health, Bethesda, Maryland
• 4 Lundquist Institute for Biomedical Innovation at Harbor-UCLA Medical Center, Torrance, California
• 5 Division of Public Health Sciences, Fred Hutchinson Cancer Center, Seattle, Washington
• 6 Stanford Prevention Research Center, Department of Medicine, Stanford University, Stanford, California
• 7 Department of Epidemiology, University of Pittsburgh School of Public Health|Epidemiology, Pittsburgh, Pennsylvania
• 8 Department of Medicine, University of Alabama, Birmingham
• 9 Division of Epidemiology, Herbert Wertheim School of Public Health and Human Longevity Science, University of California, San Diego, La Jolla, California
• 10 Department of Health Promotion Science, University of Arizona, Tucson
• 11 Department of Preventive Medicine, Northwestern University, Chicago, Illinois
• 12 MedStar Health Research Institute and Department of Medicine, Georgetown University School of Medicine, Washington, DC
• 13 Department of Epidemiology and Environmental Health, University at Buffalo–SUNY, Buffalo, New York
• 14 Department of Gerontology and Geriatric Medicine, Wake Forest University School of Medicine, Winston-Salem, North Carolina

Importance   Approximately 55 million people in the US and approximately 1.1 billion people worldwide are postmenopausal women. To inform clinical practice about the health effects of menopausal hormone therapy, calcium plus vitamin D supplementation, and a low-fat dietary pattern, the Women’s Health Initiative (WHI) enrolled 161 808 postmenopausal US women (N = 68 132 in the clinical trials) aged 50 to 79 years at baseline from 1993 to 1998, and followed them up for up to 20 years.

Observations   The WHI clinical trial results do not support hormone therapy with oral conjugated equine estrogens plus medroxyprogesterone acetate for postmenopausal women or conjugated equine estrogens alone for those with prior hysterectomy to prevent cardiovascular disease, dementia, or other chronic diseases. However, hormone therapy is effective for treating moderate to severe vasomotor and other menopausal symptoms. These benefits of hormone therapy in early menopause, combined with lower rates of adverse effects of hormone therapy in early compared with later menopause, support initiation of hormone therapy before age 60 years for women without contraindications to hormone therapy who have bothersome menopausal symptoms. The WHI results do not support routinely recommending calcium plus vitamin D supplementation for fracture prevention in all postmenopausal women. However, calcium and vitamin D are appropriate for women who do not meet national guidelines for recommended intakes of these nutrients through diet. A low-fat dietary pattern with increased fruit, vegetable, and grain consumption did not prevent the primary outcomes of breast or colorectal cancer but was associated with lower rates of the secondary outcome of breast cancer mortality during long-term follow-up.

Conclusions and Relevance   For postmenopausal women, the WHI randomized clinical trials do not support menopausal hormone therapy to prevent cardiovascular disease or other chronic diseases. Menopausal hormone therapy is appropriate to treat bothersome vasomotor symptoms among women in early menopause, without contraindications, who are interested in taking hormone therapy. The WHI evidence does not support routine supplementation with calcium plus vitamin D for menopausal women to prevent fractures or a low-fat diet with increased fruits, vegetables, and grains to prevent breast or colorectal cancer. A potential role of a low-fat dietary pattern in reducing breast cancer mortality, a secondary outcome, warrants further study.

Read More About

Manson JE , Crandall CJ , Rossouw JE, et al. The Women’s Health Initiative Randomized Trials and Clinical Practice : A Review . JAMA. Published online May 01, 2024. doi:10.1001/jama.2024.6542

Manage citations:

© 2024

Artificial Intelligence Resource Center

Cardiology in JAMA : Read the Latest

Browse and subscribe to JAMA Network podcasts!

Others Also Liked

Select your interests.

Customize your JAMA Network experience by selecting one or more topics from the list below.

• Academic Medicine
• Acid Base, Electrolytes, Fluids
• Allergy and Clinical Immunology
• American Indian or Alaska Natives
• Anesthesiology
• Anticoagulation
• Art and Images in Psychiatry
• Artificial Intelligence
• Assisted Reproduction
• Bleeding and Transfusion
• Caring for the Critically Ill Patient
• Challenges in Clinical Electrocardiography
• Climate and Health
• Climate Change
• Clinical Challenge
• Clinical Decision Support
• Clinical Implications of Basic Neuroscience
• Clinical Pharmacy and Pharmacology
• Complementary and Alternative Medicine
• Consensus Statements
• Coronavirus (COVID-19)
• Critical Care Medicine
• Cultural Competency
• Dental Medicine
• Dermatology
• Diabetes and Endocrinology
• Diagnostic Test Interpretation
• Drug Development
• Electronic Health Records
• Emergency Medicine
• End of Life, Hospice, Palliative Care
• Environmental Health
• Equity, Diversity, and Inclusion
• Facial Plastic Surgery
• Gastroenterology and Hepatology
• Genetics and Genomics
• Genomics and Precision Health
• Global Health
• Guide to Statistics and Methods
• Hair Disorders
• Health Care Delivery Models
• Health Care Economics, Insurance, Payment
• Health Care Quality
• Health Care Reform
• Health Care Safety
• Health Care Workforce
• Health Disparities
• Health Inequities
• Health Policy
• Health Systems Science
• History of Medicine
• Hypertension
• Images in Neurology
• Implementation Science
• Infectious Diseases
• Innovations in Health Care Delivery
• JAMA Infographic
• Law and Medicine
• Leading Change
• Less is More
• LGBTQIA Medicine
• Lifestyle Behaviors
• Medical Coding
• Medical Devices and Equipment
• Medical Education
• Medical Education and Training
• Medical Journals and Publishing
• Mobile Health and Telemedicine
• Narrative Medicine
• Neuroscience and Psychiatry
• Notable Notes
• Nutrition, Obesity, Exercise
• Obstetrics and Gynecology
• Occupational Health
• Ophthalmology
• Orthopedics
• Otolaryngology
• Pain Medicine
• Palliative Care
• Pathology and Laboratory Medicine
• Patient Care
• Patient Information
• Performance Improvement
• Performance Measures
• Perioperative Care and Consultation
• Pharmacoeconomics
• Pharmacoepidemiology
• Pharmacogenetics
• Pharmacy and Clinical Pharmacology
• Physical Medicine and Rehabilitation
• Physical Therapy
• Physician Leadership
• Population Health
• Primary Care
• Professional Well-being
• Professionalism
• Psychiatry and Behavioral Health
• Public Health
• Pulmonary Medicine
• Regulatory Agencies
• Reproductive Health
• Research, Methods, Statistics
• Resuscitation
• Rheumatology
• Risk Management
• Scientific Discovery and the Future of Medicine
• Shared Decision Making and Communication
• Sleep Medicine
• Sports Medicine
• Stem Cell Transplantation
• Substance Use and Addiction Medicine
• Surgical Innovation
• Surgical Pearls
• Teachable Moment
• Technology and Finance
• The Art of JAMA
• The Arts and Medicine
• The Rational Clinical Examination
• Tobacco and e-Cigarettes
• Translational Medicine
• Trauma and Injury
• Treatment Adherence
• Ultrasonography
• Users' Guide to the Medical Literature
• Vaccination
• Venous Thromboembolism
• Veterans Health
• Women's Health
• Workflow and Process
• Wound Care, Infection, Healing
• Register for email alerts with links to free full-text articles
• Access PDFs of free articles
• Manage your interests
• Save searches and receive search alerts

• International edition
• Australia edition
• Europe edition

The Road to Freedom by Joseph Stiglitz review – against Hayek

The former world bank economist argues that neoliberalism paves the way for populism

I n 1944 the Austrian-born economist Friedrich Hayek, displaced to Britain, was disquieted by his leftwing academic peers. As Hayek saw it, their political philosophy committed the same error as the fascism that was ravaging his homeland. He wrote that the desire to plan an economy centrally was – in what became the title of his most famous book – The Road to Serfdom: “many who sincerely hate all of nazism’s manifestations are working for ideals whose realisation would lead straight to the abhorred tyranny”. Hayek cast fascism not as a reaction to progressive success, but as its natural endpoint.

Joseph Stiglitz, a former chief economist of the World Bank and adviser to Bill Clinton, tackles this idea head on in The Road to Freedom, his rejoinder to Hayek’s work and that of his libertarian fellow traveller Milton Friedman. As Stiglitz sees it, rather than too much government leading to tyranny, the shift to neoliberalism has reduced freedom and “provided fertile ground for populists”. Social democracy, with its greater role for the state, generates freer, robust societies that are resilient to authoritarians like former president Donald Trump.

The mistake Hayek and Friedman made, repeated by their latter day fans on the right – Stiglitz lumps them all together in a basket of climate-denying, Covid-rubbishing, Fox-News-watching ignoramuses – was in not truly understanding freedom. Freedom for one person can come at the expense of another – indeed a certain amount of coercion can expand the total amount of freedom, Stiglitz argues. Hayek and Friedman understood this principle when it came to national defence and the protection of private property, but it should be expanded to include the environment, public health and investments in infrastructure that make us all richer.

Stiglitz takes this a bit too far. Criticising the lack of urban planning in Houston, for example, he points to a sex toy shop located in the car park of a mall that also hosts a private school. The laissez-faire approach, he says, creates negative externalities – the phrase economists use to label harms imposed on third parties. In this instance, the freedom to buy products for use by consenting adults comes at the expense of others’ freedom not to be made aware of those activities. Stiglitz is clearly trying to address market excesses, but risks justifying a world of curtain-twitching puritanism.

At the same time, he points out the psychological constraints the market imposes on freedom. Advertising and social media limit our perspectives, reducing our ability to make choices just as much as laws and the power of the state do. Liberating us from these restrictions requires regulating others’ freedom, curbing their power to deceive us or to promote their polarising and distorted version of reality.

The supposedly freedom-enhancing role for coercion that he establishes in the first few chapters is, however, gradually forgotten, as the book becomes a recitation of familiar arguments in favour of social democracy and the role of government in mitigating market failure. There is little novel or surprising in this analysis.

Ultimately Hayek’s predictions were almost exactly wrong. The postwar welfare states did not lead to tyranny: the latter half of the 20th century saw freedom’s frontiers expand. Not only did censorship diminish – obscenity and blasphemy laws were overturned, for instance – but the civil rights movement, feminism and gay liberation all ensured more people were able to access traditional liberal rights. The most troubling feature of Stiglitz’s analysis, on the other hand, is that he may well be right. The neoliberal period has paved the way for the ascendancy of illiberal democrats, authoritarians like Trump, who have undermined or attempted to overthrow democracy. But do these people, with their apparent distaste for rules and restrictions, really not understand freedom? Or do they simply not care, seeing it as just another inconvenience standing in their way?

after newsletter promotion

• Philosophy books
• Politics books
• Economic policy

Most viewed

• Share full article

Advertisement

Supported by

10 Big Biden Environmental Rules, and What They Mean

Asbestos, “forever” chemicals, E.V.s and endangered species. Here’s what 10 new rules cover, and why the administration has been churning them out.

By Coral Davenport

The Biden administration has been racing this spring to finalize a slew of major environmental regulations, including rules to combat climate change, a first-ever ban on asbestos and new limits on toxic chemicals in tap water.

Many of the rules had been in the works since President Biden’s first day in office, when he ordered federal agencies to reinstate or strengthen more than 100 environmental regulations that President Donald J. Trump had weakened or removed . The president has pledged to cut the emissions that are driving climate change roughly in half by 2030. That’s something that scientists say all industrialized nations must achieve to keep global warming to relatively safe levels.

Lawyers in the Biden administration have sought to use every available tool to protect the rules from being gutted by a future administration or a new Congress.

Under the 1996 Congressional Review Act, Congress can delete new federal regulations by a simple majority vote within 60 legislative days of their publication in the Federal Register. Senate Republicans used that procedure in early 2017 to wipe out 14 regulations within 16 days that had been written by the Obama administration.

To avoid that fate, the White House told federal agencies to get major rules on the books by this spring. That doesn’t mean a new occupant of the White House couldn’t undo them through the regular rule-making process, or that the Supreme Court couldn’t eventually strike them down. But it cuts off one possible line of attack.

Here are 10 major environmental rules that the Biden administration rushed out the door to meet its self-imposed spring deadline.

Electrifying Cars

The federal government’s most significant climate regulation , this rule by the Environmental Protection Agency is designed to slash tailpipe pollution. Transportation is the segment of the American economy that generates the most greenhouse gases. The rule does not ban sales of gasoline-powered cars or mandate sales of all-electric vehicles, but it increasingly limits the amount of pollution allowed from auto tailpipes over time so that, by 2032, more than half the new cars sold in the United States would most likely be zero-emissions vehicles, up from just 7.6 percent last year.

That would avoid more than seven billion tons of carbon dioxide emissions over the next 30 years, according to the E.P.A. That’s the equivalent of removing a year’s worth of all the greenhouse gases generated by the United States.

Slashing Power Plant Pollution

This E.P.A. regulation cuts pollution from power plants, the nation’s second-largest source of planet-warming emissions. It requires existing coal plants in the United States to reduce 90 percent of their greenhouse pollution by 2039. It also requires future high-capacity power plants that burn natural gas to reduce their emissions 90 percent by 2032.

The rule is widely seen as a death knell for American coal plants. It will also make it difficult for many natural gas plants to operate without using carbon capture and sequestration, a process that traps emissions from smokestacks before they reach the atmosphere and then stores them. That technology is extremely expensive and not fully deployed at any American coal plant.

The E.P.A. estimates that the rule controlling greenhouse gases from power plants would eliminate 1.38 billion metric tons of carbon dioxide between now and 2047, which is equivalent to preventing the annual emissions from 328 million gasoline-powered cars.

Plugging Methane Leaks

This E.P.A. rule requires oil and gas producers to detect and fix leaks of methane, a potent greenhouse gas that wafts into the atmosphere from pipelines, drill sites and storage facilities.

Methane lingers in the atmosphere for about a decade after it is released, but it is about 80 times more powerful in the short term at trapping heat than carbon dioxide, which remains in the air for centuries.

The regulation would prevent 58 million tons of methane emissions by 2038, about the equivalent of all the carbon dioxide emitted by American coal-fired power plants in a single year.

Banning Asbestos

The E.P.A. banned chrysotile asbestos, the only type of asbestos still used in the United States, which has been linked to mesothelioma and other cancer.

Known as white asbestos, the mineral is used in roofing materials, textiles and cement as well as gaskets, clutches, brake pads and other automotive parts. It is also a component in diaphragms used to make chlorine.

The rule bans imports but allows companies up to 12 years to phase out the use of asbestos in manufacturing, depending on the facility.

Ending ‘Forever’ Chemicals in Tap Water

The E.P.A. for the first time is requiring municipal water systems to remove six synthetic chemicals linked to cancers, metabolic disorders and other health problems that are present in the tap water of hundreds of millions of Americans.

The perfluoroalkyl and polyfluoroalkyl substances, known collectively as PFAS, are found in everything from dental floss to firefighting foams to children’s toys. They are called forever chemicals because they degrade very slowly and can accumulate in the body and the environment.

Under the new rule, water utilities must monitor supplies for PFAS chemicals and are required to notify the public and reduce contamination if levels exceed a standard of 4 parts per trillion for perfluoroalkyl and polyfluoroalkyl substances.

Protecting Endangered Species

The administration restored several protections under the Endangered Species Act for imperiled animals and plants that had been loosened under Mr. Trump.

The rules, issued by the U.S. Fish and Wildlife Service and the National Oceanic and Atmospheric Administration’s fisheries service, give federal officials more leeway to protect species in a changing climate, bring back protections for animals that are classified as “threatened” with extinction, which is one step short of “endangered,” and clarify that decisions about whether to list a species must be made without considering economic factors.

Protecting the Alaskan Wilderness

The Interior Department denied permission for Ambler Road, a proposed 211-mile industrial road through fragile Alaskan wilderness to a large copper deposit. It was an enormous victory for opponents who argued that it would threaten wildlife as well as Alaska Native tribes that rely on hunting and fishing.

The road was essential to reach what is estimated to be a $7.5 billion copper deposit that lies under ecologically sensitive land. There are currently no mines in the area and no requests for permits have been filed with the government; the road was a first step.

The Interior Department found that a road would disturb wildlife habitat, pollute spawning grounds for salmon and threaten the hunting and fishing traditions of more than 30 Alaska Native communities.

Chemical Plant Safety

The E.P.A. rules for the first time require that almost 12,000 chemical plants and other industrial sites nationwide that handle hazardous materials must explicitly plan for and invest in safety measures against disasters, such as storms or floods, that could trigger an accidental release. For the first time, chemical sites that have had accidents will need to undergo an independent audit. And the rules require chemical plants to share more information with neighbors and emergency responders.

Raising the Price to Drill on Public Lands

The Interior Department made it more expensive for fossil fuel companies to pull oil, gas and coal from public lands, raising royalty rates for the first time in 100 years in a bid to end bargain-basement fees enjoyed by one of the country’s most profitable industries.

The government also increased more than tenfold the amount of the bonds that companies must secure before they start drilling.

The rate increase was mandated by Congress under the 2022 Inflation Reduction Act, which directed the Interior Department to raise the royalty fee from 12.5 percent, set in 1920, to 16.67 percent. Congress also stipulated that the minimum bid at auctions for drilling leases should be raised from $2 per acre to $10 per acre.

But the sharp jump in bond payments, the first increase since 1960, was decided by the Biden administration, not Congress. It came in response to arguments from environmental groups, watchdog groups and the U.S. Government Accountability Office that the bonds do not cover the cost of cleaning up abandoned, uncapped wells, leaving taxpayers with that burden.

Permitting for Highways, Power Lines and Pipelines

The White House released rules designed to speed up federal construction permits for clean energy projects while requiring federal agencies to more heavily weigh damaging effects on the climate and on low-income communities before approving projects like highways and oil wells.

Coral Davenport covers energy and environment policy, with a focus on climate change, for The Times. More about Coral Davenport