how to write findings in article review

• Search Menu
• Sign in through your institution
• Advance Articles
• Editor's Choice
• CME Reviews
• Best of 2021 collection
• Abbreviated Breast MRI Virtual Collection
• Contrast-enhanced Mammography Collection
• Author Guidelines
• Submission Site
• Open Access
• Self-Archiving Policy
• Accepted Papers Resource Guide
• About Journal of Breast Imaging
• About the Society of Breast Imaging
• Guidelines for Reviewers
• Resources for Reviewers and Authors
• Editorial Board
• Advertising Disclaimer
• Advertising and Corporate Services
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

• Introduction
• Selection of a Topic
• Scientific Literature Search and Analysis
• Structure of a Scientific Review Article
• Tips for Success
• Acknowledgments
• Conflict of Interest Statement
• < Previous

A Step-by-Step Guide to Writing a Scientific Review Article

• Article contents
• Figures & tables
• Supplementary Data

Manisha Bahl, A Step-by-Step Guide to Writing a Scientific Review Article, Journal of Breast Imaging , Volume 5, Issue 4, July/August 2023, Pages 480–485, https://doi.org/10.1093/jbi/wbad028

• Permissions Icon Permissions

Scientific review articles are comprehensive, focused reviews of the scientific literature written by subject matter experts. The task of writing a scientific review article can seem overwhelming; however, it can be managed by using an organized approach and devoting sufficient time to the process. The process involves selecting a topic about which the authors are knowledgeable and enthusiastic, conducting a literature search and critical analysis of the literature, and writing the article, which is composed of an abstract, introduction, body, and conclusion, with accompanying tables and figures. This article, which focuses on the narrative or traditional literature review, is intended to serve as a guide with practical steps for new writers. Tips for success are also discussed, including selecting a focused topic, maintaining objectivity and balance while writing, avoiding tedious data presentation in a laundry list format, moving from descriptions of the literature to critical analysis, avoiding simplistic conclusions, and budgeting time for the overall process.

• narrative discourse

Email alerts

Citing articles via.

• Recommend to your Librarian
• Journals Career Network

Affiliations

• Online ISSN 2631-6129
• Print ISSN 2631-6110
• Copyright © 2024 Society of Breast Imaging
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

• PRO Courses Guides New Tech Help Pro Expert Videos About wikiHow Pro Upgrade Sign In
• EDIT Edit this Article
• EXPLORE Tech Help Pro About Us Random Article Quizzes Request a New Article Community Dashboard This Or That Game Popular Categories Arts and Entertainment Artwork Books Movies Computers and Electronics Computers Phone Skills Technology Hacks Health Men's Health Mental Health Women's Health Relationships Dating Love Relationship Issues Hobbies and Crafts Crafts Drawing Games Education & Communication Communication Skills Personal Development Studying Personal Care and Style Fashion Hair Care Personal Hygiene Youth Personal Care School Stuff Dating All Categories Arts and Entertainment Finance and Business Home and Garden Relationship Quizzes Cars & Other Vehicles Food and Entertaining Personal Care and Style Sports and Fitness Computers and Electronics Health Pets and Animals Travel Education & Communication Hobbies and Crafts Philosophy and Religion Work World Family Life Holidays and Traditions Relationships Youth
• Browse Articles
• Learn Something New
• Quizzes Hot
• This Or That Game
• Train Your Brain
• Explore More
• Support wikiHow
• About wikiHow
• Log in / Sign up
• Education and Communications
• Critical Reviews

How to Write an Article Review (With Examples)

Last Updated: April 24, 2024 Fact Checked

Preparing to Write Your Review

Writing the article review, sample article reviews, expert q&a.

This article was co-authored by Jake Adams . Jake Adams is an academic tutor and the owner of Simplifi EDU, a Santa Monica, California based online tutoring business offering learning resources and online tutors for academic subjects K-College, SAT & ACT prep, and college admissions applications. With over 14 years of professional tutoring experience, Jake is dedicated to providing his clients the very best online tutoring experience and access to a network of excellent undergraduate and graduate-level tutors from top colleges all over the nation. Jake holds a BS in International Business and Marketing from Pepperdine University. There are 12 references cited in this article, which can be found at the bottom of the page. This article has been fact-checked, ensuring the accuracy of any cited facts and confirming the authority of its sources. This article has been viewed 3,110,125 times.

An article review is both a summary and an evaluation of another writer's article. Teachers often assign article reviews to introduce students to the work of experts in the field. Experts also are often asked to review the work of other professionals. Understanding the main points and arguments of the article is essential for an accurate summation. Logical evaluation of the article's main theme, supporting arguments, and implications for further research is an important element of a review . Here are a few guidelines for writing an article review.

Education specialist Alexander Peterman recommends: "In the case of a review, your objective should be to reflect on the effectiveness of what has already been written, rather than writing to inform your audience about a subject."

Article Review 101

• Read the article very closely, and then take time to reflect on your evaluation. Consider whether the article effectively achieves what it set out to.
• Write out a full article review by completing your intro, summary, evaluation, and conclusion. Don't forget to add a title, too!
• Proofread your review for mistakes (like grammar and usage), while also cutting down on needless information.

• Article reviews present more than just an opinion. You will engage with the text to create a response to the scholarly writer's ideas. You will respond to and use ideas, theories, and research from your studies. Your critique of the article will be based on proof and your own thoughtful reasoning.
• An article review only responds to the author's research. It typically does not provide any new research. However, if you are correcting misleading or otherwise incorrect points, some new data may be presented.
• An article review both summarizes and evaluates the article.

• Summarize the article. Focus on the important points, claims, and information.
• Discuss the positive aspects of the article. Think about what the author does well, good points she makes, and insightful observations.
• Identify contradictions, gaps, and inconsistencies in the text. Determine if there is enough data or research included to support the author's claims. Find any unanswered questions left in the article.

• Make note of words or issues you don't understand and questions you have.
• Look up terms or concepts you are unfamiliar with, so you can fully understand the article. Read about concepts in-depth to make sure you understand their full context.

• Pay careful attention to the meaning of the article. Make sure you fully understand the article. The only way to write a good article review is to understand the article.

• With either method, make an outline of the main points made in the article and the supporting research or arguments. It is strictly a restatement of the main points of the article and does not include your opinions.
• After putting the article in your own words, decide which parts of the article you want to discuss in your review. You can focus on the theoretical approach, the content, the presentation or interpretation of evidence, or the style. You will always discuss the main issues of the article, but you can sometimes also focus on certain aspects. This comes in handy if you want to focus the review towards the content of a course.
• Review the summary outline to eliminate unnecessary items. Erase or cross out the less important arguments or supplemental information. Your revised summary can serve as the basis for the summary you provide at the beginning of your review.

• What does the article set out to do?
• What is the theoretical framework or assumptions?
• Are the central concepts clearly defined?
• How adequate is the evidence?
• How does the article fit into the literature and field?
• Does it advance the knowledge of the subject?
• How clear is the author's writing? Don't: include superficial opinions or your personal reaction. Do: pay attention to your biases, so you can overcome them.

• For example, in MLA , a citation may look like: Duvall, John N. "The (Super)Marketplace of Images: Television as Unmediated Mediation in DeLillo's White Noise ." Arizona Quarterly 50.3 (1994): 127-53. Print. [9] X Trustworthy Source Purdue Online Writing Lab Trusted resource for writing and citation guidelines Go to source

• For example: The article, "Condom use will increase the spread of AIDS," was written by Anthony Zimmerman, a Catholic priest.

• Your introduction should only be 10-25% of your review.
• End the introduction with your thesis. Your thesis should address the above issues. For example: Although the author has some good points, his article is biased and contains some misinterpretation of data from others’ analysis of the effectiveness of the condom.

• Use direct quotes from the author sparingly.
• Review the summary you have written. Read over your summary many times to ensure that your words are an accurate description of the author's article.

• Support your critique with evidence from the article or other texts.
• The summary portion is very important for your critique. You must make the author's argument clear in the summary section for your evaluation to make sense.
• Remember, this is not where you say if you liked the article or not. You are assessing the significance and relevance of the article.
• Use a topic sentence and supportive arguments for each opinion. For example, you might address a particular strength in the first sentence of the opinion section, followed by several sentences elaborating on the significance of the point.

• This should only be about 10% of your overall essay.
• For example: This critical review has evaluated the article "Condom use will increase the spread of AIDS" by Anthony Zimmerman. The arguments in the article show the presence of bias, prejudice, argumentative writing without supporting details, and misinformation. These points weaken the author’s arguments and reduce his credibility.

• Make sure you have identified and discussed the 3-4 key issues in the article.

You Might Also Like

• ↑ https://libguides.cmich.edu/writinghelp/articlereview
• ↑ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4548566/
• ↑ Jake Adams. Academic Tutor & Test Prep Specialist. Expert Interview. 24 July 2020.
• ↑ https://guides.library.queensu.ca/introduction-research/writing/critical
• ↑ https://www.iup.edu/writingcenter/writing-resources/organization-and-structure/creating-an-outline.html
• ↑ https://writing.umn.edu/sws/assets/pdf/quicktips/titles.pdf
• ↑ https://owl.purdue.edu/owl/research_and_citation/mla_style/mla_formatting_and_style_guide/mla_works_cited_periodicals.html
• ↑ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4548565/
• ↑ https://writingcenter.uconn.edu/wp-content/uploads/sites/593/2014/06/How_to_Summarize_a_Research_Article1.pdf
• ↑ https://www.uis.edu/learning-hub/writing-resources/handouts/learning-hub/how-to-review-a-journal-article
• ↑ https://writingcenter.unc.edu/tips-and-tools/editing-and-proofreading/

About This Article

If you have to write an article review, read through the original article closely, taking notes and highlighting important sections as you read. Next, rewrite the article in your own words, either in a long paragraph or as an outline. Open your article review by citing the article, then write an introduction which states the article’s thesis. Next, summarize the article, followed by your opinion about whether the article was clear, thorough, and useful. Finish with a paragraph that summarizes the main points of the article and your opinions. To learn more about what to include in your personal critique of the article, keep reading the article! Did this summary help you? Yes No

• Send fan mail to authors

Reader Success Stories

Prince Asiedu-Gyan

Apr 22, 2022

Did this article help you?

Sammy James

Sep 12, 2017

Juabin Matey

Aug 30, 2017

Vanita Meghrajani

Jul 21, 2016

Nov 27, 2018

Featured Articles

Trending Articles

Watch Articles

• Terms of Use
• Privacy Policy
• Do Not Sell or Share My Info
• Not Selling Info

Don’t miss out! Sign up for

wikiHow’s newsletter

The Tech Edvocate

• Advertisement
• Home Page Five (No Sidebar)
• Home Page Four
• Home Page Three
• Home Page Two
• Icons [No Sidebar]
• Left Sidbear Page
• Lynch Educational Consulting
• My Speaking Page
• Newsletter Sign Up Confirmation
• Newsletter Unsubscription
• Page Example
• Privacy Policy
• Protected Content
• Request a Product Review
• Shortcodes Examples
• Terms and Conditions
• The Edvocate
• The Tech Edvocate Product Guide
• Write For Us
• Dr. Lynch’s Personal Website
• The Edvocate Podcast
• Assistive Technology
• Child Development Tech
• Early Childhood & K-12 EdTech
• EdTech Futures
• EdTech News
• EdTech Policy & Reform
• EdTech Startups & Businesses
• Higher Education EdTech
• Online Learning & eLearning
• Parent & Family Tech
• Personalized Learning
• Product Reviews
• Tech Edvocate Awards
• School Ratings

Best First Aid Kits: A Comprehensive Guide

Teaching writing in kindergarten: everything you need to know, haiti names new prime minister to try to lead country out of crisis, israel pushes into rafah as displaced palestinians search for safety, gazan officials say a strike killed 21 in al-mawasi, pope apologizes after reports that he used an anti-gay slur, growing pressure on western nations to expand the range of weaponry provided to ukraine has been escalating as the conflict with russia continues. leaders and military officials are increasingly debating the possibility of allowing ukraine to employ western-supplied weapons to carry out strikes against targets on russian territory. the crux of the argument for allowing ukraine such offensive capabilities is grounded in the desire to create a significant deterrent effect. proponents argue that enabling ukraine to strike back at russia could force moscow to reconsider its strategy and potentially lead to a de-escalation of hostilities. opponents, however, warn of the risks associated with such a move. escalation dominance, wherein one side’s increase in capabilities leads to an arms race, poses a serious concern. there is also fear that enabling ukraine to strike inside russia might provoke a strong retaliation, not just against ukraine but potentially involving western nations more directly in the conflict. the debate involves complex strategic calculations. on one hand, there’s a moral and strategic impetus to support ukraine in defending its sovereignty and territorial integrity. on the other hand, there’s a need for caution and consideration of long-term regional stability and global security. as discussions continue without definitive conclusions, it is clear that decisions made today will have lasting implications for international norms and future geopolitical conflicts. the international community awaits further developments while contemplating the far-reaching consequences of this critical juncture in east-west relations., why lawmakers are brawling and people are protesting in taiwan, three european countries formally recognize palestinian statehood, what we know about the papua new guinea landslide, how to write an article review (with sample reviews)  .

An article review is a critical evaluation of a scholarly or scientific piece, which aims to summarize its main ideas, assess its contributions, and provide constructive feedback. A well-written review not only benefits the author of the article under scrutiny but also serves as a valuable resource for fellow researchers and scholars. Follow these steps to create an effective and informative article review:

1. Understand the purpose: Before diving into the article, it is important to understand the intent of writing a review. This helps in focusing your thoughts, directing your analysis, and ensuring your review adds value to the academic community.

2. Read the article thoroughly: Carefully read the article multiple times to get a complete understanding of its content, arguments, and conclusions. As you read, take notes on key points, supporting evidence, and any areas that require further exploration or clarification.

3. Summarize the main ideas: In your review’s introduction, briefly outline the primary themes and arguments presented by the author(s). Keep it concise but sufficiently informative so that readers can quickly grasp the essence of the article.

4. Evaluate the strengths and weaknesses: In subsequent paragraphs, assess the strengths and limitations of the article based on factors such as methodology, quality of evidence presented, coherence of arguments, and alignment with existing literature in the field. Be fair and objective while providing your critique.

5. Discuss any implications: Deliberate on how this particular piece contributes to or challenges existing knowledge in its discipline. You may also discuss potential improvements for future research or explore real-world applications stemming from this study.

6. Provide recommendations: Finally, offer suggestions for both the author(s) and readers regarding how they can further build on this work or apply its findings in practice.

7. Proofread and revise: Once your initial draft is complete, go through it carefully for clarity, accuracy, and coherence. Revise as necessary, ensuring your review is both informative and engaging for readers.

Sample Review:

A Critical Review of “The Effects of Social Media on Mental Health”

Introduction:

“The Effects of Social Media on Mental Health” is a timely article which investigates the relationship between social media usage and psychological well-being. The authors present compelling evidence to support their argument that excessive use of social media can result in decreased self-esteem, increased anxiety, and a negative impact on interpersonal relationships.

Strengths and weaknesses:

One of the strengths of this article lies in its well-structured methodology utilizing a variety of sources, including quantitative surveys and qualitative interviews. This approach provides a comprehensive view of the topic, allowing for a more nuanced understanding of the effects of social media on mental health. However, it would have been beneficial if the authors included a larger sample size to increase the reliability of their conclusions. Additionally, exploring how different platforms may influence mental health differently could have added depth to the analysis.

Implications:

The findings in this article contribute significantly to ongoing debates surrounding the psychological implications of social media use. It highlights the potential dangers that excessive engagement with online platforms may pose to one’s mental well-being and encourages further research into interventions that could mitigate these risks. The study also offers an opportunity for educators and policy-makers to take note and develop strategies to foster healthier online behavior.

Recommendations:

Future researchers should consider investigating how specific social media platforms impact mental health outcomes, as this could lead to more targeted interventions. For practitioners, implementing educational programs aimed at promoting healthy online habits may be beneficial in mitigating the potential negative consequences associated with excessive social media use.

Conclusion:

Overall, “The Effects of Social Media on Mental Health” is an important and informative piece that raises awareness about a pressing issue in today’s digital age. Given its minor limitations, it provides valuable

3 Ways to Make a Mini Greenhouse ...

3 ways to teach yourself to play ....

Matthew Lynch

Related articles more from author.

3 Ways to Be a Good Boxer

How to Freeze Runner Beans: 13 Steps

10 Simple Ways to Organize Desk Cables

3 Ways to Write a Disabled Character

4 Ways to Treat Tennis Elbow

3 Easy Ways to Use Crep Protect Spray

How to Write an Article Review: Tips and Examples

Did you know that article reviews are not just academic exercises but also a valuable skill in today's information age? In a world inundated with content, being able to dissect and evaluate articles critically can help you separate the wheat from the chaff. Whether you're a student aiming to excel in your coursework or a professional looking to stay well-informed, mastering the art of writing article reviews is an invaluable skill.

Short Description

In this article, our research paper writing service experts will start by unraveling the concept of article reviews and discussing the various types. You'll also gain insights into the art of formatting your review effectively. To ensure you're well-prepared, we'll take you through the pre-writing process, offering tips on setting the stage for your review. But it doesn't stop there. You'll find a practical example of an article review to help you grasp the concepts in action. To complete your journey, we'll guide you through the post-writing process, equipping you with essential proofreading techniques to ensure your work shines with clarity and precision!

What Is an Article Review: Grasping the Concept 

A review article is a type of professional paper writing that demands a high level of in-depth analysis and a well-structured presentation of arguments. It is a critical, constructive evaluation of literature in a particular field through summary, classification, analysis, and comparison.

If you write a scientific review, you have to use database searches to portray the research. Your primary goal is to summarize everything and present a clear understanding of the topic you've been working on.

Writing Involves:

• Summarization, classification, analysis, critiques, and comparison.
• The analysis, evaluation, and comparison require the use of theories, ideas, and research relevant to the subject area of the article.
• It is also worth nothing if a review does not introduce new information, but instead presents a response to another writer's work.
• Check out other samples to gain a better understanding of how to review the article.

Types of Review

When it comes to article reviews, there's more than one way to approach the task. Understanding the various types of reviews is like having a versatile toolkit at your disposal. In this section, we'll walk you through the different dimensions of review types, each offering a unique perspective and purpose. Whether you're dissecting a scholarly article, critiquing a piece of literature, or evaluating a product, you'll discover the diverse landscape of article reviews and how to navigate it effectively.

Journal Article Review

Just like other types of reviews, a journal article review assesses the merits and shortcomings of a published work. To illustrate, consider a review of an academic paper on climate change, where the writer meticulously analyzes and interprets the article's significance within the context of environmental science.

Research Article Review

Distinguished by its focus on research methodologies, a research article review scrutinizes the techniques used in a study and evaluates them in light of the subsequent analysis and critique. For instance, when reviewing a research article on the effects of a new drug, the reviewer would delve into the methods employed to gather data and assess their reliability.

Science Article Review

In the realm of scientific literature, a science article review encompasses a wide array of subjects. Scientific publications often provide extensive background information, which can be instrumental in conducting a comprehensive analysis. For example, when reviewing an article about the latest breakthroughs in genetics, the reviewer may draw upon the background knowledge provided to facilitate a more in-depth evaluation of the publication.

Need a Hand From Professionals?

Address to Our Writers and Get Assistance in Any Questions!

Formatting an Article Review

The format of the article should always adhere to the citation style required by your professor. If you're not sure, seek clarification on the preferred format and ask him to clarify several other pointers to complete the formatting of an article review adequately.

How Many Publications Should You Review?

• In what format should you cite your articles (MLA, APA, ASA, Chicago, etc.)?
• What length should your review be?
• Should you include a summary, critique, or personal opinion in your assignment?
• Do you need to call attention to a theme or central idea within the articles?
• Does your instructor require background information?

When you know the answers to these questions, you may start writing your assignment. Below are examples of MLA and APA formats, as those are the two most common citation styles.

Using the APA Format

Articles appear most commonly in academic journals, newspapers, and websites. If you write an article review in the APA format, you will need to write bibliographical entries for the sources you use:

• Web : Author [last name], A.A [first and middle initial]. (Year, Month, Date of Publication). Title. Retrieved from {link}
• Journal : Author [last name], A.A [first and middle initial]. (Publication Year). Publication Title. Periodical Title, Volume(Issue), pp.-pp.
• Newspaper : Author [last name], A.A [first and middle initial]. (Year, Month, Date of Publication). Publication Title. Magazine Title, pp. xx-xx.

Using MLA Format

• Web : Last, First Middle Initial. “Publication Title.” Website Title. Website Publisher, Date Month Year Published. Web. Date Month Year Accessed.
• Newspaper : Last, First M. “Publication Title.” Newspaper Title [City] Date, Month, Year Published: Page(s). Print.
• Journal : Last, First M. “Publication Title.” Journal Title Series Volume. Issue (Year Published): Page(s). Database Name. Web. Date Month Year Accessed.

Enhance your writing effortlessly with EssayPro.com , where you can order an article review or any other writing task. Our team of expert writers specializes in various fields, ensuring your work is not just summarized, but deeply analyzed and professionally presented. Ideal for students and professionals alike, EssayPro offers top-notch writing assistance tailored to your needs. Elevate your writing today with our skilled team at your article review writing service !

The Pre-Writing Process

Facing this task for the first time can really get confusing and can leave you unsure of where to begin. To create a top-notch article review, start with a few preparatory steps. Here are the two main stages from our dissertation services to get you started:

Step 1: Define the right organization for your review. Knowing the future setup of your paper will help you define how you should read the article. Here are the steps to follow:

• Summarize the article — seek out the main points, ideas, claims, and general information presented in the article.
• Define the positive points — identify the strong aspects, ideas, and insightful observations the author has made.
• Find the gaps —- determine whether or not the author has any contradictions, gaps, or inconsistencies in the article and evaluate whether or not he or she used a sufficient amount of arguments and information to support his or her ideas.
• Identify unanswered questions — finally, identify if there are any questions left unanswered after reading the piece.

Step 2: Move on and review the article. Here is a small and simple guide to help you do it right:

• Start off by looking at and assessing the title of the piece, its abstract, introductory part, headings and subheadings, opening sentences in its paragraphs, and its conclusion.
• First, read only the beginning and the ending of the piece (introduction and conclusion). These are the parts where authors include all of their key arguments and points. Therefore, if you start with reading these parts, it will give you a good sense of the author's main points.
• Finally, read the article fully.

These three steps make up most of the prewriting process. After you are done with them, you can move on to writing your own review—and we are going to guide you through the writing process as well.

Outline and Template

As you progress with reading your article, organize your thoughts into coherent sections in an outline. As you read, jot down important facts, contributions, or contradictions. Identify the shortcomings and strengths of your publication. Begin to map your outline accordingly.

If your professor does not want a summary section or a personal critique section, then you must alleviate those parts from your writing. Much like other assignments, an article review must contain an introduction, a body, and a conclusion. Thus, you might consider dividing your outline according to these sections as well as subheadings within the body. If you find yourself troubled with the pre-writing and the brainstorming process for this assignment, seek out a sample outline.

Your custom essay must contain these constituent parts:

• Pre-Title Page - Before diving into your review, start with essential details: article type, publication title, and author names with affiliations (position, department, institution, location, and email). Include corresponding author info if needed.
• Running Head - In APA format, use a concise title (under 40 characters) to ensure consistent formatting.
• Summary Page - Optional but useful. Summarize the article in 800 words, covering background, purpose, results, and methodology, avoiding verbatim text or references.
• Title Page - Include the full title, a 250-word abstract, and 4-6 keywords for discoverability.
• Introduction - Set the stage with an engaging overview of the article.
• Body - Organize your analysis with headings and subheadings.
• Works Cited/References - Properly cite all sources used in your review.
• Optional Suggested Reading Page - If permitted, suggest further readings for in-depth exploration.
• Tables and Figure Legends (if instructed by the professor) - Include visuals when requested by your professor for clarity.

Example of an Article Review

You might wonder why we've dedicated a section of this article to discuss an article review sample. Not everyone may realize it, but examining multiple well-constructed examples of review articles is a crucial step in the writing process. In the following section, our essay writing service experts will explain why.

Looking through relevant article review examples can be beneficial for you in the following ways:

• To get you introduced to the key works of experts in your field.
• To help you identify the key people engaged in a particular field of science.
• To help you define what significant discoveries and advances were made in your field.
• To help you unveil the major gaps within the existing knowledge of your field—which contributes to finding fresh solutions.
• To help you find solid references and arguments for your own review.
• To help you generate some ideas about any further field of research.
• To help you gain a better understanding of the area and become an expert in this specific field.
• To get a clear idea of how to write a good review.

View Our Writer’s Sample Before Crafting Your Own!

Why Have There Been No Great Female Artists?

Steps for Writing an Article Review

Here is a guide with critique paper format on how to write a review paper:

Step 1: Write the Title

First of all, you need to write a title that reflects the main focus of your work. Respectively, the title can be either interrogative, descriptive, or declarative.

Step 2: Cite the Article

Next, create a proper citation for the reviewed article and input it following the title. At this step, the most important thing to keep in mind is the style of citation specified by your instructor in the requirements for the paper. For example, an article citation in the MLA style should look as follows:

Author's last and first name. "The title of the article." Journal's title and issue(publication date): page(s). Print

Abraham John. "The World of Dreams." Virginia Quarterly 60.2(1991): 125-67. Print.

Step 3: Article Identification

After your citation, you need to include the identification of your reviewed article:

• Title of the article
• Title of the journal
• Year of publication

All of this information should be included in the first paragraph of your paper.

The report "Poverty increases school drop-outs" was written by Brian Faith – a Health officer – in 2000.

Step 4: Introduction

Your organization in an assignment like this is of the utmost importance. Before embarking on your writing process, you should outline your assignment or use an article review template to organize your thoughts coherently.

• If you are wondering how to start an article review, begin with an introduction that mentions the article and your thesis for the review.
• Follow up with a summary of the main points of the article.
• Highlight the positive aspects and facts presented in the publication.
• Critique the publication by identifying gaps, contradictions, disparities in the text, and unanswered questions.

Step 5: Summarize the Article

Make a summary of the article by revisiting what the author has written about. Note any relevant facts and findings from the article. Include the author's conclusions in this section.

Step 6: Critique It

Present the strengths and weaknesses you have found in the publication. Highlight the knowledge that the author has contributed to the field. Also, write about any gaps and/or contradictions you have found in the article. Take a standpoint of either supporting or not supporting the author's assertions, but back up your arguments with facts and relevant theories that are pertinent to that area of knowledge. Rubrics and templates can also be used to evaluate and grade the person who wrote the article.

Step 7: Craft a Conclusion

In this section, revisit the critical points of your piece, your findings in the article, and your critique. Also, write about the accuracy, validity, and relevance of the results of the article review. Present a way forward for future research in the field of study. Before submitting your article, keep these pointers in mind:

• As you read the article, highlight the key points. This will help you pinpoint the article's main argument and the evidence that they used to support that argument.
• While you write your review, use evidence from your sources to make a point. This is best done using direct quotations.
• Select quotes and supporting evidence adequately and use direct quotations sparingly. Take time to analyze the article adequately.
• Every time you reference a publication or use a direct quotation, use a parenthetical citation to avoid accidentally plagiarizing your article.
• Re-read your piece a day after you finish writing it. This will help you to spot grammar mistakes and to notice any flaws in your organization.
• Use a spell-checker and get a second opinion on your paper.

The Post-Writing Process: Proofread Your Work

Finally, when all of the parts of your article review are set and ready, you have one last thing to take care of — proofreading. Although students often neglect this step, proofreading is a vital part of the writing process and will help you polish your paper to ensure that there are no mistakes or inconsistencies.

To proofread your paper properly, start by reading it fully and checking the following points:

• Punctuation
• Other mistakes

Afterward, take a moment to check for any unnecessary information in your paper and, if found, consider removing it to streamline your content. Finally, double-check that you've covered at least 3-4 key points in your discussion.

And remember, if you ever need help with proofreading, rewriting your essay, or even want to buy essay , our friendly team is always here to assist you.

Need an Article REVIEW WRITTEN?

Just send us the requirements to your paper and watch one of our writers crafting an original paper for you.

What Is A Review Article?

How to write an article review, how to write an article review in apa format.

Daniel Parker

is a seasoned educational writer focusing on scholarship guidance, research papers, and various forms of academic essays including reflective and narrative essays. His expertise also extends to detailed case studies. A scholar with a background in English Literature and Education, Daniel’s work on EssayPro blog aims to support students in achieving academic excellence and securing scholarships. His hobbies include reading classic literature and participating in academic forums.

is an expert in nursing and healthcare, with a strong background in history, law, and literature. Holding advanced degrees in nursing and public health, his analytical approach and comprehensive knowledge help students navigate complex topics. On EssayPro blog, Adam provides insightful articles on everything from historical analysis to the intricacies of healthcare policies. In his downtime, he enjoys historical documentaries and volunteering at local clinics.

Related Articles

How to write a journal article review: Do the writing

• What's in this Guide
• What is a journal article?
• Create a template
• Choose your article to review
• Read your article carefully

Do the writing

• Remember to edit
• Additional resources

Start to write. Follow the instructions of your assessment, then structure your writing accordingly.

The four key parts of a journal article review are:

3. A critique, or a discussion about the key points of the journal article.

A critique is a discussion about the key points of the journal article. It should be a balanced discussion about the  strengths and weaknesses of the key points and structure of the article.

You will also need to discuss if the author(s) points are valid (supported by other literature) and robust (would you get the same outcome if the way the information was gathered was repeated).

Example of part of a critique

4. A conclusion - a final evaluation of the article

1. Give an overall opinion of the text.

2. Briefly summarise key points and determine if they are valid, useful, accurate etc.

3. Remember, do not include new ideas or opinions in the conclusion.

Pathways and Academic Learning Support

• << Previous: Read your article carefully
• Next: Remember to edit >>
• Last Updated: Apr 27, 2023 4:28 PM
• URL: https://libguides.newcastle.edu.au/how-to-write-a-journal-article-review

How to Write an Article Review: Practical Tips and Examples

Table of contents

• 1 What Is an Article Review?
• 2 Different Types of Article Review
• 3.1 Critical review
• 3.2 Literature review
• 3.3 Mapping review/systematic map
• 3.4 Meta-analysis
• 3.5 Overview
• 3.6 Qualitative Systematic Review/Qualitative Evidence Synthesis
• 3.7 Rapid review
• 3.8 Scoping review
• 3.9 Systematic review
• 3.10 Umbrella review
• 4 Formatting
• 5 How To Write An Article Review
• 6 Article Review Outline
• 7 10 Tips for Writing an Article Review
• 8 An Article Review Example

What Is an Article Review?

Before you get started, learn what an article review is. It can be defined as a work that combines elements of summary and critical analysis. If you are writing an article review, you should take a close look at another author’s work. Many experts regularly practice evaluating the work of others. The purpose of this is to improve writing skills.

This kind of work belongs to professional pieces of writing because the process of crafting this paper requires reviewing, summarizing, and understanding the topic. Only experts are able to compose really good reviews containing a logical evaluation of a paper as well as a critique.

Your task is not to provide new information. You should process what you have in a certain publication.

Different Types of Article Review

In academic writing, the landscape of article reviews is diverse and nuanced, encompassing a variety of formats that cater to different research purposes and methodologies. Among these, three main types of article reviews stand out due to their distinct approaches and applications:

• Narrative. The basic focus here is the author’s personal experience. Judgments are presented through the prism of experiences and subsequent realizations. Besides, the use of emotional recollections is acceptable.
• Evidence. There is a significant difference from the narrative review. An in-depth study of the subject is assumed, and conclusions are built on arguments. The author may consider theories or concrete facts to support that.
• Systematic. The structure of the piece explains the approach to writing. The answer to what’s a systematic review lies on the surface. The writer should pay special attention to the chronology and logic of the narrative.

Understanding 10 Common Types

Don`t rush looking at meta-analysis vs. systematic review. We recommend that you familiarize yourself with other formats and topics of texts. This will allow you to understand the types of essays better and select them based on your request. For this purpose, we`ll discuss the typology of reviews below.

Critical review

The critical review definition says that the author must be objective and have arguments for each thought. Sometimes, amateur authors believe that they should “criticize” something. However, it is important to understand the difference since objectivity and the absence of emotional judgments are prioritized. The structure of this type of review article is as follows:

• Introduction;
• Conclusion.

“Stuffing” of the text is based on such elements as methodology, argumentation, evidence, and theory base. The subject of study is stated at the beginning of the material. Then follows the transition to the main part (facts). The final word summarizes all the information voiced earlier.

It is a mistake to believe that critical reviews are devoid of evaluation. The author’s art lies in maneuvering between facts. Smooth transition from one argument to another and lays out the conclusions in the reader. That is why such texts are used in science. The critical reviews meaning is especially tangible in medical topics.

Literature review

Literature is the basis for this type of work ─ books, essays, and articles become a source of information. Thus, the author should rethink the voiced information. After that, it is possible to proceed to conclusions. The methodology aims to find interconnections, repetitions, and even “gaps” in the literature. One important item is the referencing of sources. Footnotes are possible in the work itself or the list of resources used.

These types of research reviews often explore myths since there are often inconsistencies in mythology. Sometimes, there is contrary information. In this case, the author has to gather all existing theories. The essence does not always lie in the confirmation of facts. There are other different types of reviews for this purpose. In literary reviews, the object of study may be characters or traditions. This is where the author’s space for discovery opens up. Inconsistencies in the data can tell important details about particular periods or cultures. At the same time, patterns reveal well-established facts. Make sure to outline your work before you write. This will help you with essay writing .

Mapping review/systematic map

A mapping review, also known as a systematic map, is a unique approach to surveying and organizing existing literature, providing a panoramic view of the research landscape. This paper systematically categorizes and maps out the available literature on a particular topic, emphasizing breadth over depth. Its primary goal is to present a comprehensive visual representation of the research distribution, offering insights into the overall scope of a subject.

One of the strengths of systematic reviews is that they deeply focus on a research question with detailed analysis and synthesis, while mapping review prioritizes breadth. It identifies and categorizes a broad range of studies without necessarily providing in-depth critique or content synthesis. This approach allows for a broader understanding of the field, making it especially useful in the early stages of research. Mapping reviews excel in identifying gaps in the existing body of literature.

By systematically mapping the distribution of research, researchers can pinpoint areas where studies are scarce or nonexistent, helping to guide future research directions. This makes mapping reviews a valuable tool for researchers seeking to contribute meaningfully to a field by addressing unexplored or underexplored areas.

Meta-analysis

Meta-analysis is a powerful statistical technique. It systematically combines the results of multiple studies to derive comprehensive and nuanced insights. This method goes beyond the limitations of individual studies, offering a more robust understanding of a particular phenomenon by synthesizing data from diverse sources.

Meta-analysis employs a rigorous methodology. It involves the systematic collection and statistical integration of data from multiple studies. This methodological rigor ensures a standardized and unbiased approach to data synthesis. It is applied across various disciplines, from medicine and psychology to social sciences, providing a quantitative assessment of the overall effect of an intervention or the strength of an association.

In evidence-based fields, where informed decision-making relies on a thorough understanding of existing research, meta-analysis plays a pivotal role. It offers a quantitative overview of the collective evidence, helping researchers, policymakers, and practitioners make more informed decisions. By synthesizing results from diverse studies, meta-analysis contributes to the establishment of robust evidence-based practices, enhancing the reliability and credibility of findings in various fields. To present your research findings in the most readable way possible, learn how to write a summary of article .

If the key purpose of systematic review is to maximize the disclosure of facts, the opposite is true here. Imagine a video shot by a quadcopter from an altitude. The viewer sees a vast area of terrain without focusing on individual details. Overviews follow the same principle. The author gives a general picture of the events or objects described.

These types of reviews often seem simple. However, the role of the researcher becomes a very demanding one. The point is not just to list facts. Here, the search for information comes to the fore. After all, it is such reports that, in the future, will provide the basis for researching issues more narrowly. In essence, you yourself create a new source of information ─ students who worry that somebody may critique the author’s article love this type of material. However, there are no questions for the author; they just set the stage for discussions in different fields.

An example of this type of report would be a collection of research results from scientists. For example, statistics on the treatment of patients with certain diseases. In such a case, reference is made to scientific articles and doctrines. Based on this information, readers can speak about the effectiveness of certain treatment methods.

Qualitative Systematic Review/Qualitative Evidence Synthesis

One of the next types of review articles represents a meticulous effort to synthesize and analyze qualitative studies within a specific research domain.

The focus is synthesizing qualitative studies, employing a systematic and rigorous approach to extract meaningful insights. Its significance lies in its ability to provide a nuanced understanding of complex phenomena, offering a qualitative lens to complement quantitative analyses. Researchers can uncover patterns, themes, and contextual nuances that may elude traditional quantitative approaches by systematically reviewing and synthesizing qualitative data.

Often, you may meet discussion: is a systematic review quantitative or qualitative? The application of qualitative systematic reviews extends across diverse research domains, from healthcare and social sciences to education and psychology. For example, this approach can offer a comprehensive understanding of patient experiences and preferences in healthcare. In social sciences, it can illuminate cultural or societal dynamics. Its versatility makes it a valuable tool for researchers exploring, interpreting, and integrating qualitative findings to enrich their understanding of complex phenomena within their respective fields.

Rapid review

If you don’t know how to write an article review , try starting with this format. It is the complete opposite of everything we talked about above. The key advantage and feature is speed. Quick overviews are used when time is limited. The focus can go to individual details (key). Often, the focus is still on the principal points.

Often, these types of review papers are critically needed in politics. This method helps to communicate important information to the reader quickly. An example can be a comparison of the election programs of two politicians. The author can show the key differences. Or it can make an overview based on the theses of the opponents’ proposals on different topics.

Seeming simplicity becomes power. Such texts allow the reader to make a quick decision. The author’s task is to understand potential interests and needs. Then, highlight and present the most important data as concisely as possible. In addition to politics, such reports are often used in communications, advertising, and marketing. Experienced writers mention the one-minute principle. This means you can count on 60 seconds of the reader’s attention. If you managed to hook them ─ bravo, you have done the job!

Scoping review

If you read the official scoping review definition, you may find similarities with the systematic type of review. However, recall is a sequential and logical study in the second case. It’s like you stack things on a shelf by color, size, and texture.

This type of review can be more difficult to understand. The basic concept is to explore what is called the field of subjects. This means, on the one hand, exploring a particular topic through the existing data about it. The author tries to find gaps or patterns by drawing on sources of information.

Another good comparison between systematic and this type of review is imagining as if drawing a picture. In the first case, you will think through every nuance and detail, why it is there, and how it “moves the story.” In the second case, it is as if you are painting a picture with “broad strokes.” In doing so, you can explain your motives for choosing the primary color. For example: “I chose the emerald color because all the cultural publications say it’s a trend”. The same goes for texts.

Systematic review

Sometimes, you may encounter a battle: narrative review vs. systematic review. The point is not to compare but to understand the different types of papers. Once you understand their purpose, you can present your data better and choose a more readable format. The systematic approach can be called the most scientific. Such a review relies on the following steps:

• Literature search;
• Evaluating the information;
• Data processing;
• Careful analysis of the material.

It is the fourth point that is key. The writer should carefully process the information before using it. However, 80% of your work’s result depends on this stage’s seriousness.

A rigorous approach to data selection produces an array of factual data. That is why this method is so often used in science, education, and social fields. Where accuracy is important. At the same time, the popularity of this approach is growing in other directions.

Systematic reviews allow for using different data and methodologies,, but with one important caveat ─ if the author manages to keep the narrative structured and explain the reason for certain methods. It is not about rigor. The task of this type of review is to preserve the facts, which dictates consistency and rationality.

Umbrella review

An umbrella review is a distinctive approach that involves the review of existing reviews, providing a comprehensive synthesis of evidence on a specific topic. The methodology of an umbrella review entails systematically examining and summarizing findings from multiple systematic reviews and meta-analyses.

This method ensures a rigorous and consolidated analysis of the existing evidence. The application of an umbrella review is broad, spanning various fields such as medicine, public health, and social sciences. It is particularly useful when a substantial body of systematic reviews exists, allowing researchers to draw overarching conclusions from the collective findings.

It allows the summarization of existing reviews and provides a new perspective on individual subtopics of the main object of study. In the context of the umbrella method, the comparison “bird’s eye view” is often cited. A bird in flight can see the whole panorama and shift its gaze to specific objects simultaneously. What becomes relevant at a particular moment? The author will face the same task.

On the one hand, you must delve into the offshoots of the researched topic. On the other hand, focus on the topic or object of study as a whole. Such a concept allows you to open up new perspectives and thoughts.

Different types of formatting styles are used for article review writing. It mainly depends on the guidelines that are provided by the instructor, sometimes, professors even provide an article review template that needs to be followed.

Here are some common types of formatting styles that you should be aware of when you start writing an article review:

• APA (American Psychological Association) – An APA format article review is commonly used for social sciences. It has guidelines for formatting the title, abstract, body paragraphs, and references. For example, the title of an article in APA format is in sentence case, whereas the publication title is in title case.
• MLA (Modern Language Association): This is a formatting style often used in humanities, such as language studies and literature. There are specific guidelines for the formatting of the title page, header, footer, and citation style.
• Chicago Manual of Style: This is one of the most commonly used formatting styles. It is often used for subjects in humanities and social sciences, but also commonly found in a newspaper title. This includes guidelines for formatting the title page, end notes, footnotes, publication title, article citation, and bibliography.
• Harvard Style: Harvard style is commonly used for social sciences and provides specific guidelines for formatting different sections of the pages, including publication title, summary page, website publisher, and more.

To ensure that your article review paper is properly formatted and meets the requirements, it is crucial to adhere to the specific guidelines for the formatting style you are using. This helps you write a good article review.

• Free unlimited checks
• All common file formats
• Accurate results
• Intuitive interface

How To Write An Article Review

There are several steps that must be followed when you are starting to review articles. You need to follow these to make sure that your thoughts are organized properly. In this way, you can present your ideas in a more concise and clear manner. Here are some tips on how to start an article review and how to cater to each writing stage.

• Read the Article Closely: Even before you start to write an article review, it’s important to make sure that you have read the specific article thoroughly. Write down the central points and all the supporting ideas. It’s important also to note any questions or comments that you have about the content.
• Identify the Thesis: Make sure that you understand the author’s main points, and identify the main thesis of the article. This will help you focus on your review and ensure that you are addressing all of the key points.
• Formulate an Introduction: The piece should start with an introduction that has all the necessary background information, possibly in the first paragraph or in the first few paragraphs. This can include a brief summary of the important points or an explanation of the importance.
• Summarize the Article : Summarize the main points when you review the article, and make sure that you include all supporting elements of the author’s thesis.
• Start with Personal Critique : Now is the time to include a personal opinion on the research article or the journal article review. Start with evaluating all the strengths and weaknesses of the reviewed article. Discuss all of the flaws that you found in the author’s evidence and reasoning. Also, point out whether the conclusion provided by the author was well presented or not.
• Add Personal Perspective: Offer your perspective on the original article, do you agree or disagree with the ideas that the article supports or not. Your critical review, in your own words, is an essential part of a good review. Make sure you address all unanswered questions in your review.
• Conclude the Article Review : In this section of the writing process, you need to be very careful and wrap up the whole discussion in a coherent manner. This is should summarize all the main points and offer an overall assessment.

Make sure to stay impartial and provide proof to back up your assessment. By adhering to these guidelines, you can create a reflective and well-structured article review.

Article Review Outline

Here is a basic, detailed outline for an article review you should be aware of as a pre-writing process if you are wondering how to write an article review.

Introduction

• Introduce the article that you are reviewing (author name, publication date, title, etc.) Now provide an overview of the article’s main topic

Summary section

• Summarize the key points in the article as well as any arguments Identify the findings and conclusion

Critical Review

• Assess and evaluate the positive aspects and the drawbacks
• Discuss if the authors arguments were verified by the evidence of the article
• Identify if the text provides substantial information for any future paper or further research
• Assess any gaps in the arguments
• Restate the thesis statement
• Provide a summary for all sections
• Write any recommendations and thoughts that you have on the article
• Never forget to add and cite any references that you used in your article

10 Tips for Writing an Article Review

Have you ever written such an assignment? If not, study the helpful tips for composing a paper. If you follow the recommendations provided here, the process of writing a summary of the article won’t be so time-consuming, and you will be able to write an article in the most effective manner.

The guidelines below will help to make the process of preparing a paper much more productive. Let’s get started!

• Check what kind of information your work should contain. After answering the key question “What is an article review?” you should learn how to structure it the right way. To succeed, you need to know what your work should be based on. An analysis with insightful observations is a must for your piece of writing.
• Identify the central idea: In your first reading, focus on the overall impression. Gather ideas about what the writer wants to tell, and consider whether he or she managed to achieve it.
• Look up unfamiliar terms. Don’t know what certain words and expressions mean? Highlight them, and don’t forget to check what they mean with a reliable source of information.
• Highlight the most important ideas. If you are reading it a second time, use a highlighter to highlight the points that are most important to understanding the passage.
• Write an outline. A well-written outline will make your life a lot easier. All your thoughts will be grouped. Detailed planning helps not to miss anything important. Think about the questions you should answer when writing.
• Brainstorm headline ideas. When choosing a project, remember: it should reflect the main idea. Make it bold and concise.
• Check an article review format example. You should check that you know how to cite an article properly. Note that citation rules are different in APA and MLA formats. Ask your teacher which one to prioritize.
• Write a good introduction. Use only one short paragraph to state the central idea of ​​the work. Emphasize the author’s key concepts and arguments. Add the thesis at the end of the Introduction.
• Write in a formal style. Use the third person, remembering that this assignment should be written in a formal academic writing style.
• Wrap up, offer your critique, and close. Give your opinion on whether the author achieved his goals. Mention the shortcomings of the job, if any, and highlight its strengths.

If you have checked the tips and you still doubt whether you have all the necessary skills and time to prepare this kind of educational work, follow one more tip that guarantees 100% success- ask for professional assistance by asking the custom writing service PapersOwl to craft your paper instead of you. Just submit an order online and get the paper completed by experts.

An Article Review Example

If you have a task to prepare an analysis of a certain piece of literature, have a look at the article review sample. There is an article review example for you to have a clear picture of what it must look like.

Journal Article on Ayn Rand’s Works Review Example

“The purpose of the article is to consider the features of the poetics of Ayn Rand’s novels “Atlas Shrugged,” “We the living,” and “The Fountainhead.” In the analysis of the novels, the structural-semantic and the method of comparative analysis were used.

With the help of these methods, genre features of the novels were revealed, and a single conflict and a cyclic hero were identified.

In-depth reading allows us to more fully reveal the worldview of the author reflected in the novels. It becomes easier to understand the essence of the author’s ideas about the connection between being and consciousness, embodied in cyclic ideas and images of plot twists and heroes. The author did a good job highlighting the strong points of the works and mentioning the reasons for the obvious success of Ayn Rand.“

You can also search for other relevant article review examples before you start.

In conclusion, article reviews play an important role in evaluating and analyzing different scholarly articles. Writing a review requires critical thinking skills and a deep understanding of the article’s content, style, and structure. It is crucial to identify the type of article review and follow the specific guidelines for formatting style provided by the instructor or professor.

The process of writing an article review requires several steps, such as reading the article attentively, identifying the thesis, and formulating an introduction. By following the tips and examples provided in this article, students can write a worthy review that demonstrates their ability to evaluate and critique another writer’s work.

Learning how to write an article review is a critical skill for students and professionals alike. Before diving into the nitty-gritty of reviewing an article, it’s important to understand what an article review is and the elements it should include. An article review is an assessment of a piece of writing that summarizes and evaluates a work. To complete a quality article review, the author should consider the text’s purpose and content, its organization, the author’s style, and how the article fits into a larger conversation. But if you don’t have the time to do all of this work, you can always purchase a literature review from Papers Owl .

Readers also enjoyed

WHY WAIT? PLACE AN ORDER RIGHT NOW!

Just fill out the form, press the button, and have no worries!

We use cookies to give you the best experience possible. By continuing we’ll assume you board with our cookie policy.

• Privacy Policy

Home » Research Findings – Types Examples and Writing Guide

Research Findings – Types Examples and Writing Guide

Table of Contents

Research Findings

Definition:

Research findings refer to the results obtained from a study or investigation conducted through a systematic and scientific approach. These findings are the outcomes of the data analysis, interpretation, and evaluation carried out during the research process.

Types of Research Findings

There are two main types of research findings:

Qualitative Findings

Qualitative research is an exploratory research method used to understand the complexities of human behavior and experiences. Qualitative findings are non-numerical and descriptive data that describe the meaning and interpretation of the data collected. Examples of qualitative findings include quotes from participants, themes that emerge from the data, and descriptions of experiences and phenomena.

Quantitative Findings

Quantitative research is a research method that uses numerical data and statistical analysis to measure and quantify a phenomenon or behavior. Quantitative findings include numerical data such as mean, median, and mode, as well as statistical analyses such as t-tests, ANOVA, and regression analysis. These findings are often presented in tables, graphs, or charts.

Both qualitative and quantitative findings are important in research and can provide different insights into a research question or problem. Combining both types of findings can provide a more comprehensive understanding of a phenomenon and improve the validity and reliability of research results.

Parts of Research Findings

Research findings typically consist of several parts, including:

• Introduction: This section provides an overview of the research topic and the purpose of the study.
• Literature Review: This section summarizes previous research studies and findings that are relevant to the current study.
• Methodology : This section describes the research design, methods, and procedures used in the study, including details on the sample, data collection, and data analysis.
• Results : This section presents the findings of the study, including statistical analyses and data visualizations.
• Discussion : This section interprets the results and explains what they mean in relation to the research question(s) and hypotheses. It may also compare and contrast the current findings with previous research studies and explore any implications or limitations of the study.
• Conclusion : This section provides a summary of the key findings and the main conclusions of the study.
• Recommendations: This section suggests areas for further research and potential applications or implications of the study’s findings.

How to Write Research Findings

Writing research findings requires careful planning and attention to detail. Here are some general steps to follow when writing research findings:

• Organize your findings: Before you begin writing, it’s essential to organize your findings logically. Consider creating an outline or a flowchart that outlines the main points you want to make and how they relate to one another.
• Use clear and concise language : When presenting your findings, be sure to use clear and concise language that is easy to understand. Avoid using jargon or technical terms unless they are necessary to convey your meaning.
• Use visual aids : Visual aids such as tables, charts, and graphs can be helpful in presenting your findings. Be sure to label and title your visual aids clearly, and make sure they are easy to read.
• Use headings and subheadings: Using headings and subheadings can help organize your findings and make them easier to read. Make sure your headings and subheadings are clear and descriptive.
• Interpret your findings : When presenting your findings, it’s important to provide some interpretation of what the results mean. This can include discussing how your findings relate to the existing literature, identifying any limitations of your study, and suggesting areas for future research.
• Be precise and accurate : When presenting your findings, be sure to use precise and accurate language. Avoid making generalizations or overstatements and be careful not to misrepresent your data.
• Edit and revise: Once you have written your research findings, be sure to edit and revise them carefully. Check for grammar and spelling errors, make sure your formatting is consistent, and ensure that your writing is clear and concise.

Research Findings Example

Following is a Research Findings Example sample for students:

Title: The Effects of Exercise on Mental Health

Sample : 500 participants, both men and women, between the ages of 18-45.

Methodology : Participants were divided into two groups. The first group engaged in 30 minutes of moderate intensity exercise five times a week for eight weeks. The second group did not exercise during the study period. Participants in both groups completed a questionnaire that assessed their mental health before and after the study period.

Findings : The group that engaged in regular exercise reported a significant improvement in mental health compared to the control group. Specifically, they reported lower levels of anxiety and depression, improved mood, and increased self-esteem.

Conclusion : Regular exercise can have a positive impact on mental health and may be an effective intervention for individuals experiencing symptoms of anxiety or depression.

Applications of Research Findings

Research findings can be applied in various fields to improve processes, products, services, and outcomes. Here are some examples:

• Healthcare : Research findings in medicine and healthcare can be applied to improve patient outcomes, reduce morbidity and mortality rates, and develop new treatments for various diseases.
• Education : Research findings in education can be used to develop effective teaching methods, improve learning outcomes, and design new educational programs.
• Technology : Research findings in technology can be applied to develop new products, improve existing products, and enhance user experiences.
• Business : Research findings in business can be applied to develop new strategies, improve operations, and increase profitability.
• Public Policy: Research findings can be used to inform public policy decisions on issues such as environmental protection, social welfare, and economic development.
• Social Sciences: Research findings in social sciences can be used to improve understanding of human behavior and social phenomena, inform public policy decisions, and develop interventions to address social issues.
• Agriculture: Research findings in agriculture can be applied to improve crop yields, develop new farming techniques, and enhance food security.
• Sports : Research findings in sports can be applied to improve athlete performance, reduce injuries, and develop new training programs.

When to use Research Findings

Research findings can be used in a variety of situations, depending on the context and the purpose. Here are some examples of when research findings may be useful:

• Decision-making : Research findings can be used to inform decisions in various fields, such as business, education, healthcare, and public policy. For example, a business may use market research findings to make decisions about new product development or marketing strategies.
• Problem-solving : Research findings can be used to solve problems or challenges in various fields, such as healthcare, engineering, and social sciences. For example, medical researchers may use findings from clinical trials to develop new treatments for diseases.
• Policy development : Research findings can be used to inform the development of policies in various fields, such as environmental protection, social welfare, and economic development. For example, policymakers may use research findings to develop policies aimed at reducing greenhouse gas emissions.
• Program evaluation: Research findings can be used to evaluate the effectiveness of programs or interventions in various fields, such as education, healthcare, and social services. For example, educational researchers may use findings from evaluations of educational programs to improve teaching and learning outcomes.
• Innovation: Research findings can be used to inspire or guide innovation in various fields, such as technology and engineering. For example, engineers may use research findings on materials science to develop new and innovative products.

Purpose of Research Findings

The purpose of research findings is to contribute to the knowledge and understanding of a particular topic or issue. Research findings are the result of a systematic and rigorous investigation of a research question or hypothesis, using appropriate research methods and techniques.

The main purposes of research findings are:

• To generate new knowledge : Research findings contribute to the body of knowledge on a particular topic, by adding new information, insights, and understanding to the existing knowledge base.
• To test hypotheses or theories : Research findings can be used to test hypotheses or theories that have been proposed in a particular field or discipline. This helps to determine the validity and reliability of the hypotheses or theories, and to refine or develop new ones.
• To inform practice: Research findings can be used to inform practice in various fields, such as healthcare, education, and business. By identifying best practices and evidence-based interventions, research findings can help practitioners to make informed decisions and improve outcomes.
• To identify gaps in knowledge: Research findings can help to identify gaps in knowledge and understanding of a particular topic, which can then be addressed by further research.
• To contribute to policy development: Research findings can be used to inform policy development in various fields, such as environmental protection, social welfare, and economic development. By providing evidence-based recommendations, research findings can help policymakers to develop effective policies that address societal challenges.

Characteristics of Research Findings

Research findings have several key characteristics that distinguish them from other types of information or knowledge. Here are some of the main characteristics of research findings:

• Objective : Research findings are based on a systematic and rigorous investigation of a research question or hypothesis, using appropriate research methods and techniques. As such, they are generally considered to be more objective and reliable than other types of information.
• Empirical : Research findings are based on empirical evidence, which means that they are derived from observations or measurements of the real world. This gives them a high degree of credibility and validity.
• Generalizable : Research findings are often intended to be generalizable to a larger population or context beyond the specific study. This means that the findings can be applied to other situations or populations with similar characteristics.
• Transparent : Research findings are typically reported in a transparent manner, with a clear description of the research methods and data analysis techniques used. This allows others to assess the credibility and reliability of the findings.
• Peer-reviewed: Research findings are often subject to a rigorous peer-review process, in which experts in the field review the research methods, data analysis, and conclusions of the study. This helps to ensure the validity and reliability of the findings.
• Reproducible : Research findings are often designed to be reproducible, meaning that other researchers can replicate the study using the same methods and obtain similar results. This helps to ensure the validity and reliability of the findings.

Advantages of Research Findings

Research findings have many advantages, which make them valuable sources of knowledge and information. Here are some of the main advantages of research findings:

• Evidence-based: Research findings are based on empirical evidence, which means that they are grounded in data and observations from the real world. This makes them a reliable and credible source of information.
• Inform decision-making: Research findings can be used to inform decision-making in various fields, such as healthcare, education, and business. By identifying best practices and evidence-based interventions, research findings can help practitioners and policymakers to make informed decisions and improve outcomes.
• Identify gaps in knowledge: Research findings can help to identify gaps in knowledge and understanding of a particular topic, which can then be addressed by further research. This contributes to the ongoing development of knowledge in various fields.
• Improve outcomes : Research findings can be used to develop and implement evidence-based practices and interventions, which have been shown to improve outcomes in various fields, such as healthcare, education, and social services.
• Foster innovation: Research findings can inspire or guide innovation in various fields, such as technology and engineering. By providing new information and understanding of a particular topic, research findings can stimulate new ideas and approaches to problem-solving.
• Enhance credibility: Research findings are generally considered to be more credible and reliable than other types of information, as they are based on rigorous research methods and are subject to peer-review processes.

Limitations of Research Findings

While research findings have many advantages, they also have some limitations. Here are some of the main limitations of research findings:

• Limited scope: Research findings are typically based on a particular study or set of studies, which may have a limited scope or focus. This means that they may not be applicable to other contexts or populations.
• Potential for bias : Research findings can be influenced by various sources of bias, such as researcher bias, selection bias, or measurement bias. This can affect the validity and reliability of the findings.
• Ethical considerations: Research findings can raise ethical considerations, particularly in studies involving human subjects. Researchers must ensure that their studies are conducted in an ethical and responsible manner, with appropriate measures to protect the welfare and privacy of participants.
• Time and resource constraints : Research studies can be time-consuming and require significant resources, which can limit the number and scope of studies that are conducted. This can lead to gaps in knowledge or a lack of research on certain topics.
• Complexity: Some research findings can be complex and difficult to interpret, particularly in fields such as science or medicine. This can make it challenging for practitioners and policymakers to apply the findings to their work.
• Lack of generalizability : While research findings are intended to be generalizable to larger populations or contexts, there may be factors that limit their generalizability. For example, cultural or environmental factors may influence how a particular intervention or treatment works in different populations or contexts.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Dissertation – Format, Example and Template

Appendices – Writing Guide, Types and Examples

Appendix in Research Paper – Examples and...

Research Summary – Structure, Examples and...

Limitations in Research – Types, Examples and...

Data Verification – Process, Types and Examples

• Research Process

Writing a good review article

• 3 minute read
• 81.8K views

Table of Contents

As a young researcher, you might wonder how to start writing your first review article, and the extent of the information that it should contain. A review article is a comprehensive summary of the current understanding of a specific research topic and is based on previously published research. Unlike research papers, it does not contain new results, but can propose new inferences based on the combined findings of previous research.

Types of review articles

Review articles are typically of three types: literature reviews, systematic reviews, and meta-analyses.

A literature review is a general survey of the research topic and aims to provide a reliable and unbiased account of the current understanding of the topic.

A systematic review , in contrast, is more specific and attempts to address a highly focused research question. Its presentation is more detailed, with information on the search strategy used, the eligibility criteria for inclusion of studies, the methods utilized to review the collected information, and more.

A meta-analysis is similar to a systematic review in that both are systematically conducted with a properly defined research question. However, unlike the latter, a meta-analysis compares and evaluates a defined number of similar studies. It is quantitative in nature and can help assess contrasting study findings.

Tips for writing a good review article

Here are a few practices that can make the time-consuming process of writing a review article easier:

• Define your question: Take your time to identify the research question and carefully articulate the topic of your review paper. A good review should also add something new to the field in terms of a hypothesis, inference, or conclusion. A carefully defined scientific question will give you more clarity in determining the novelty of your inferences.
• Identify credible sources: Identify relevant as well as credible studies that you can base your review on, with the help of multiple databases or search engines. It is also a good idea to conduct another search once you have finished your article to avoid missing relevant studies published during the course of your writing.
• Take notes: A literature search involves extensive reading, which can make it difficult to recall relevant information subsequently. Therefore, make notes while conducting the literature search and note down the source references. This will ensure that you have sufficient information to start with when you finally get to writing.
• Describe the title, abstract, and introduction: A good starting point to begin structuring your review is by drafting the title, abstract, and introduction. Explicitly writing down what your review aims to address in the field will help shape the rest of your article.
• Be unbiased and critical: Evaluate every piece of evidence in a critical but unbiased manner. This will help you present a proper assessment and a critical discussion in your article.
• Include a good summary: End by stating the take-home message and identify the limitations of existing studies that need to be addressed through future studies.
• Ask for feedback: Ask a colleague to provide feedback on both the content and the language or tone of your article before you submit it.
• Check your journal’s guidelines: Some journals only publish reviews, while some only publish research articles. Further, all journals clearly indicate their aims and scope. Therefore, make sure to check the appropriateness of a journal before submitting your article.

Writing review articles, especially systematic reviews or meta-analyses, can seem like a daunting task. However, Elsevier Author Services can guide you by providing useful tips on how to write an impressive review article that stands out and gets published!

• Manuscript Preparation

What are Implications in Research?

How to write the results section of a research paper

You may also like.

Descriptive Research Design and Its Myriad Uses

Five Common Mistakes to Avoid When Writing a Biomedical Research Paper

Making Technical Writing in Environmental Engineering Accessible

To Err is Not Human: The Dangers of AI-assisted Academic Writing

When Data Speak, Listen: Importance of Data Collection and Analysis Methods

Choosing the Right Research Methodology: A Guide for Researchers

Why is data validation important in research?

Scholarly Sources: What are They and Where can You Find Them?

Input your search keywords and press Enter.

The Sheridan Libraries

• Write a Literature Review
• Sheridan Libraries
• Find This link opens in a new window
• Evaluate This link opens in a new window

Not every source you found should be included in your annotated bibliography or lit review. Only include the most relevant and most important sources.

Get Organized

• Lit Review Prep Use this template to help you evaluate your sources, create article summaries for an annotated bibliography, and a synthesis matrix for your lit review outline.

Summarize your Sources

Summarize each source: Determine the most important and relevant information from each source, such as the findings, methodology, theories, etc.  Consider using an article summary, or study summary to help you organize and summarize your sources.

Paraphrasing

• Use your own words, and do not copy and paste the abstract
• The library's tutorials about plagiarism are excellent, and will help you with paraphasing correctly

Annotated Bibliographies

     Annotated bibliographies can help you clearly see and understand the research before diving into organizing and writing your literature review.        Although typically part of the "summarize" step of the literature review, annotations should not merely be summaries of each article - instead, they should be critical evaluations of the source, and help determine a source's usefulness for your lit review.  

Definition:

A list of citations on a particular topic followed by an evaluation of the source’s argument and other relevant material including its intended audience, sources of evidence, and methodology

• Explore your topic.
• Appraise issues or factors associated with your professional practice and research topic.
• Help you get started with the literature review.
• Think critically about your topic, and the literature.

Steps to Creating an Annotated Bibliography:

• Find Your Sources
• Read Your Sources
• Identify the Most Relevant Sources
• Cite your Sources
• Write Annotations

Annotated Bibliography Resources

• Purdue Owl Guide
• Cornell Annotated Bibliography Guide
• << Previous: Evaluate
• Next: Synthesize >>
• Last Updated: Sep 26, 2023 10:25 AM
• URL: https://guides.library.jhu.edu/lit-review

How to Write a Review Article

• Types of Review Articles
• Before Writing a Review Article
• Determining Where to Publish
• Searching the Literature
• Citation Management
• Reading a Review Article

What is a Review Article?

The purpose of writing a review article is for knowledge updating concerning a topic.

A review article aims to highlight:

• What has been done?
• What has been found?
• What issues have not been addressed?
• What issues remain to be debated?
• What new issues have been raised?
• What will be the future direction of research?

Similarities and Differences to Original Research Articles

Differences Between Original Research Articles and Review Articles

• An original research article aims to: Provides background information (Intro.) on prior research, Reasons for present study, Issues to be investigated by the present study, Written for experts. Authors describe: Research methods & materials, Data acquisition/analysis tools, Results, Discussion of results.
• Both are Peer-reviewed for: Accuracy, Quality, Biases, Conflict of interest.
• A review article aims to: Extensive survey of published research articles about a specific topic, Critical appraising of research findings, summarize up-to-date research findings, Identify critical issues to be addressed, Written for experts and general audiences, Be a source of original research.

Figure by Zhiyong Han, PhD

• << Previous: Home
• Next: Types of Review Articles >>

• The Interprofessional Health Sciences Library
• 123 Metro Boulevard
• Nutley, NJ 07110
• [email protected]
• Visiting Campus
• News and Events
• Parents and Families
• Web Accessibility
• Career Center
• Public Safety
• Accountability
• Privacy Statements
• Report a Problem
• Login to LibApps

• Our Writers
• How to Order
• Assignment Writing Service
• Report Writing Service
• Buy Coursework
• Dissertation Writing Service
• Research Paper Writing Service
• All Essay Services
• Buy Research Paper
• Buy Term Paper
• Buy Dissertation
• Buy Case study
• Buy Presentation
• Buy Personal statement

Article Review

Article Review Writing: A Complete Step-by-Step Guide with Examples

People also read

Learn How to Write an Editorial on Any Topic

Best Tips on How to Avoid Plagiarism

How to Write a Movie Review - Guide & Examples

A Complete Guide on How to Write a Summary for Students

Write Opinion Essay Like a Pro: A Detailed Guide

Evaluation Essay - Definition, Examples, and Writing Tips

How to Write a Thematic Statement - Tips & Examples

How to Write a Bio - Quick Tips, Structure & Examples

How to Write a Synopsis – A Simple Format & Guide

How to Write a Comparative Essay – A Complete Guide

Visual Analysis Essay - A Writing Guide with Format & Sample

List of Common Social Issues Around the World

Writing Character Analysis - Outline, Steps, and Examples

11 Common Types of Plagiarism Explained Through Examples

A Detailed Guide on How to Write a Poem Step by Step

Detailed Guide on Appendix Writing: With Tips and Examples

Struggling to write a review that people actually want to read? Feeling lost in the details and wondering how to make your analysis stand out?

You're not alone!

Many writers find it tough to navigate the world of article reviews, not sure where to start or how to make their reviews really grab attention.

No worries! 

In this blog, we're going to guide you through the process of writing an article review that stands out. We'll also share tips, and examples to make this process easier for you.

Let’s get started.

• 1. What is an Article Review?
• 2. Types of Article Reviews
• 3. Article Review Format
• 4. How to Write an Article Review? 10 Easy Steps
• 5. Article Review Outline
• 6. Article Review Examples
• 7. Tips for Writing an Effective Article Review

What is an Article Review?

An article review is a critical evaluation and analysis of a piece of writing, typically an academic or journalistic article. 

It goes beyond summarizing the content; it involves an in-depth examination of the author's ideas, arguments, and methodologies. 

The goal is to provide a well-rounded understanding of the article's strengths, weaknesses, and overall contribution to the field.

Tough Essay Due? Hire Tough Writers!

Types of Article Reviews

Article reviews come in various forms, each serving a distinct purpose in the realm of academic or professional discourse. Understanding these types is crucial for tailoring your approach. 

Here are some common types of article reviews:

Journal Article Review

A journal article review involves a thorough evaluation of scholarly articles published in academic journals. 

It requires summarizing the article's key points, methodology, and findings, emphasizing its contributions to the academic field. 

Take a look at the following example to help you understand better.

Example of Journal Article Review

Research Article Review

A research article review focuses on scrutinizing articles with a primary emphasis on research.

This type of review involves evaluating the research design, methodology, results, and their broader implications. 

Discussions on the interpretation of results, limitations, and the article's overall contributions are key. 

Here is a sample for you to get an idea.

Example of Research Article Review

Science Article Review

A science article review specifically addresses articles within scientific disciplines. It includes summarizing scientific concepts, hypotheses, and experimental methods.

The type of review assesses the reliability of the experimental design, and evaluates the author's interpretation of findings. 

Take a look at the following example.

Example of Science Article Review

Critical Review

A critical review involves a balanced critique of a given article. It encompasses providing a comprehensive summary, highlighting key points, and engaging in a critical analysis of strengths and weaknesses. 

To get a clearer idea of a critical review, take a look at this example.

Critical Review Example

Article Review Format

When crafting an article review in either APA or MLA format, it's crucial to adhere to the specific guidelines for citing sources. 

Below are the bibliographical entries for different types of sources in both APA and MLA styles:

How to Write an Article Review? 10 Easy Steps

Writing an effective article review involves a systematic approach. Follow this step-by-step process to ensure a comprehensive and well-structured analysis.

Step 1: Understand the Assignment

Before diving into the review, carefully read and understand the assignment guidelines. 

Pay attention to specific requirements, such as word count, formatting style (APA, MLA), and the aspects your instructor wants you to focus on.

Step 2: Read the Article Thoroughly

Begin by thoroughly reading the article. Take notes on key points, arguments, and evidence presented by the author. 

Understand the author's main thesis and the context in which the article was written.

Step 3: Create a Summary

Summarize the main points of the article. Highlight the author's key arguments and findings. 

While writing the summary ensure that you capture the essential elements of the article to provide context for your analysis.

Step 4: Identify the Author's Thesis

In this step, pinpoint the author's main thesis or central argument. Understand the purpose of the article and how the author supports their position. 

This will serve as a foundation for your critique.

Step 5: Evaluate the Author's Evidence and Methodology

Examine the evidence provided by the author to support their thesis. Assess the reliability and validity of the methodology used. 

Consider the sources, data collection methods, and any potential biases.

Step 6: Analyze the Author's Writing Style

Evaluate the author's writing style and how effectively they communicate their ideas. 

Consider the clarity of the language, the organization of the content, and the overall persuasiveness of the article.

Step 7: Consider the Article's Contribution

Reflect on the article's contribution to its field of study. Analyze how it fits into the existing literature, its significance, and any potential implications for future research or applications.

Step 8: Write the Introduction

Craft an introduction that includes the article's title, author, publication date, and a brief overview. 

State the purpose of your review and your thesis—the main point you'll be analyzing in your review.

Step 9: Develop the Body of the Review

Organize your review by addressing specific aspects such as the author's thesis, methodology, writing style, and the article's contribution. 

Use clear paragraphs to structure your analysis logically.

Step 10: Conclude with a Summary and Evaluation

Summarize your main points and restate your overall assessment of the article. 

Offer insights into its strengths and weaknesses, and conclude with any recommendations for improvement or suggestions for further research.

Paper Due? Why Suffer? That's our Job!

Article Review Outline

Creating a well-organized outline is an essential part of writing a coherent and insightful article review.

This outline given below will guide you through the key sections of your review, ensuring that your analysis is comprehensive and logically structured.

Refer to the following template to understand outlining the article review in detail.

Article Review Format Template

Article Review Examples

Examining article review examples can provide valuable insights into the structure, tone, and depth of analysis expected. 

Below are sample article reviews, each illustrating a different approach and focus.

Example of Article Review

Sample of article review assignment pdf

Tips for Writing an Effective Article Review

Crafting an effective article review involves a combination of critical analysis, clarity, and structure. 

Here are some valuable tips to guide you through the process:

• Start with a Clear Introduction

Kick off your article review by introducing the article's main points and mentioning the publication date, which you can find on the re-title page. Outline the topics you'll cover in your review.

• Concise Summary with Unanswered Questions

Provide a short summary of the article, emphasizing its main ideas. Highlight any lingering questions, known as "unanswered questions," that the article may have triggered. Use a basic article review template to help structure your thoughts.

• Illustrate with Examples

Use examples from the article to illustrate your points. If there are tables or figures in the article, discuss them to make your review more concrete and easily understandable.

• Organize Clearly with a Summary Section

Keep your review straightforward and well-organized. Begin with the start of the article, express your thoughts on what you liked or didn't like, and conclude with a summary section. This follows a basic plan for clarity.

• Constructive Criticism

When providing criticism, be constructive. If there are elements you don't understand, frame them as "unanswered questions." This approach shows engagement and curiosity.

• Smoothly Connect Your Ideas

Ensure your thoughts flow naturally throughout your review. Use simple words and sentences. If you have questions about the article, let them guide your review organically.

• Revise and Check for Clarity

Before finishing, go through your review. Correct any mistakes and ensure it sounds clear. Check if you followed your plan, used simple words, and incorporated the keywords effectively. This makes your review better and more accessible for others.

In conclusion , writing an effective article review involves a thoughtful balance of summarizing key points, and addressing unanswered questions. 

By following a simple and structured approach, you can create a review that not only analyzes the content but also adds value to the reader's understanding.

Remember to organize your thoughts logically, use clear language, and provide examples from the article to support your points. 

Ready to elevate your article reviewing skills? Explore the valuable resources and expert assistance at MyPerfectWords.com. 

Our team of experienced writers is here to help you with article reviews and other school tasks. 

So why wait? Place your " write my essays online " request today!

Write Essay Within 60 Seconds!

Dr. Barbara is a highly experienced writer and author who holds a Ph.D. degree in public health from an Ivy League school. She has worked in the medical field for many years, conducting extensive research on various health topics. Her writing has been featured in several top-tier publications.

Paper Due? Why Suffer? That’s our Job!

Keep reading

Page Content

Overview of the review report format, the first read-through, first read considerations, spotting potential major flaws, concluding the first reading, rejection after the first reading, before starting the second read-through, doing the second read-through, the second read-through: section by section guidance, how to structure your report, on presentation and style, criticisms & confidential comments to editors, the recommendation, when recommending rejection, additional resources, step by step guide to reviewing a manuscript.

When you receive an invitation to peer review, you should be sent a copy of the paper's abstract to help you decide whether you wish to do the review. Try to respond to invitations promptly - it will prevent delays. It is also important at this stage to declare any potential Conflict of Interest.

The structure of the review report varies between journals. Some follow an informal structure, while others have a more formal approach.

" Number your comments!!! " (Jonathon Halbesleben, former Editor of Journal of Occupational and Organizational Psychology)

Informal Structure

Many journals don't provide criteria for reviews beyond asking for your 'analysis of merits'. In this case, you may wish to familiarize yourself with examples of other reviews done for the journal, which the editor should be able to provide or, as you gain experience, rely on your own evolving style.

Formal Structure

Other journals require a more formal approach. Sometimes they will ask you to address specific questions in your review via a questionnaire. Or they might want you to rate the manuscript on various attributes using a scorecard. Often you can't see these until you log in to submit your review. So when you agree to the work, it's worth checking for any journal-specific guidelines and requirements. If there are formal guidelines, let them direct the structure of your review.

In Both Cases

Whether specifically required by the reporting format or not, you should expect to compile comments to authors and possibly confidential ones to editors only.

Following the invitation to review, when you'll have received the article abstract, you should already understand the aims, key data and conclusions of the manuscript. If you don't, make a note now that you need to feedback on how to improve those sections.

The first read-through is a skim-read. It will help you form an initial impression of the paper and get a sense of whether your eventual recommendation will be to accept or reject the paper.

Keep a pen and paper handy when skim-reading.

Try to bear in mind the following questions - they'll help you form your overall impression:

• What is the main question addressed by the research? Is it relevant and interesting?
• How original is the topic? What does it add to the subject area compared with other published material?
• Is the paper well written? Is the text clear and easy to read?
• Are the conclusions consistent with the evidence and arguments presented? Do they address the main question posed?
• If the author is disagreeing significantly with the current academic consensus, do they have a substantial case? If not, what would be required to make their case credible?
• If the paper includes tables or figures, what do they add to the paper? Do they aid understanding or are they superfluous?

While you should read the whole paper, making the right choice of what to read first can save time by flagging major problems early on.

Editors say, " Specific recommendations for remedying flaws are VERY welcome ."

Examples of possibly major flaws include:

• Drawing a conclusion that is contradicted by the author's own statistical or qualitative evidence
• The use of a discredited method
• Ignoring a process that is known to have a strong influence on the area under study

If experimental design features prominently in the paper, first check that the methodology is sound - if not, this is likely to be a major flaw.

You might examine:

• The sampling in analytical papers
• The sufficient use of control experiments
• The precision of process data
• The regularity of sampling in time-dependent studies
• The validity of questions, the use of a detailed methodology and the data analysis being done systematically (in qualitative research)
• That qualitative research extends beyond the author's opinions, with sufficient descriptive elements and appropriate quotes from interviews or focus groups

Major Flaws in Information

If methodology is less of an issue, it's often a good idea to look at the data tables, figures or images first. Especially in science research, it's all about the information gathered. If there are critical flaws in this, it's very likely the manuscript will need to be rejected. Such issues include:

• Insufficient data
• Unclear data tables
• Contradictory data that either are not self-consistent or disagree with the conclusions
• Confirmatory data that adds little, if anything, to current understanding - unless strong arguments for such repetition are made

If you find a major problem, note your reasoning and clear supporting evidence (including citations).

After the initial read and using your notes, including those of any major flaws you found, draft the first two paragraphs of your review - the first summarizing the research question addressed and the second the contribution of the work. If the journal has a prescribed reporting format, this draft will still help you compose your thoughts.

The First Paragraph

This should state the main question addressed by the research and summarize the goals, approaches, and conclusions of the paper. It should:

• Help the editor properly contextualize the research and add weight to your judgement
• Show the author what key messages are conveyed to the reader, so they can be sure they are achieving what they set out to do
• Focus on successful aspects of the paper so the author gets a sense of what they've done well

The Second Paragraph

This should provide a conceptual overview of the contribution of the research. So consider:

• Is the paper's premise interesting and important?
• Are the methods used appropriate?
• Do the data support the conclusions?

After drafting these two paragraphs, you should be in a position to decide whether this manuscript is seriously flawed and should be rejected (see the next section). Or whether it is publishable in principle and merits a detailed, careful read through.

Even if you are coming to the opinion that an article has serious flaws, make sure you read the whole paper. This is very important because you may find some really positive aspects that can be communicated to the author. This could help them with future submissions.

A full read-through will also make sure that any initial concerns are indeed correct and fair. After all, you need the context of the whole paper before deciding to reject. If you still intend to recommend rejection, see the section "When recommending rejection."

Once the paper has passed your first read and you've decided the article is publishable in principle, one purpose of the second, detailed read-through is to help prepare the manuscript for publication. You may still decide to recommend rejection following a second reading.

" Offer clear suggestions for how the authors can address the concerns raised. In other words, if you're going to raise a problem, provide a solution ." (Jonathon Halbesleben, Editor of Journal of Occupational and Organizational Psychology)

Preparation

To save time and simplify the review:

• Don't rely solely upon inserting comments on the manuscript document - make separate notes
• Try to group similar concerns or praise together
• If using a review program to note directly onto the manuscript, still try grouping the concerns and praise in separate notes - it helps later
• Note line numbers of text upon which your notes are based - this helps you find items again and also aids those reading your review

Now that you have completed your preparations, you're ready to spend an hour or so reading carefully through the manuscript.

As you're reading through the manuscript for a second time, you'll need to keep in mind the argument's construction, the clarity of the language and content.

With regard to the argument’s construction, you should identify:

• Any places where the meaning is unclear or ambiguous
• Any factual errors
• Any invalid arguments

You may also wish to consider:

• Does the title properly reflect the subject of the paper?
• Does the abstract provide an accessible summary of the paper?
• Do the keywords accurately reflect the content?
• Is the paper an appropriate length?
• Are the key messages short, accurate and clear?

Not every submission is well written. Part of your role is to make sure that the text’s meaning is clear.

Editors say, " If a manuscript has many English language and editing issues, please do not try and fix it. If it is too bad, note that in your review and it should be up to the authors to have the manuscript edited ."

If the article is difficult to understand, you should have rejected it already. However, if the language is poor but you understand the core message, see if you can suggest improvements to fix the problem:

• Are there certain aspects that could be communicated better, such as parts of the discussion?
• Should the authors consider resubmitting to the same journal after language improvements?
• Would you consider looking at the paper again once these issues are dealt with?

On Grammar and Punctuation

Your primary role is judging the research content. Don't spend time polishing grammar or spelling. Editors will make sure that the text is at a high standard before publication. However, if you spot grammatical errors that affect clarity of meaning, then it's important to highlight these. Expect to suggest such amendments - it's rare for a manuscript to pass review with no corrections.

A 2010 study of nursing journals found that 79% of recommendations by reviewers were influenced by grammar and writing style (Shattel, et al., 2010).

1. The Introduction

A well-written introduction:

• Sets out the argument
• Summarizes recent research related to the topic
• Highlights gaps in current understanding or conflicts in current knowledge
• Establishes the originality of the research aims by demonstrating the need for investigations in the topic area
• Gives a clear idea of the target readership, why the research was carried out and the novelty and topicality of the manuscript

Originality and Topicality

Originality and topicality can only be established in the light of recent authoritative research. For example, it's impossible to argue that there is a conflict in current understanding by referencing articles that are 10 years old.

Authors may make the case that a topic hasn't been investigated in several years and that new research is required. This point is only valid if researchers can point to recent developments in data gathering techniques or to research in indirectly related fields that suggest the topic needs revisiting. Clearly, authors can only do this by referencing recent literature. Obviously, where older research is seminal or where aspects of the methodology rely upon it, then it is perfectly appropriate for authors to cite some older papers.

Editors say, "Is the report providing new information; is it novel or just confirmatory of well-known outcomes ?"

It's common for the introduction to end by stating the research aims. By this point you should already have a good impression of them - if the explicit aims come as a surprise, then the introduction needs improvement.

2. Materials and Methods

Academic research should be replicable, repeatable and robust - and follow best practice.

Replicable Research

This makes sufficient use of:

• Control experiments
• Repeated analyses
• Repeated experiments

These are used to make sure observed trends are not due to chance and that the same experiment could be repeated by other researchers - and result in the same outcome. Statistical analyses will not be sound if methods are not replicable. Where research is not replicable, the paper should be recommended for rejection.

Repeatable Methods

These give enough detail so that other researchers are able to carry out the same research. For example, equipment used or sampling methods should all be described in detail so that others could follow the same steps. Where methods are not detailed enough, it's usual to ask for the methods section to be revised.

Robust Research

This has enough data points to make sure the data are reliable. If there are insufficient data, it might be appropriate to recommend revision. You should also consider whether there is any in-built bias not nullified by the control experiments.

Best Practice

During these checks you should keep in mind best practice:

• Standard guidelines were followed (e.g. the CONSORT Statement for reporting randomized trials)
• The health and safety of all participants in the study was not compromised
• Ethical standards were maintained

If the research fails to reach relevant best practice standards, it's usual to recommend rejection. What's more, you don't then need to read any further.

3. Results and Discussion

This section should tell a coherent story - What happened? What was discovered or confirmed?

Certain patterns of good reporting need to be followed by the author:

• They should start by describing in simple terms what the data show
• They should make reference to statistical analyses, such as significance or goodness of fit
• Once described, they should evaluate the trends observed and explain the significance of the results to wider understanding. This can only be done by referencing published research
• The outcome should be a critical analysis of the data collected

Discussion should always, at some point, gather all the information together into a single whole. Authors should describe and discuss the overall story formed. If there are gaps or inconsistencies in the story, they should address these and suggest ways future research might confirm the findings or take the research forward.

4. Conclusions

This section is usually no more than a few paragraphs and may be presented as part of the results and discussion, or in a separate section. The conclusions should reflect upon the aims - whether they were achieved or not - and, just like the aims, should not be surprising. If the conclusions are not evidence-based, it's appropriate to ask for them to be re-written.

5. Information Gathered: Images, Graphs and Data Tables

If you find yourself looking at a piece of information from which you cannot discern a story, then you should ask for improvements in presentation. This could be an issue with titles, labels, statistical notation or image quality.

Where information is clear, you should check that:

• The results seem plausible, in case there is an error in data gathering
• The trends you can see support the paper's discussion and conclusions
• There are sufficient data. For example, in studies carried out over time are there sufficient data points to support the trends described by the author?

You should also check whether images have been edited or manipulated to emphasize the story they tell. This may be appropriate but only if authors report on how the image has been edited (e.g. by highlighting certain parts of an image). Where you feel that an image has been edited or manipulated without explanation, you should highlight this in a confidential comment to the editor in your report.

6. List of References

You will need to check referencing for accuracy, adequacy and balance.

Where a cited article is central to the author's argument, you should check the accuracy and format of the reference - and bear in mind different subject areas may use citations differently. Otherwise, it's the editor’s role to exhaustively check the reference section for accuracy and format.

You should consider if the referencing is adequate:

• Are important parts of the argument poorly supported?
• Are there published studies that show similar or dissimilar trends that should be discussed?
• If a manuscript only uses half the citations typical in its field, this may be an indicator that referencing should be improved - but don't be guided solely by quantity
• References should be relevant, recent and readily retrievable

Check for a well-balanced list of references that is:

• Helpful to the reader
• Fair to competing authors
• Not over-reliant on self-citation
• Gives due recognition to the initial discoveries and related work that led to the work under assessment

You should be able to evaluate whether the article meets the criteria for balanced referencing without looking up every reference.

7. Plagiarism

By now you will have a deep understanding of the paper's content - and you may have some concerns about plagiarism.

Identified Concern

If you find - or already knew of - a very similar paper, this may be because the author overlooked it in their own literature search. Or it may be because it is very recent or published in a journal slightly outside their usual field.

You may feel you can advise the author how to emphasize the novel aspects of their own study, so as to better differentiate it from similar research. If so, you may ask the author to discuss their aims and results, or modify their conclusions, in light of the similar article. Of course, the research similarities may be so great that they render the work unoriginal and you have no choice but to recommend rejection.

"It's very helpful when a reviewer can point out recent similar publications on the same topic by other groups, or that the authors have already published some data elsewhere ." (Editor feedback)

Suspected Concern

If you suspect plagiarism, including self-plagiarism, but cannot recall or locate exactly what is being plagiarized, notify the editor of your suspicion and ask for guidance.

Most editors have access to software that can check for plagiarism.

Editors are not out to police every paper, but when plagiarism is discovered during peer review it can be properly addressed ahead of publication. If plagiarism is discovered only after publication, the consequences are worse for both authors and readers, because a retraction may be necessary.

For detailed guidelines see COPE's Ethical guidelines for reviewers and Wiley's Best Practice Guidelines on Publishing Ethics .

8. Search Engine Optimization (SEO)

After the detailed read-through, you will be in a position to advise whether the title, abstract and key words are optimized for search purposes. In order to be effective, good SEO terms will reflect the aims of the research.

A clear title and abstract will improve the paper's search engine rankings and will influence whether the user finds and then decides to navigate to the main article. The title should contain the relevant SEO terms early on. This has a major effect on the impact of a paper, since it helps it appear in search results. A poor abstract can then lose the reader's interest and undo the benefit of an effective title - whilst the paper's abstract may appear in search results, the potential reader may go no further.

So ask yourself, while the abstract may have seemed adequate during earlier checks, does it:

• Do justice to the manuscript in this context?
• Highlight important findings sufficiently?
• Present the most interesting data?

Editors say, " Does the Abstract highlight the important findings of the study ?"

If there is a formal report format, remember to follow it. This will often comprise a range of questions followed by comment sections. Try to answer all the questions. They are there because the editor felt that they are important. If you're following an informal report format you could structure your report in three sections: summary, major issues, minor issues.

• Give positive feedback first. Authors are more likely to read your review if you do so. But don't overdo it if you will be recommending rejection
• Briefly summarize what the paper is about and what the findings are
• Try to put the findings of the paper into the context of the existing literature and current knowledge
• Indicate the significance of the work and if it is novel or mainly confirmatory
• Indicate the work's strengths, its quality and completeness
• State any major flaws or weaknesses and note any special considerations. For example, if previously held theories are being overlooked

Major Issues

• Are there any major flaws? State what they are and what the severity of their impact is on the paper
• Has similar work already been published without the authors acknowledging this?
• Are the authors presenting findings that challenge current thinking? Is the evidence they present strong enough to prove their case? Have they cited all the relevant work that would contradict their thinking and addressed it appropriately?
• If major revisions are required, try to indicate clearly what they are
• Are there any major presentational problems? Are figures & tables, language and manuscript structure all clear enough for you to accurately assess the work?
• Are there any ethical issues? If you are unsure it may be better to disclose these in the confidential comments section

Minor Issues

• Are there places where meaning is ambiguous? How can this be corrected?
• Are the correct references cited? If not, which should be cited instead/also? Are citations excessive, limited, or biased?
• Are there any factual, numerical or unit errors? If so, what are they?
• Are all tables and figures appropriate, sufficient, and correctly labelled? If not, say which are not

Your review should ultimately help the author improve their article. So be polite, honest and clear. You should also try to be objective and constructive, not subjective and destructive.

You should also:

• Write clearly and so you can be understood by people whose first language is not English
• Avoid complex or unusual words, especially ones that would even confuse native speakers
• Number your points and refer to page and line numbers in the manuscript when making specific comments
• If you have been asked to only comment on specific parts or aspects of the manuscript, you should indicate clearly which these are
• Treat the author's work the way you would like your own to be treated

Most journals give reviewers the option to provide some confidential comments to editors. Often this is where editors will want reviewers to state their recommendation - see the next section - but otherwise this area is best reserved for communicating malpractice such as suspected plagiarism, fraud, unattributed work, unethical procedures, duplicate publication, bias or other conflicts of interest.

However, this doesn't give reviewers permission to 'backstab' the author. Authors can't see this feedback and are unable to give their side of the story unless the editor asks them to. So in the spirit of fairness, write comments to editors as though authors might read them too.

Reviewers should check the preferences of individual journals as to where they want review decisions to be stated. In particular, bear in mind that some journals will not want the recommendation included in any comments to authors, as this can cause editors difficulty later - see Section 11 for more advice about working with editors.

You will normally be asked to indicate your recommendation (e.g. accept, reject, revise and resubmit, etc.) from a fixed-choice list and then to enter your comments into a separate text box.

Recommending Acceptance

If you're recommending acceptance, give details outlining why, and if there are any areas that could be improved. Don't just give a short, cursory remark such as 'great, accept'. See Improving the Manuscript

Recommending Revision

Where improvements are needed, a recommendation for major or minor revision is typical. You may also choose to state whether you opt in or out of the post-revision review too. If recommending revision, state specific changes you feel need to be made. The author can then reply to each point in turn.

Some journals offer the option to recommend rejection with the possibility of resubmission – this is most relevant where substantial, major revision is necessary.

What can reviewers do to help? " Be clear in their comments to the author (or editor) which points are absolutely critical if the paper is given an opportunity for revisio n." (Jonathon Halbesleben, Editor of Journal of Occupational and Organizational Psychology)

Recommending Rejection

If recommending rejection or major revision, state this clearly in your review (and see the next section, 'When recommending rejection').

Where manuscripts have serious flaws you should not spend any time polishing the review you've drafted or give detailed advice on presentation.

Editors say, " If a reviewer suggests a rejection, but her/his comments are not detailed or helpful, it does not help the editor in making a decision ."

In your recommendations for the author, you should:

• Give constructive feedback describing ways that they could improve the research
• Keep the focus on the research and not the author. This is an extremely important part of your job as a reviewer
• Avoid making critical confidential comments to the editor while being polite and encouraging to the author - the latter may not understand why their manuscript has been rejected. Also, they won't get feedback on how to improve their research and it could trigger an appeal

Remember to give constructive criticism even if recommending rejection. This helps developing researchers improve their work and explains to the editor why you felt the manuscript should not be published.

" When the comments seem really positive, but the recommendation is rejection…it puts the editor in a tough position of having to reject a paper when the comments make it sound like a great paper ." (Jonathon Halbesleben, Editor of Journal of Occupational and Organizational Psychology)

Visit our Wiley Author Learning and Training Channel for expert advice on peer review.

Watch the video, Ethical considerations of Peer Review

How to Write an Article Review: Guide with Examples - Studybay

The Knowledge Nest - Community and Society

Welcome to The Knowledge Nest, your go-to resource for all things related to education, learning, and academic success in the community and society. In this insightful guide, we will delve into the art of writing a stellar article review.

1. Introduction to Article Reviews

Article reviews are essential in academic and professional settings as they allow individuals to evaluate and analyze research papers, scholarly articles, and other sources of information. By writing an article review, you not only enhance your critical thinking skills but also contribute to the knowledge base in your field of study.

When writing an article review, it is crucial to provide a comprehensive analysis and offer valuable insights. Here at The Knowledge Nest, we believe in empowering students and researchers to produce high-quality reviews that stand out from the rest.

2. Structure of an Article Review

To craft an exceptional article review, it is essential to follow a well-structured format. Here is a breakdown of the typical sections to include:

• Introduction: Begin your review by presenting the title, author, and publication details of the article you are reviewing. Provide a brief overview of the author's background and the relevance of the article to your field of study.
• Summary: Summarize the main points, arguments, and key findings of the article. Be concise yet comprehensive in your summary.
• Analysis: Analyze the strengths and weaknesses of the article. Evaluate the author's methodology, sources, and supporting evidence. Reflect on the article's relevance and contribution to the field.
• Implications: Discuss the implications of the article's findings and how they relate to existing literature or research in your domain. Highlight any unanswered questions and potential areas for future exploration.
• Conclusion: Sum up your overall assessment of the article. Reiterate its significance and suggest further avenues for research, if applicable.

3. Examples of Well-Written Article Reviews

Here at The Knowledge Nest, we understand that learning by example is an effective way to grasp complex concepts. Below, we have provided a couple of examples to illustrate what a well-written article review should look like:

Example 1: Title of the Article Review

In this example, we present a review of an article titled "Exploring the Impact of Climate Change on Biodiversity." The review delves into the author's research methodology, provides a detailed analysis of the findings, and offers insights into the implications of the study.

Example 2: Another Title of the Article Review

Here, we showcase a review of an article titled "The Role of Artificial Intelligence in Transforming Healthcare." The review highlights the strengths and limitations of the research, discusses the potential implications on the healthcare industry, and suggests avenues for further research in this burgeoning field.

4. Tips for Writing an Outstanding Article Review

To help you excel in writing article reviews, we have compiled a list of valuable tips:

• Thoroughly read and understand the article: Before you begin writing, carefully read through the article multiple times to ensure a comprehensive understanding of its content and arguments.
• Take meticulous notes: Make annotations and jot down key points and relevant quotes while reading the article. This will make it easier for you to structure your review effectively.
• Use a formal and academic tone: Since article reviews are scholarly pieces, maintain a professional tone throughout your writing. Avoid using slang or colloquial language.
• Support your claims with evidence: When offering critique or analysis, back up your statements with evidence from the article or other reputable sources. This demonstrates your credibility as a reviewer.
• Proofread and edit: Before submitting your review, thoroughly proofread it to eliminate any grammatical or spelling errors. Pay attention to the overall flow and coherence of your writing.

Writing an article review is an invaluable skill that can enhance your academic and professional growth. By following our comprehensive guide, you will be equipped with the knowledge and tools needed to produce outstanding article reviews that stand out in the competitive landscape.

Remember, continuous practice and refinement of your writing skills will ultimately lead to mastery. The Knowledge Nest is here to support you on your journey towards excellence. Start writing remarkable article reviews today!

Check out our blog at http://theknowledgenest.org/blog/article-review-example for more educational resources and insightful articles.

The Effects of Homeschooling: Essay Example and Writing Tips

Good Topics to Write About ✏️ Essay Themes

Come Up With an Amazing Case Study Template - Studybay

How Do You Write An Argumentative Essay For Middle School

Creative Research Proposal Topics

How to Write a Good Term Paper (Format, Structure, Outline)

Buy College Papers Online - High-Quality and Affordable

Pay For Case Study - Studybay

Derivative of Sec x: A Quick Math Tutorial

Pre-Calculus Homework Help - The Knowledge Nest

Systematic Review

• Systematic reviews
• Developing a search strategy
• Search techniques
• Systematically search databases
• Appraisal & synthesis

Writing the review

Publishing review articles.

• Systematic review tools

When writing up findings, systematic reviews commonly include an Introduction, Method, Results and Discussion/Conclusion. 

The following articles may help you structure and write a systematic review:

Bulter, A., Hall, H., & Copnell, B. (2016). A guide to writing a qualitative systematic review protocol to enhance evidence-based practice in nursing and health care.  World views on Evidence-based Nursing, 13 (3), 241-249. https://sigmapubs.onlinelibrary.wiley.com/doi/full/10.1111/wvn.12134  

Cronin, M. A., & George, E. (2020). The why and how of the integrative review. Organizational Research Methods, 1 . http://10.0.4.153/1094428120935507

Henderson, L. K., Craig, J. C., Willis, N. S., Tovey, D. & Webster, A. C. (2010). How to write a Cochrane systematic review. Nephrology,  15 (6), 617-624.  https://doi.org/10.1111/j.1440-1797.2010.01380.x

Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Annals of Internal Medicine, 151, 264–269. https://doi.org/10.7326/0003-4819-151-4-200908180-00135

Muka, T., Glisic, M., Milic, J., Verhoog, S., Bohlius, J., Franco, O. H., Milic, J., Bramer, W. & Chowdhury, R. (2020). A 24-step guide on how to design, conduct, and successfully publish a systematic review and meta-analysis in medical research.  European Journal of Epidemiology,   35 (1),   49–60. https://doi.org/10.1007/s10654-019-00576-5 

Pati, D. & Lorusso, L. N. (2018). How to write a systematic review of the literature. Health Environments Research and Design Journal, 11 (1), 15-30.   https://doi.org/10.1177/1937586717747384

Torraco, R. J. (2005). Writing integrative literature reviews: Guidelines and examples.  Human Resource Development Review, 4 (3), 356-367.  https://doi.org/10.1177/1534484305278283

Whittemore, R., & Knafl, K. (2005). The integrative review: Updated methodology. Journal of Advanced Nursing, 52 (5), 546-553. https://doi-org/10.1111/j.1365-2648.2005.03621.x

Find different types of reviews in theses and dissertations

Tuwhera aut research repository.

• Search Tuwhera for reviews in your field and note how the different types of review are written in theses and dissertations.

ProQuest Dissertations & Theses Global

• Find theses and dissertations around the world using the methods of systematic review in the  ProQuest dissertations & theses database.

Examples of review articles

Click the following link to find different types of systematic review articles.

Examples of review articles 

Systematic review

Falkner, E. M., & Hiebl, M. R.W. (2015). Risk management in SMEs: A systematic review of available evidence.  Journal of Risk Finance, 16 (2), 122-144.  https://doi.org./10.1108/JRF-06-2014-0079

Kubacki, K., Ronto, R., Lahtinen, V., Pang, B., & Rundle-Thiele, S. (2017). Social marketing interventions aiming to increase physical activity among adults: A systematic review. Health Education, 117 (1), 69–89. https://doi.org/10.1108/HE-02-2016-0008

Kubacki, K., & Szablewska, N. (2019). Social marketing targeting Indigenous peoples: A systematic review. Health Promotion International, 34 (1), 133–143. https://doi.org/10.1093/heapro/dax060

Leonidou, E., Christofi, M., Vrontis, D., & Thrassou, A. (2020). An integrative framework of stakeholder engagement for innovation management and entrepreneurship development. Journal of Business Research, 119 , 245–258. https://doi.org/10.1016/j.jbusres.2018.11.054

Siemieniako, D., Kubacki, K., & Mitrega, M. (2021). Inter-organisational relationships for social impact: A systematic literature review.  Journal of Business Research, 132,  453-469. https://doi.org/10.1016/j.jbusres.2021.04.026

Snyder, H. (2019). Literature review as a research methodology: An overview and guidelines.  Journal of Business Research, 104,  333-339. https://doi.org/10.1016/j.jbusres.2019.07.039

Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management, 14 , 207–222. https://doi.org/10.1111/1467-8551.00375

Engineering 

Abdelmegid, M. A., González, V. A., O’Sullivan, M., Walker, C. G., Poshdar, M., & Ying, F. (2020). The roles of conceptual modelling in improving construction simulation studies: A comprehensive review. Advanced Engineering Informatics, 46 , Article 101175. https://doi.org/10.1016/j.aei.2020.101175

Kelsen, B. A., Sumich, A., Kasabov, N., Liang, S. H. Y., & Wang, G. Y. (2020). What has social neuroscience learned from hyperscanning studies of spoken communication? A systematic review. Neuroscience & Biobehavioral Reviews . Advanced online publication.  https://doi.org/10.1016/j.neubiorev.2020.09.008

Health science

Alder, G., Signal, N., Olsen, S., & Taylor, D. (2019). A systematic review of paired associative Stimulation (PAS) to modulate lower limb corticomotor excitability: Implications for stimulation parameter selection and experimental design. Frontiers in Neuroscience, 13 , Article 895. https://doi.org/10.3389/fnins.2019.00895

Baig, M. M., GholamHosseini, H., Moqeem, A. A., Mirza, F. & Linden, M. (2017). A systematic review of wearable patient monitoring systems – Current challenges and opportunities for clinical adoption. Journal of Medical Systems, 41 , Article 115. https://doi.org/10.1007/s10916-017-0760-1

Graham, R., & Masters‐Awatere, B. (2020). Experiences of Māori of Aotearoa New Zealand’s public health system: a systematic review of two decades of published qualitative research. Australian & New Zealand Journal of Public Health, 44 (3), 193–200. http://10.0.4.87/1753-6405.12971

Jarden, R. J., Jones, V., McClunie-Trust, P., Winnington, R., Merrick, E., Shannon, K., Turner, R., Donaldson, A. E., & Macdiarmid, R. (2021). Exploring the experiences and perceptions of students in a graduate entry nursing programme: A qualitative meta-synthesis. Nurse Education Today, 107 . Article 105121.  https://doi.org/10.1016/j.nedt.2021.10512 1

McGeown, J. P., Hume, P., Theadom, A., Quarrie, K. L. & Borotkanics, R. (2020). Nutritional interventions to improve neurophysiological impariments following traumatic brain injury: A systematic review. Journal of  Neuroscience Research, 99 (2), 573-603.  https://doi-org/10.1002/jnr.24746  

Montayre, J., Neville, S., Dunn, I., Shrestha-Ranjit, J., & Wright-St. Clair, V. (2020). What makes community-based physical activity programs for culturally and linguistically diverse older adults effective? A systematic review. Australasian Journal on Ageing, 39 (4), 331–340. https://doi.org/10.1111/ajag.12815

Scoping reviews

Burga, R., & Rezania, D. (2015). A scoping review of accountability in social entrepreneurship.  SAGE Open , 1-10. https://doi.org/10.1177/2158244015614606

Heavey, E., Baxter, K., & Birks, Y. (2019). Financial advice for funding later life care: A scoping review of evidence from England. Journal of Long-Term Care , 51-65. http://eprints.lse.ac.uk/91497/1/Heavey_Financial-advice.pdf

Hickmott, D., Prieto-Rodriguez, E., & Holmes, K. A. (2018). Scoping review of studies on computational thinking in K–12 mathematics classrooms.  Digital Experiences in Mathematics Education , 4, 48-69. https://doi.org/10.1007/s40751-017-0038-8

O'Flaherty, J., & Phillips, C. (2015). The use of flipped classrooms in higher education: A scoping review. The Internet and Higher Education , 25, 85-95. https://doi.org/10.1016/j.iheduc.2015.02.002

Engineering

Engebø, A., Lædre, O., Young, B., Larssen, P. F., Lohne, J., & Klakegg, O. J. (2020). Collaborative project delivery methods: A scoping review.  Journal of Civil Engineering and Management, 26 (3), 278-303. https://doi.org/10.3846/jcem.2020.12186

Reiman, A., Kaivo-oja, J., Parviainen, E., Takala, E., & Lauraeus, T. (2021). Human factors and ergonomics in manufacturing in the industry 4.0 context – A scoping review. T echnology in Society, 65 , 1-9. https://doi.org/10.1016/j.techsoc.2021.101572

Jesus, T.S., Bright, F., Kayes, N., et al. (2016). Person-centred rehabilitation: What exactly does it mean? Protocol for a scoping review with thematic analysis towards framing the concept and practice of person-centred rehabilitation. British Medical Journal Open 6 (7). https://doi.org/10.1136/bmjopen-2016-011959

LeClair, J., Watts, T. & Zahner, S. (2021). Nursing strategies for environmental justice: A scoping review. Public Health Nursing, 38 (2), 296-308.   https://doi.org/10.1111/phn.12840

Macdiarmid, R., Turner, R., Winnington, R. et al. (2021). What motivates people to commence a graduate entry nursing programme: a mixed method scoping review.  BMC Nursing, 20 (47). https://doi.org/10.1186/s12912-021-00564-9

Madanian, S., Norris, T., & Parry, D. (2020). Disaster eHealth: Scoping review. Journal of Medical Internet Research, 22 (10), Article e18310.  https://doi.org/ 10.2196/18310

Thompson, D.S., Fazio, X., Kustra, E., Patrick, L., & Stanley, D. (2016). Scoping review of complexity theory in health services research.  BMC Health Services Research, 16 , Article number 87.  https://doi.org/10.1186/s12913-016-1343-4

Rapid reviews

Jahangirian, M., Eldabi, T., Garg, L., Jun, G. T., Naseer, A., Patel, B., Stergioulas, L., & Young, T. (2011). A rapid review method for extremely large corpora of literature: Applications to the domains of modelling, simulation, and management. International Journal of Information Management, 31 (3), 234–243. http://10.0.3.248/j.ijinfomgt.2010.07.004

Brooks, S. K., Webster, R. K., Smith, L. E., Woodland, L., Wessely, S., Greenberg, N., & Rubin, G. J. (2020). The psychological impact of quarantine and how to reduce it: Rapid review of the evidence. Lancet, 395 (10227), 912–920. http://10.0.3.248/S0140-6736(20)30460-8

Evans, C. J., Ison, L., Ellis-Smith, C., Nicholson, C., Costa, A., Oluyase, A. O., Namisango, E. V. E., Bone, A. E., Brighton, L. J., Y, D., Combes, S., Bajwah, S., Cao, W. E. I., Harding, R., Ong, P., Higginson, I. J., & Maddocks, M. (2019). Service delivery models to maximize quality of life for older people at the end of life: A rapid review. Milbank Quarterly, 97 (1), 113–175. http://10.0.4.87/1468-0009.12373

Furlong, L., Serry, T., Bridgman, K., & Erickson, S. (2021). An evidence‐based synthesis of instructional reading and spelling procedures using telepractice: A rapid review in the context of COVID‐19. International Journal of Language & Communication Disorders, 56 (3), 456–472. http://10.0.4.87/1460-6984.12619

Meis-Harris, J., Klemm, C., Kaufman, S., Curtis, J., Borg, K., & Bragge, P. (2021). What is the role of eco-labels for a circular economy? A rapid review of the literature. Journal of Cleaner Production, 306 , Article 127134. http://10.0.3.248/j.jclepro.2021.127134

Integrative reviews

Sajjad, A., & Shahbaz, W. (2020). Mindfulness and social sustainability: An Integrative Review. Social Indicators Research, 150 (2), 73–94. http://10.0.3.239/s11205-020-02297-9

Yap, S.-F., & Kapitan, S. (2017). Consumption coping and life transitions: An integrative review. Australasian Marketing Journal, 25 (3), 194–205. https://doi.org/10.1016/j.ausmj.2017.04.003

Almeida, S., & Montayre, J. (2019). An integrative review of nurse-led virtual clinics: He arotakenga hanumi i ngā tari hauora mariko nā te tapuhi i ārahi. Nursing Praxis in Aotearoa New Zealand, 35 (1), 18–28. https://doi.org/10.36951/NgPxNZ.2019.003

Classen, B., Tudor, K., Johnson, F., & McKenna, B. (2021). Embedding lived experience expertise across the mental health tertiary education sector: An integrative review in the context of Aotearoa New Zealand. Journal of Psychiatric and Mental Health Nursing . Advanced online publication.  https://doi.org/10.1111/jpm.12756

Gautam, S., Neville, S., & Montayre, J. (2019). What is known about the spirituality in older adults living in residential care facilities? An Integrative review. International Journal of Older People Nursing, 14 (2). https://doi.org/10.1111/opn.12228

Kanengoni, B., Andajani-Sutjahjo, S., & Holroyd, E. (2018). Setting the stage: Reviewing current knowledge on the health of New Zealand immigrants - an integrative review. PeerJ, 6, Article e5184.  https://doi.org/10.7717/peerj.5184

Neville, S., Wright-St Clair, V., Montayre, J., Adams, J., & Larmer, P. (2018). Promoting Age-Friendly Communities: an Integrative Review of Inclusion for Older Immigrants. Journal of Cross-Cultural Gerontology, 33 (4), 427–440. https://doi.org/10.1007/s10823-018-9359-3

It is worthwhile to spend time on evaluating and selecting a good journal for publishing. We suggest you:

• Read our Get Published guide
• Consult your colleagues and supervisors 
• Check whether a journal publishes review articles. For example, the following image shows the proportion of articles published in a journal (indexed by Scopus). Most publications in this journal are research articles. Only 1.9% of the publications are review articles.

• Get help from Research Services librarians
• << Previous: Appraisal & synthesis
• Next: Systematic review tools >>
• Last Updated: May 29, 2024 9:20 AM
• URL: https://aut.ac.nz.libguides.com/systematic_reviews

How to write a review article?

Affiliation.

• 1 Department of Urology, Faculty of Medicine İbni Sina Hospital, Ankara University, Ankara, Turkey.
• PMID: 26328136
• PMCID: PMC4548566
• DOI: 10.5152/tud.2013.054

In the medical sciences, the importance of review articles is rising. When clinicians want to update their knowledge and generate guidelines about a topic, they frequently use reviews as a starting point. The value of a review is associated with what has been done, what has been found and how these findings are presented. Before asking 'how,' the question of 'why' is more important when starting to write a review. The main and fundamental purpose of writing a review is to create a readable synthesis of the best resources available in the literature for an important research question or a current area of research. Although the idea of writing a review is attractive, it is important to spend time identifying the important questions. Good review methods are critical because they provide an unbiased point of view for the reader regarding the current literature. There is a consensus that a review should be written in a systematic fashion, a notion that is usually followed. In a systematic review with a focused question, the research methods must be clearly described. A 'methodological filter' is the best method for identifying the best working style for a research question, and this method reduces the workload when surveying the literature. An essential part of the review process is differentiating good research from bad and leaning on the results of the better studies. The ideal way to synthesize studies is to perform a meta-analysis. In conclusion, when writing a review, it is best to clearly focus on fixed ideas, to use a procedural and critical approach to the literature and to express your findings in an attractive way.

Keywords: How to write; review; writing.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Noro Psikiyatr Ars
• v.59(1); 2022

Basics of Writing Review Articles

Almıla erol.

Adjunct Faculty, Psychiatry & Psychology, Mayo Clinic, Rochester, MN, USA

Evidence-based medicine forms the essence of medical practice in the modern world. No wonder review articles are the mainstay for evidence-based medicine.

Review articles provide a critical summary of the existing literature to explain the current state of scientific evidence on a particular topic. A well-written review article must summarize key research findings, reference must-read articles, describe current areas of agreement as well as controversies and debates, point out gaps in current knowledge, depict unanswered questions, and suggest directions for future research ( 1 ).

During the last decades, there has been a great expansion in the range of review methodologies resulting in many new review types ( 2 , 3 ). In an attempt to classify review types, Sutton et al. defined 48 different review types which they categorized into seven review families: traditional reviews, systematic reviews, review of reviews, rapid reviews, qualitative reviews, mixed method reviews and purpose specific reviews (for the full list of review types please see Sutton et al.) ( 2 ). To date, traditional reviews and systematic reviews have been most widely used in the field of medicine.

Traditional reviews usually cover advances in different aspects of a chosen topic and provide assessment of the subject within a broad spectrum. No formal guidance exists for traditional reviews. However, they have become increasingly more comprehensive and systematic since the emergence of systematic reviews. Narrative review, narrative summary, critical review, integrative review, and state of the art review are examples of traditional reviews ( 2 ).

Systematic reviews adopt a specific aim and a well-defined, rigorous methodology to enlighten a particular question. They usually focus on specific study types such as randomized controlled studies, observational studies, etc. They have well-defined reporting standards and guidance. Systematic reviews provide the highest level of evidence in medical sciences, playing an important role in the development of clinical guidelines ( 4 ). Meta-analysis is the most popular example of quantitative systematic review types.

• Review articles summarize the current state of evidence on a particular topic
• Review articles translate the relevance of evidence for readers
• Independent of the review type, all reviews must have a predefined methodology
• The methods utilized for the review should be explained clearly in the review paper
• Review papers should be written in a structured format

Considering the overwhelming number of diverse review types, the initial burden authors face is to choose the review type that matches their purpose best. Despite the continuous rise in the number of review types, there are sources that provide guidance about this issue ( 5 ). Authors are highly recommended to examine and learn about different review methodologies before they decide on their review approach.

International guidelines such as PRISMA ( 6 ), Cochrane ( 7 ), and JBI ( 8 ) provide detailed information about how to conduct reviews starting from the planning and protocol writing phases. The purpose of these international guidelines is to ensure transparent, unbiased, and complete reporting. Although the guidelines are focused on systematic reviews, they can also be used as bases for conducting other types of reviews. PRISMA encourages journal editors and reviewers to use the guideline for evaluation of review papers. PRISMA checklist is available online in different languages including Turkish at www.prisma-statement.org ( 9 ).

No matter what type of review is undertaken, the key points in a review article are to have a predefined methodology which is clearly explained in the text, and to have a structured format. Just like research papers, the most common and convenient practice is to write review papers in “introduction, methods, results, and discussion (IMRaD)” format accompanied by title, abstract, key words, and references.

The title makes the first introductory and is the most important sentence of the review paper. Like research paper titles, it must be brief, informative, and interesting all at the same time. It must contain the key words or their derivatives to increase the discoverability of the article via search engines. In addition, the type of the review should be accurately stated in the title.

The aim of the introduction is to explain why the review is undertaken and to persuade the readers for its necessity. In the introduction section, the authors must mention the latest developments about the subject of concern and explain why a review is needed. It is a good practice to refer to previous review papers on the subject and state what makes the current review different than the previous ones.

The methods section of the review paper should be written detailed enough to prove its adequacy and make it possible to be reconducted including more recent papers in the future. Explicit scientific methods are required for systematic reviews as defined by international guidelines ( 7 – 9 ). Although no guidelines exist for traditional narrative reviews, they too should have a rational methodology explained clearly. The methods section of every review article should state the key words used for the search, data bases screened, and the time frame chosen for the literature search. It should also explain the inclusion and exclusion criteria used for the selection of papers.

The results section should include a flow chart which shows the number of identified, included, and excluded papers along with the reasons for exclusion, as described in PRISMA flow diagram guidelines ( 9 ). Results section should cite and present characteristics and outcomes of each one of the included studies, providing the necessary information to assess their quality, validity, and contribution. The most relevant information about the included articles should be depicted in literature summary tables. They are an essential part of the review article as they provide information at one glance and make the paper more readable. Literature summary tables must contain information about methods, frameworks, strengths, limitations, and conceptual contribution of each article ( 10 ). Oversized tables must be presented as supplementary files.

Discussion section provides a general interpretation of the results and presents expert opinion. Writing a review article is not only about extracting relevant previous work and analyzing them, but also about making synthesis and drawing conclusions. Therefore, providing an objective interpretation of the results and guiding readers for better understanding of the current evidence should form the central part of the discussion. Wherever there is not enough evidence to make objective conclusions, the lack of evidence should be stated instead. Limitations, biases and gaps of the included literature should be discussed along with the limitations of the review process itself. It is critical to discuss the potential impacts of the results for future research and clinical practice.

In conclusion, reviews are objective attempts to examine the current state of evidence on a particular topic and its impacts. A review paper should explain why the review is undertaken, describe the methodology used, introduce the articles included, and provide expert opinion on the evidence achieved in a structured format. High quality reviews are essential in guiding clinical practice and future research along with policy making.

• Open access
• Published: 03 June 2024

Evaluating competency-based medical education: a systematized review of current practices

• Nouf Sulaiman Alharbi 1 , 2 , 3  

BMC Medical Education volume  24 , Article number:  612 ( 2024 ) Cite this article

46 Accesses

2 Altmetric

Metrics details

Few published articles provide a comprehensive overview of the available evidence on the topic of evaluating competency-based medical education (CBME) curricula. The purpose of this review is therefore to synthesize the available evidence on the evaluation practices for competency-based curricula employed in schools and programs for undergraduate and postgraduate health professionals.

This systematized review was conducted following the systematic reviews approach with minor modifications to synthesize the findings of published studies that examined the evaluation of CBME undergraduate and postgraduate programs for health professionals.

Thirty-eight articles met the inclusion criteria and reported evaluation practices in CBME curricula from various countries and regions worldwide, such as Canada, China, Turkey, and West Africa. 57% of the evaluated programs were at the postgraduate level, and 71% were in the field of medicine. The results revealed variation in reporting evaluation practices, with numerous studies failing to clarify evaluations’ objectives, approaches, tools, and standards as well as how evaluations were reported and communicated. It was noted that questionnaires were the primary tool employed for evaluating programs, often combined with interviews or focus groups. Furthermore, the utilized evaluation standards considered the well-known competencies framework, specialized association guidelines, and accreditation criteria.

This review calls attention to the importance of ensuring that reports of evaluation experiences include certain essential elements of evaluation to better inform theory and practice.

Peer Review reports

Medical education worldwide is embracing the move toward outcome-based education (OBME) [ 1 , 2 ]. One of the most popular outcome-based approaches being widely adopted by medical schools worldwide is competency-based medical education (CBME) [ 3 ]. CBME considers competencies as the ultimate outcomes that should guide curriculum development at all steps or stages—that is, implementation, assessment, and evaluation [ 3 , 4 , 5 ]. To embrace CBME and prepare medical students for practice, medical educators usually utilize an organized national or international competency framework that describes the abilities that physicians must possess to meet the needs of patients and society. There are numerous global competency frameworks that reflect the characteristics of a competent doctor, for example, CanMEDS, Scottish Doctor, Medical School Projects, ACGME Outcome Project, the Netherlands National Framework, and Saudi Meds [ 1 , 6 , 7 , 8 ].

With the worldwide implementation of CBME and availability of different competency frameworks, educators are expected to evaluate various modifications made to existing medical curricula [ 9 , 10 ]. Such evaluation is intended to explore whether the program is operating as planned and its outcomes are achieved as intended in comparison to predetermined standards as well as to ensure improvement [ 11 , 12 , 13 ]. Furthermore, program evaluation revolves around two main concepts, that is, merit and worth [ 12 , 14 ]. In 1981, Guba and Lincoln explained that the merits of a program are intrinsic, implicit, and independent and do not refer to a specific context or application, while evaluating a program’s worth entails judging the value of any aspect of it in reference to a certain context or precise application [ 12 , 14 ].

To enable educators to determine the merits and worth of an educational program or curriculum, evaluation experts have proposed several models [ 14 , 15 ]. Evaluation models are guiding frameworks that demonstrate what appropriate evaluation looks like and detail how it should be designed and implemented [ 16 ]. Although almost all evaluation models focus on exploring whether a program attains its objectives, they vary in numerous aspects, including their evaluation philosophy, approaches, and the specific areas that they encompass [ 17 ].

It is essential that educators choose a suitable evaluation model when they implement CBME, as the right model will enable them to pinpoint [ 15 , 18 , 19 ]. In other words, a program helps identify areas of success, challenges, and opportunities for improvement in CBME implementation, leading to a deeper comprehension of CBME strategies and their effectiveness. Moreover, implementing CBME demands significant efforts and a wide range of financial, human, time, and infrastructure resources [ 20 ]. Thus, ensuring that these efforts and resources are well utilized to enhance educational and healthcare outcomes is crucial. In addition, evaluation provides valuable evidence for accreditation, quality assurance, policies, and guidelines. Otherwise put, it supports informed decision-making on many levels [ 21 ]. On another front, sharing evaluation results and being transparent about evaluation processes can enhance public trust in available programs, colleges, and universities [ 19 ]. However, deciding which evaluation model to adopt can be challenging [ 9 ].

Not only can it be difficult to select an appropriate model to evaluate a CBME program, but CBME evaluation itself has numerous challenges, particularly given the lack of a common definition or standardized description of what constitutes a CBME program [ 9 , 22 , 23 ]. The complexity of CBME further tangles evaluation efforts, given the multilayered nature of CBME’s activities and outcomes and the need to engage a wide variety of stakeholders [ 11 ]. Moreover, the scarcity and variable quality of reporting in studies focusing on the evaluation of CBME curricula exacerbate these challenges [ 24 ]. Furthermore, few published articles provide a comprehensive overview of available evidence on the topic.

This review is therefore designed to synthesize the findings of published studies that have reported CBME evaluation practices in undergraduate and postgraduate medical schools and programs. Its objective is to explore which CBME program evaluation practices have been reported in the literature by inspecting which evaluation objectives, models, tools, and standards were described in the included studies. In addition, the review inspects the results of evaluations and how they were shared. Thus, the review will serve in supporting educators to make evidence-based decisions when designing a CBME program. In addition, it will provide a useful resource for educators to embrace what was done right, learn from what was done wrong, improve many current evaluation practices, and compare different CBME interventions across various contexts.

Following a preliminary search within relevant journals for publications addressing evaluation practices utilized to assess competency-based curricula in medical education, the researcher used the PEO (participant, educational aspect, and outcomes) model to set and formulate the search question [ 24 ] as follows: participants : healthcare professionals and healthcare profession students; educational aspect : CBME curricula; outcome : program evaluation practices.

Next, the researcher created a clear plan for the review protocol. This review is classified as a systematized review rather than a systematic review [ 25 ]. While it does not meet the criteria for a systematic review because it relies on a single researcher and does not evaluate the quality of the studies included, it adheres to most of the steps outlined in the “Systematic Reviews in Medical Education: A Practical Approach: AMEE guide 94” [ 26 ]. Moreover, the researcher met with a medical educator with a strong background in CBME, an expert in review methods, and a librarian who is an expert in available databases and provided guidance and support for navigating such databases. Feedback was obtained from all three and used to finalize the review protocol. The protocol was followed to ensure that the research progressed in a consistent and systematized manner.

For this review, full-text articles published in peer-reviewed journals in English from 1 January 2000 to 31 December 2022 were searched within the following electronic data bases: PubMed, ERIC, Education Source, and CINHAL. The following terms were utilized to conduct the search: (Competency Based Medical Education OR Outcome Based Medical Education) AND (Evaluation OR assessment) AND (Undergraduate OR Postgraduate) AND (Implementing OR Performance OR Framework OR Program* OR Project OR Curriculum OR Outcome) (Additional file 1 ).

The researcher included articles that were published in English and reported evaluation practices for CBME or OBME curricula whether for undergraduate or postgraduate healthcare professionals. The researcher did not consider research reviews, commentaries, perspective articles, conference proceedings, and graduate theses in this review. In addition, articles that addressed students’ assessments rather than program evaluation were not included. Furthermore, articles that focused on teaching a particular skill (e.g., communication skills) or specific educational strategies (e.g., the effectiveness of Problem Based Learning) were excluded from this review.

To facilitate the screening of articles and ensure the process was properly documented, an online review software that streamlines the production of reviews (Covidence) was utilized, and all the lists of articles retrieved from the specified databases were uploaded to the tool website (available at www.covidence.org ). The tool set the screening to start with the titles and abstracts then to proceed to full texts. During these stages, the reasons for excluding an article were precisely noted. Moreover, the PRISMA diagram (available at http://www.prisma-statement.org/ ) was produced by Covidence to illustrate the process of screening and including articles in this review.

After the decision was made to include an article, a data extraction tool created for the purpose of this review was used (Additional file 2 ). Since the term “program evaluation practices” is general and does not clearly define the method or focus of the analysis involved in critiquing evaluation efforts, the analysis of available evaluation practices in this review was based on the Embedded Evaluation Model (EEM) provided by Giancola (2020) for educators to consider when embedding evaluations into educational program designs and development [ 27 ]. The EEM outlines several steps. In the first step, “Define,” educators are expected to build an understanding of the evaluated program, including its logic and context. In the second step, “Plan,” educators must establish the evaluation-specific objectives and questions and select the model or approach along with the methods or tools that will be utilized to achieve those objectives. The next step, “Implement and Analyze,” requires educators to determine how the data will be collected, analyzed, and managed. In the fourth step, “Interpret the Results,” educators are expected to derive insights from the results in terms of how the evaluation can help with resolving issues and improving the program as well as how the results should be communicated and employed. Finally, in the “Inform and Refine” step, educators should focus on applying the results to realize improvements to the program and promote accountability [ 27 ].

In addition to supporting the aim of the current review, the theoretical insights from Giancola (2020) help to ensure alignment with best practices in curriculum evaluation. Thus, for each article, the extraction tool collected the following information: the author, the publication year, the country and name of the institution that implemented the CBME curriculum, the aim and method of the article, the type of curriculum based on the health profession specialty (e.g., medicine, nursing), the level of the curriculum (postgraduate or undergraduate), the evaluation objective, the approach/model or tool, the evaluation standard, the evaluation results, and the sharing of the evaluation results. The extracted information points are essential to contextualize the evaluation and allow educators to make sense of it and adapt or adjust it to their own situations. Understanding the context of an evaluation is important considering the wide variety of available educational environments, the diversity of evaluators, and the differences in goals, modes, and benchmarks for evaluation, all of which influence how an evaluation is framed and conducted [ 27 ].

The author, publication year , and name and country of the institution that implemented the CBME curriculum provide identifiers for the original article and enable educators to seek further information about a study. The aim and method of the article were highlighted because they clarify the general context in which the evaluation was conducted. For example, this information can help educators understand whether an evaluation was carried out as a single action in response to a certain problem or was a phase or part of a larger project. The type of curriculum based on the health profession specialty (e.g., medicine, nursing) along with the level of the curriculum (postgraduate or undergraduate) have specific implications related to the nature of each specialty and the level of the competencies associated with the advancement of the program. All of the previously mentioned information is vital for educators to define and understand the program they are aiming to evaluate, which is the first step in the EEM. The evaluation objective , approach/model or tool , evaluation standard , evaluation results , and sharing of the evaluation results help to answer the research question of the current review by dissecting various aspects of the evaluation activities. In addition, the reporting of these aspects provides valuable insight into evaluation directives, plans, and execution. For educators, the evaluation objective usually clarifies the focus of the evaluation (e.g., how the program was implemented, the action done to execute education or outcomes of the program, and its effectiveness). The approach/model or tool of an evaluation is a core element of the design and implementation of the evaluation, as it determines the theoretical guidelines that underlie the evaluation and the practical steps for its execution. Based on the evaluation standard, which refers to the target used to compare the evidence or results of the evaluation, educators can judge the relevance of the evaluation to their own practices or activities. This information aligns with steps two and three of the EEM. The evaluation results are the results of the evaluation, which form the cornerstone for emerging solutions or future improvements. Finally, sharing the evaluation results , or communicating the evaluation, is a key part of handling the results and working toward their application. This information is aligned with steps four and five of the EEM.

Search results

Searching the identified databases revealed a total of 640 articles, and 183 total duplicates were removed. A total of 457 articles was considered for screening (371 PubMed, 13 ERIC, 23 Education Source, 50 CINHAL) (Fig.  1 ). Of those articles, 87 were retrieved for full-text screening. Ultimately, 38 studies met the inclusion criteria and were considered eligible to be included in the current review.

Flowchart illustrating the process of including articles in the review

Findings of the included studies

The 38 studies that met the inclusion criteria were published between 2010 and 2021, and the majority (15%; n  = 6) were published in 2019. The studies represented the following countries: Canada (37%, n  = 14) [ 10 , 11 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 ], USA (27.5%, n  = 11) [ 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 ], Australia (5%, n  = 2) [ 51 , 52 ], China (5%, n  = 2) [ 53 , 54 ], Dutch Caribbean islands (2.5%, n  = 1) [ 55 ], Germany (2.5%, n  = 1) [ 56 ], Guatemala (2.5%, n  = 1) [ 57 ], Korea (2.5%, n  = 1) [ 58 ], the Netherlands (2.5%, n  = 1) [ 59 ], New Zealand (2.5%, n  = 1) [ 60 ], The Republic of Haiti (2.5%, n  = 1) [ 61 ], Turkey (2.5%, n  = 1) [ 62 ], and the region of West Africa (2.5%, n  = 1) [ 63 ].

According to the evidence synthesized from the included studies, most of the evaluation practices were reported in competency-based curricula that targeted the level of postgraduate professionals (57%, n  = 22) and were medical in nature (71%, n  = 27) (Fig.  2 ).

Curricula specialties in included articles

The findings showed that 37% ( n  = 14) of the articles did not report the precise objective of evaluating the curriculum. Moreover, 84% ( n  = 32) did not report the evaluation approach or model used to assess the described curricula. The approaches or models reported include Pawson’s model of realist program evaluation [ 37 ], theory-based evaluation approaches [ 10 ], Stufflebeam’s context, inputs, processes, and products (CIPP) model [ 62 ], the concerns-based adoption model, sensemaking and outcome harvesting [ 33 ]the CIPP model [ 48 ], and quality improvement (QI) for program and process improvement [ 50 ]. On the other hand, a wide variety of evaluation tools was reported including observations (3%, n  = 1) [ 28 ] surveys or questionnaires (58%, n  = 22) [ 10 , 28 , 29 , 31 , 34 , 35 , 36 , 38 , 39 , 41 , 42 , 45 , 49 , 50 , 51 , 52 , 53 , 55 , 56 , 58 , 59 , 63 ] interviews (16%, n  = 6) [ 10 , 28 , 37 , 41 , 47 , 62 ], focus groups (13%, n  = 5) [ 35 , 37 , 41 , 50 , 59 ], historical document review or analysis (8%, n  = 3) [ 10 , 29 , 33 ], educational activity assessment or analysis of the activity by separate reviewers (5%, n  = 2) [ 55 , 61 ], stakeholder discussions or reports about their inputs (5%, n  = 2) [ 43 , 44 ], curriculum mapping (3%, n  = 1) [ 32 ], feedback from external reviews from accrediting bodies (3%, n  = 1) [ 32 ], the Dundee Ready Education Environment Measure (DREEM) (3%, n  = 1) [ 56 ], and students’ or participants’ assessments (5%, n  = 2) [ 38 , 46 ].

Of the studies, 37% ( n  = 14) utilized multi methods [ 10 , 28 , 29 , 30 , 32 , 34 , 35 , 37 , 38 , 41 , 48 , 52 , 56 , 59 ]. Furthermore, 7.8% ( n  = 3) of the studies reported the nature of the tool, for example, quantitative or qualitative, without specifying the exact tool utilized [ 57 , 60 ]. Moreover, 63% ( n  = 24) of the studies included in this review did not report the evaluation standards applied while assessing the competency-based curricula addressed. Yet, those studies that reported their standards were stated in various ways as follows: some publications referred to the standards of specific specialized associations or societies, such as the American Academy of Family Physicians and College of Family Physicians Canada [ 61 ], Royal Australian and New Zealand College of Psychiatrists [ 60 ], and The American Association of Occupational Health Nurses [ 45 ]. Other publications utilized known competency frameworks as their standards, such as CanMED [ 36 , 37 , 59 ], or the competencies of the American Board of Surgery [ 43 ] Association of Canadian Faculties of Dentistry [ 31 ], Royal College of Ophthalmologists [ 52 ], The Florida Consortium for Geriatric Medical Education [ 50 ], or the Dutch Advisory Board for Postgraduate Curriculum Development for Medical Specialists [ 59 ]. Furthermore, many of the publications referred to accreditation standards, such as the Accreditation Standards of the Australian Medical Council [ 51 ], Competencies of Accreditation Council for Graduate Medical Education [ 43 ], Accreditation Body in the Competency-based Curriculum [ 32 ], and the Commission on Dental Accreditation of Canada [ 31 ]. All the publications included in the review reported the results of their evaluations.

Finally, the results revealed that almost half (52.6%, n  = 20) of the authors of the articles mentioned that they were publishing their experience with the intent of sharing lessons learned, yet they did not refer to any other means of sharing the results of their evaluations. In contrast, the other half did not mention any measures taken to communicate and share the evaluation results. Additional file 3 includes the characteristics and details of the data extracted from the studies included addressing evaluation practices in healthcare professionals’ education.

Evaluating a curriculum appropriately is important to ensure that the program is operating as intended [ 13 ]. The present study aimed to review the available literature on the evaluation practices of competency-based undergraduate and postgraduate health professionals’ schools and programs. This review inspected which evaluation objectives, models, tools, and standards were described as well as the results of evaluations and how the results were shared. The synthesized evidence indicates that most of the programs reporting evaluation practices were postgraduate-level medical programs. This focus on CMBE among postgraduate programs can be related to the fact that competency-based education is organized around the most critical competencies useful for health professionals after graduation. Thus, they are better judged at practice [ 64 , 65 , 66 ]. Moreover, although competency-based curricula were introduced to many health professions over 60 years ago, such as pharmacology and chiropractic therapy, within the medical field they have only evolved in the last decade [ 67 ].

Furthermore, the data revealed that there is a discrepancy in how evaluation practices were reported in the literature in terms of evaluation objectives, approaches/models, tools, standards, documenting of results, and communication plans. Each area will be further discussed in the following paragraphs considering the ten-task approach and embedded evaluation model [ 27 , 68 ]. Both guide evaluation as an important step in curriculum development in medical education, detail the evaluation process, and outline many important considerations from design to execution [ 27 , 68 ].

Evaluation is a crucial part of curriculum development, and it can serve many purposes, such as ensuring attaining educational objectives, identifying areas of improvement, improving decision-making, and assuring quality [ 13 , 27 ]. Consequently, when addressing evaluation, it is important for educators to start by explaining the logic of the curriculum by asking, for example, what the program’s outcomes are and whether it is designed for postgraduates or undergraduates [ 27 ]. Moreover, educators must be precise in setting evaluation objectives, which entails answering certain questions: who will use the evaluation data; how will the data be used at both the individual and program level; will the evaluation be summative or formative; and what evaluation questions must be answered [ 27 , 68 , 69 ] However, many of the studies included in this review did not clearly explain the context of the curricula or report the objectives of their evaluation endeavors; rather, they settled for clarifying the objectives of the study or of the publication itself. One reason for this is that evaluation and educational research have many similarities [ 13 ] Nevertheless, the distinction between the two should be clarified, as doing so will enable other medical educators to better understand and benefit from the evaluation experience shared. Moreover, since CBME outcomes are complicated and should be considered on many levels, evaluation plans should include a focus, level, and timeline. The focus of an evaluation can be educational, with outcomes relevant to learners, or clinical, with health outcomes relevant to patients. The level of an evaluation can be micro, meso, or macro, targeting an individual, a program, or a system, respectively. The timeline of an evaluation can investigate outcomes during the program, after the program (i.e., how well learners have put what they learned in a CBME program into practice), and in the long term (i.e., how well learners are doing as practicing physicians) [ 70 ].

Once the evaluation objectives are clearly identified and prioritized, it is logical to start considering the evaluation approach or model that is most appropriate to attain these objectives considering the available resources. In other words, evaluation design should be outlined [ 27 , 68 ]. The choice of an evaluation approach or model affects the accuracy of assessing certain tasks carried out by or to specific subjects in a particular setting [ 68 , 71 , 72 , 73 , 74 , 75 ] This accuracy is referred to as an evaluation’s internal validity. Yet, the external validity of an evaluation entails that the evaluation results are generalizable to other subjects and other settings [ 68 ]. Each model has its own strengths and weaknesses, which require careful examination when planning an evaluation [ 14 , 73 , 74 , 75 ]. Explaining and justifying why a particular evaluation approach was chosen for a specific curriculum can enrich the lessons learned from the evaluation and aid other educators. Furthermore, some of the available models were more utilized within various educational contexts than others [ 17 ] that calls for a continuous documentation of the evaluation approaches or models used to inform theory and practice. Considering the importance of reporting the approaches and models used, it is unfortunate that most of the publications did not indicate the approach/model they used for evaluation, which limits educators’ abilities to utilize the plans and build on their evidence.

Another critical task in the evaluation process is deciding on the measurement tool or instrument to be used. The tool choice will determine what data will be gathered and how they will be collected and analyzed [ 27 , 68 ]. Thus, the choice should consider the evaluation objective as well as the uses, strengths, and limitations of each tool. The evidence in this review indicates that questionnaires or surveys were the most utilized tools in evaluating competency-based curricula. This result can be attributed to the advantages of this method (for example, it is a convenient and economical tool that is easy to administer and analyze and can be utilized with many individuals) [ 27 , 68 ]. Nevertheless, it is important to highlight that questionnaires and surveys usually target attitudes and perceptions, which usually entails only a surface-level evaluation, according to the Kirkpatrick model [ 76 ]. The results also showed that in around 50% of the mixed-methods evaluations, the questionnaires were combined with another tool, such as interviews or focus groups. Understandably, utilizing an additional tool aims to deepen the level of the evaluation focus to include learning, behaviors, or results [ 76 ].

The evaluation evidence must be compared with a standard or target for educators to judge the program and make decisions [ 12 ]. Standards can be implicit or explicit, but they usually provide an understanding of what is ideal [ 12 ]. Worryingly, the results of this review revealed that many of the included studies did not clarify the standards they used to judge different CBME curricula. However, the studies that reported their standards used accreditation criteria or broad competencies frameworks, such as CanMeds, which consider the guides of specialized associations, such as family physicians or nursing. Although deciding what standard to use can be challenging to those designing and evaluating programs, evaluating without an understanding of the level of quality desired can lead to many complications and a waste of resources.

Communicating and reporting evaluation results are crucial to attaining the evaluation objectives [ 27 , 68 , 75 ]. Moreover, effective communication strategies have many important functions, such as providing decision makers with the necessary data to make an informed decision. Informing other stakeholders about the results is also important to achieve their support in implementing program changes and nourish a culture of quality [ 77 , 78 ]. Around half of the authors of studies included in this review indicated that they were publishing to share their own evaluation experiences, while the other half did not. Regardless, none of the studies shared or indicated how their results were reported and communicated, which is an important part of the evaluation cycle that should not be overlooked when sharing evaluation lessons within the scientific community. Reporting the results also ensures quality transformation by closing the evaluation cycle and encourages future engagement in evaluation among different stakeholders [ 78 , 79 , 80 ]. Moreover, the results of the evaluation should be shared publicly to contribute to increasing public trust in educational programs and their outcomes [ 19 , 69 ].

In summary, this review of evaluation practices within competency-based curricula for undergraduate and postgraduate health professional programs provides valuable insight into the current landscape. The results of the review show that most evaluation practices published pertain to postgraduate medical programs. In addition, by examining the objectives, models, tools, standards, and communication of evaluation results, this study exposes a discrepancy between the reported evaluation practices and identified evaluation elements. This discrepancy extends to the data that are reported, which makes it even more difficult to synthesize a holistic picture and definitively fulfill the aim of the review. Moreover, the issue of missing information poses serious challenges for educators who try to leverage existing knowledge to inform their curriculum development and improvement efforts, and it highlights the need for a more systematic and transparent approach to evaluation within CBME.

This review illustrates the importance of agreeing on the main evaluation elements to be reported when publishing a CBME evaluation. Establishing a shared understanding of these fundamental elements will give educators a framework for enhancing the practical utility of evaluation methodologies. In addition, educators and practitioners can ensure that the evaluation process yields more insightful outcomes and is better tailored to meet the needs of the educational context.

Data availability

The datasets used during the current study are available from the corresponding author on reasonable request.

Abbreviations

Outcome-Based Medical Education

Competency-Based Medical Education

Embedded Evaluation Model

Zaini R, Bin Abdulrahman K, Alkhotani A, Al-Hayani AA, Al-Alwani A, Jastaniah S. Saudi meds: a competence specification for Saudi medical graduates. Med Teach. 2011;33(7):582–4.

Article   Google Scholar  

Davis MH, Amin Z, Grande JP, O’Neill AE, Pawlina W, Viggiano TR et al. Case studies in outcome-based education. Med Teach. 2009; 29(7):717–22. https://doi.org/10.1080/01421590701691429

Caccia N, Nakajima A, Kent N. Competency-based medical education: The wave of the future. JOGC. 2015; 37(4):349–53. https://doi.org/10.1016/S1701-2163(15)30286-3

Danilovich N, Kitto S, Price D, Campbell C, Hodgson A, Hendry P. Implementing competency-based medical education in family medicine: a narrative review of current trends in assessment. Fam Med. 2021;53:9–22.

Hawkins RE, Welcher CM, Holmboe ES, Kirk LM, Norcini JJ, Simons KB et al. Implementation of competency-based medical education: Are we addressing the concerns and challenges? Med Educ. 2015 Oct. 22; 49(11):1086–102. https://doi.org/10.1111/medu.12831

Simpson JG, Furnace J, Crosby J, Cumming AD, Evans PA, David MF, Ben et al. The Scottish doctor-learning outcomes for the medical undergraduate in Scotland: A foundation for competent and reflective practitioners. Med Teach. 2002; 24(2):136–43. https://doi.org/10.1080/01421590220120713

Swing SR. The ACGME outcome project: retrospective and prospective. Med Teach. 2007; 29(7):648–54. https://doi.org/10.1080/01421590701392903

Frank JR, Danoff D. The CanMEDS initiative: Implementing an outcomes-based framework of physician competencies. Med Teach. 2007; 29(7):642–7. https://doi.org/10.1080/01421590701746983

van Melle E, Frank JR, Holmboe ES, Dagnone D, Stockley D, Sherbino J. A core components framework for evaluating implementation of competency-based medical education programs. Acad Med. 2019 Jul 1 [cited 2022 Mar 19]; 94(7):1002–9. https://pubmed.ncbi.nlm.nih.gov/30973365/

Hamza DM, Ross S, Oandasan I. Process and outcome evaluation of a CBME intervention guided by program theory. J Eval Clin Pract. 2020; 26(4):1096–104. https://doi.org/10.1111/jep.13344

Van Melle E, Frank J, Holmboe E, Dagnone D, Stockley D, Sherbino J. A core components framework for evaluating implementation of competency-based medical education programs. Acad Med. 2019;94:1.

Google Scholar  

Giancola SP. Program evaluation: Embedding evaluation into program design and development. SAGE Publications; 2020. https://books.google.com.sa/books?id=H-LFDwAAQBAJ

Morrison J, Evaluation. BMJ. 2003; 326(7385):385. http://www.bmj.com/content/326/7385/385.abstract

Glatthorn AA. Curriculum leadership: Strategies for development and implementation. 4th ed. Thousand Oaks, California: SAGE Publications, Inc., 2016. https://search.library.wisc.edu/catalog/9910219273702121

Stufflebeam D. Evaluation models. New Dir Eval. 2001; 2001(89):7–98.

Mertens D, Wilson A. Program evaluation theory and practice. 2nd ed. United States of America: The Guilford; 2019.

Nouraey P, Al-Badi A, Riasati M, Maata RL. Educational program and curriculum evaluation models: a mini systematic review of the recent trends. UJER. 2020;8:4048–55.

Sarah Schiekirka, Markus A, Feufel C, Herrmann-Lingen T, Raupach. Evaluation in medical education: a topical review of target parameters, data collection tools and confounding factors. Ger Med Sci. 2015; 13 (16).

Van Melle E, Hall A, Schumacher D, Kinnear B, Gruppen L, Thoma B, et al. Capturing outcomes of competency-based medical education: the call and the challenge. Med Teach. 2021;43:1–7.

Lomis KD, Mejicano GC, Caverzagie KJ, Monrad SU, Pusic M, Hauer KE. The critical role of infrastructure and organizational culture in implementing competency-based education and individualized pathways in undergraduate medical education. Med Teach. 2021; 43(sup2):S7–16. https://doi.org/10.1080/0142159X.2021.1924364

Taber S, Frank JR, Harris KA, Glasgow NJ, Iobst W, Talbot M. Identifying the policy implications of competency-based education. Med Teach. 2010; 32(8):687–91. https://doi.org/10.3109/0142159X.2010.500706

Zaini RG, bin Abdulrahman KA, Al-Khotani AA, Al-Hayani AMA, Al-Alwan IA, Jastaniah SD. Saudi Meds: A competence specification for Saudi medical graduates. Med Teach. 2011 Jul [cited 2022 Mar 19]; 33(7):582–4. https://pubmed.ncbi.nlm.nih.gov/21696288/

Leung WC. Competency based medical training: review * Commentary: The baby is thrown out with the bathwater. BMJ. 2002 Sept 28; 325(7366):693–6. https://doi.org/10.1136/BMJ.325.7366.693

Hall AK, Schumacher DJ, Thoma B, Caretta-Weyer H, Kinnear B, Gruppen L et al. Outcomes of competency-based medical education: A taxonomy for shared language. Med Teach. 2021;43(7):788–93. https://doi.org/10.1080/0142159X.2021.1925643

Masoomi R. What is the best evidence medical education? Res Dev Med Educ. 2012;1:3–5.

Grant MJ, Booth A. A typology of reviews: An analysis of 14 review types and associated methodologies. Health Info Libr J. 2009; 26(2):91–108. https://doi.org/10.1111/j.1471-1842.2009.00848.x

Sharma R, Gordon M, Dharamsi S, Gibbs T. Systematic reviews in medical education: A practical approach: AMEE Guide 94. Med Teach. 2015; 37(2):108–24. https://doi.org/10.3109/0142159X.2014.970996

Giancola SP. Program evaluation embedding evaluation into program design and development. Thousand Oaks: Sage Publication Ltd.; 2020.

Acai A, Cupido N, Weavers A, Saperson K, Ladhani M, Cameron S et al. Competence committees: The steep climb from concept to implementation. Med Educ. 2021; 55(9):1067–77. https://doi.org/10.1111/medu.14585

Zhang P, Hamza D, Ross S, Oandasan I. Exploring change after implementation of family medicine residency curriculum reform. Fam Med. 2019;51:331–7.

Thoma B, Hall A, Clark K, Meshkat N, Cheung W, Desaulniers P et al. Evaluation of a national competency-based assessment system in emergency medicine: a CanDREAM Study. J Grad Med Educ. 2020;12(4).

Schönwetter D, Law D, Mazurat R, Sileikyte R, Nazarko O. Assessing graduating dental students’ competencies: the impact of classroom, clinic and externships learning experiences. Eur J Dent Educ. 2011;15:142–52.

Nousiainen M, Mironova P, Hynes M, Takahashi S, Reznick R, Kraemer W, et al. Eight-year outcomes of a competency-based residency training program in orthopedic surgery. Med Teach. 2018;40:1–13.

Railer J, Stockley D, Flynn L, Hastings Truelove A, Hussain A. Using outcome harvesting: assessing the efficacy of CBME implementation. J Eval Clin Pract. 2020; 26(4).

Janssen P, Keen L, Soolsma J, Seymour L, Harris S, Klein M, et al. Perinatal nursing education for single-room maternity care: an evaluation of a competency-based model. J Clin Nurs. 2005;14:95–101.

Goudreau J, Pepin J, Dubois S, Boyer L, Larue C, Legault A. A second generation of the competency-based approach to nursing education. Int J Nurs Educ Scholarsh. 2009;6:Article15.

Fahim C, Bhandari M, Yang I, Sonnadara R. Development and early piloting of a CanMEDS competency-based feedback tool for surgical grand rounds. J Surg Educ. 2016;73(3):409–15.

Ellaway R, Mackay M, Lee S, Hofmeister M, Malin G, Archibald D, et al. The impact of a national competency-based medical education initiative in family medicine. Acad Med. 2018;93:1.

D’Souza L, Jaswal J, Chan F, Johnson M, Tay KY, Fung K et al. Evaluating the impact of an integrated multidisciplinary head & neck competency-based anatomy & radiology teaching approach in radiation oncology: a prospective cohort study. BMC Med Educ. 2014; 14(1):124. https://doi.org/10.1186/1472-6920-14-124

Crawford L, Cofie N, Mcewen L, Dagnone D, Taylor S. Perceptions and barriers to competency-based education in Canadian postgraduate medical education. J Eval Clin Pract. 2020;26(4):1124–31.

Cox K, Smith A, Lichtveld M. A competency-based approach to expanding the cancer care workforce part III—Improving cancer pain and palliative care competency. J Cancer Educ. 2012;27:507–14.

Freedman AM, Simmons S, Lloyd LM, Redd TR, Alperin MM, Salek SS et al. Public health training center evaluation: A framework for using logic models to improve practice and educate the public health workforce. Health Promot Pract. 2014; 15(1 Suppl):80S-8S. http://europepmc.org/abstract/MED/24578370

Kerfoot B, Baker H, Volkan K, Church P, Federman D, Masser B, et al. Development and initial evaluation of a novel urology curriculum for medical students. J Urol. 2004;172:278–81.

Ketteler E, Auyang E, Beard K, McBride E, McKee R, Russell J, et al. Competency champions in the clinical competency committee: a successful strategy to implement milestone evaluations and competency coaching. J Surg Educ. 2014;71:36–8.

Lipp M. An objectified competency-based course in the management of malocclusion and skeletal problems. J Dent Educ. 2008;72:543–52.

Randolph S, Rogers B, Ostendorf J. Evaluation of an occupational health nursing program through competency achievement on-campus and distance education, 2005 and 2008. AAOHN J. 2011;59:387–99.

Stefanidis D, Acker C, Swiderski D, Heniford BT, Greene F. Challenges during the implementation of a laparoscopic skills curriculum in a busy general surgery residency program. J Surg Educ. 2008;65:4–7.

Stucke R, Sorensen M, Rosser A, Jung S. The surgical consult entrustable professional activity (EPA): defining competence as a basis for evaluation. Am J Surg. 2018;219(2):253–7.

Swider S, Levin P, Ailey S, Breakwell S, Cowell J, Mcnaughton D, et al. Matching a graduate curriculum in public/community health nursing to practice competencies: the Rush University experience. Public Health Nurs. 2006;23:190–5.

Taleghani M, Solomon E, Wathen W. Non-graded clinical evaluation of dental students in a competency‐based education program. J Dent Educ. 2004;68:644–55.

Zuilen M, Mintzer M, Milanez M, Kaiser R, Rodriguez O, Paniagua M, et al. A competency-based medical student curriculum targeting key geriatric syndromes. Gerontol Geriatr Educ. 2008;28:29–45.

Gibson K, Boyle P, Black D, Bennett M, Grimm M, McNeil H. Enhancing evaluation in an undergraduate medical education program. Acad Med. 2008;83:787–93.

Succar T, McCluskey P, Grigg J. Enhancing medical student education by implementing a competency based ophthalmology curriculum. Asia-Pac J Ophthalmol. 2017;6:42–6.

Gruber P, Gomersall C, Joynt G, Shields F, Chu CM, Derrick J. Teaching acute care: a course for undergraduates. Resuscitation. 2007;74:142–9.

Zhang H, Wang B, Zhang L. Reform of the method for evaluating the teaching of medical linguistics to medical students. Chinese Education & Society. 2014; 47(3):60–4. https://doi.org/10.2753/CED1061-1932470305

Koeijers J, Busari J, Duits AJ. A case study of the implementation of a competency-based curriculum in a Caribbean teaching hospital. West Indian Med J. 2012;61:726–32.

Rotthoff T, Ostapczuk MS, de Bruin J, Kröncke KD, Decking U, Schneider M et al. Development and evaluation of a questionnaire to measure the perceived implementation of the mission statement of a competency based curriculum. BMC Med Educ. 2012; 12(1):109. https://doi.org/10.1186/1472-6920-12-109

Day S, Garcia J, Antillon F, Wilimas J, Mckeon L, Carty R, et al. A sustainable model for pediatric oncology nursing education in low-income countries. Pediatr Blood Cancer. 2012;58:163–6.

Lee BH, Chae YM, Hokama T, Kim S. Competency-based learning program in system analysis and design for health professionals. APJPH. 2010; 22(3):299–309. http://www.jstor.org/stable/26723774

Westein M, Vries H, Floor-Schreudering A, Koster A, Buurma H. Development of a postgraduate workplace-based curriculum for specialization of community pharmacists using CanMEDS competencies, entrustable professional activities and programmatic assessment. Am J Pharm Educ. 2018;83:ajpe6863.

de Beer W. Is the RANZCP CPD programme a competency-based educational programme? Australas Psychiatry. 2019; 27(4):404–8. https://doi.org/10.1177/1039856219859279

Battat R, Jhonson M, Wiseblatt L, Renard C, Habib L, Normil M, et al. The Haiti Medical Education Project: Development and analysis of a competency based continuing medical education course in Haiti through distance learning. BMC Med Educ. 2016;16:275.

İlhan E. Evaluation of competency based medical education curriculum. IJPE. 2021;17(3):153–68.

Ekenze SO, Ameh EA. Evaluation of relevance of the components of Pediatric Surgery residency training in West Africa. J Pediatr Surg. 2010; 45(4):801–5. https://www.sciencedirect.com/science/article/pii/S0022346809008495

Iobst WF, Sherbino J, Cate O, Ten, Richardson DL, Dath D, Swing SR et al. Competency-based medical education in postgraduate medical education. Med Teach. 2010; 32(8):651–6. https://doi.org/10.3109/0142159X.2010.500709

Vakani F, Jafri W, Jafri F, Ahmad A. Towards a competency-based postgraduate medical education. J Coll Physicians Surg Pak; 22(7): 476–7.

Pascual TNB, Ros S, Engel-Hills P, Chhem RK. Medical competency in postgraduate medical training programs. Radiology Education. 2012; pp. 29–45. https://doi.org/10.1007/978-3-642-27600-2_4

Frank J, Snell L, Ten Cate O, Holmboe E, Carraccio C, Swing S, et al. Competency-based medical education: theory to practice. Med Teach. 2010;32:638–45.

Lindeman B, Kern D, Lipsett P. Step 6 evaluation and feedback. In: Thomas P, Kern D, Hughes M, Tackett S, Chen B, editors. Curriculum development for medical education: a six-step approach. 4th ed. Johns Hopkins University; 2022. pp. 142–97.

Morrison J. ABC of learning and teaching in medicine: Evaluation. BMJ. 2003; 326(7385):385–7. https://doi.org/10.1136/BMJ.326.7385.385

Wang MC, Ellett CD. Program validation. Topics Early Child Spec Educ. 1982; 1(4):35–49. https://doi.org/10.1177/027112148200100408

McCray E, Lottes JJ. The validation of educational programs. 1976.

Toosi M, Modarres M, Amini M, Geranmayeh M. Evaluation model in medical education: A systematic review. 2016.

Balmer DF, Rama JA, Simpson D. Program evaluation models: Evaluating processes and outcomes in graduate medical education. J Grad Med Educ. 2019; 11(1):99–100. https://doi.org/10.4300/JGME-D-18-01084.1

Vassar M, Wheeler DL, Davison M, Franklin J. Program evaluation in medical education: An overview of the utilization-focused approach. J Educ Eval Health Prof. 2010; 7:1. https://doi.org/10.3352/jeehp.2010.7.1

Kirkpatrick JD, Kirkpatrick WK. Kirkpatrick’s four levels of training evaluation. Association for Talent Development; 2016. (BusinessPro collection). https://books.google.com.sa/books?id=mo--DAAAQBAJ

Moreau KA, Eady K. Program evaluation use in graduate medical education. J Grad Med Educ. 2023; 15(1):15–8. https://doi.org/10.4300/JGME-D-22-00397.1

Durning SJ, Hanson J, Gilliland W, McManigle JM, Waechter D, Pangaro LN. Using qualitative data from a program director’s evaluation form as an outcome measurement for medical school. Mil Med. 2010; 175(6):448–52. https://doi.org/10.7205/MILMED-D-09-00044

Kek M, Hunt L, Sankey M. Closing the loop: A case study of a post-evaluation strategy. 1998.

Wolfhagen HAP, Gijselaers WH, Dolmans D, Essed G, Schmidt HG. Improving clinical education through evaluation. Med Teach. 2009; 19(2):99–103. https://doi.org/10.3109/01421599709019360

Download references

Acknowledgements

The author would like to acknowledge the support that Prof. Ahmad Alrumayyan and Dr. Emad Masuadi gave this research by reviewing its protocol and providing general feedback. In addition, the author would like to acknowledge Dr. Noof Albaz for the discussions about educational program evaluation, which contributed to improving the final version of this manuscript. Thanks, are also due to Mr. Mohammad Alsawadi for reviewing the utilized search terminologies and participating in the selection of appropriate databases and searching them to obtain the needed lists of articles.

Not applicable.

Author information

Authors and affiliations.

Department of Medical Education, College of Medicine, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia

Nouf Sulaiman Alharbi

King Abdullah International Medical Research Center, Riyadh, Saudi Arabia

Ministry of the National Guard - Health Affairs, Riyadh, Saudi Arabia

You can also search for this author in PubMed   Google Scholar

Contributions

The author takes full responsibility for study conception, design, data collection, analysis, interpretation, and manuscript preparation.

Corresponding author

Correspondence to Nouf Sulaiman Alharbi .

Ethics declarations

Ethics approval and consent to participate.

Ethical clearance for conducting this review was attained from King Abdullah International Research Center (KAIMRC), King Saud Bin Abdulaziz University for Health Sciences (RYD-22-419812-52334).

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, supplementary material 3, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Alharbi, N.S. Evaluating competency-based medical education: a systematized review of current practices. BMC Med Educ 24 , 612 (2024). https://doi.org/10.1186/s12909-024-05609-6

Download citation

Received : 13 November 2023

Accepted : 27 May 2024

Published : 03 June 2024

DOI : https://doi.org/10.1186/s12909-024-05609-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Competency-based medical education
• Program evaluation
• Undergraduate medical education
• Postgraduate medical education
• Curriculum development

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Official websites use .gov

A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS

A lock ( ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Early Safety Findings Among Persons Aged ≥60 Years Who Received a Respiratory Syncytial Virus Vaccine — United States, May 3, 2023–April 14, 2024

Weekly / May 30, 2024 / 73(21);489–494

Anne M. Hause, PhD 1 ; Pedro L. Moro, MD 1 ; James Baggs, PhD 1 ; Bicheng Zhang, MS 1 ; Paige Marquez, MSPH 1 ; Michael Melgar, MD 2 ; Amadea Britton, MD 2 ; Erin Stroud, MD 1 ; Tanya R. Myers, PhD 1 ; Jeffrey Rakickas, MD 1 ; Phillip G. Blanc, MD 3 ; Kerry Welsh, MD 3 ; Karen R. Broder, MD 1 ; John R. Su, MD 1 ; David K. Shay, MD 1 ( View author affiliations )

What is already known about this topic?

The Food and Drug Administration licensed Arexvy and Abrysvo vaccines in May 2023 for prevention of respiratory syncytial virus (RSV) lower respiratory tract disease in adults aged ≥60 years. In trials, Guillain-Barré syndrome (GBS) was identified as a potential safety concern.

What is added by this report?

Findings are consistent with those from trials; reports of GBS (5.0 and 1.5 reports per million doses of Abrysvo and Arexvy vaccine administered, respectively) were more common than expected background rates.

What are the implications for public health practice?

The Advisory Committee on Immunization Practices (ACIP) recommends adults aged ≥60 years may receive 1 dose of RSV vaccine. Population-based surveillance will evaluate the potential risk for GBS to guide ACIP recommendations.

• Article PDF
• Full Issue PDF

In May 2023, the Food and Drug Administration (FDA) licensed Arexvy and Abrysvo vaccines for prevention of respiratory syncytial virus (RSV) lower respiratory tract disease in adults aged ≥60 years. In prelicensure trials, Guillain-Barré syndrome (GBS) was identified as a potential safety concern. During August 4, 2023–March 30, 2024, at least 10.6 million adults aged ≥60 years received a recommended RSV vaccine. During May 3, 2023–April 14, 2024, CDC reviewed data reported after RSV vaccination to V-safe, an active U.S. surveillance system that invites enrolled participants to complete web-based surveys, and reports to the Vaccine Adverse Event Reporting System (VAERS), a passive, voluntary surveillance system that accepts adverse event reports from the public, providers, and manufacturers. Findings from V-safe and VAERS were generally consistent with those from trials. Reporting rates of GBS after RSV vaccination in VAERS (5.0 and 1.5 reports per million doses of Abrysvo and Arexvy vaccine administered, respectively) were higher than estimated expected background rates in a vaccinated population. CDC and FDA are conducting population-based surveillance to assess risks for GBS and other adverse events. Findings from these studies will help guide development of Advisory Committee on Immunization Practices recommendations.

Introduction

Respiratory syncytial virus (RSV) infection can cause lower respiratory tract disease, hospitalization, and death in older adults and is responsible for substantial morbidity and mortality among this age group ( 1 ). The Food and Drug Administration (FDA) licensed Arexvy (GlaxoSmithKline Biologicals [GSK]) and Abrysvo (Pfizer Inc.) vaccines on May 3 and May 31, 2023, respectively, for prevention of lower respiratory tract disease caused by RSV in adults aged ≥60 years ( 2 , 3 ). On June 21, 2023, the Advisory Committee on Immunization Practices (ACIP) recommended that adults aged ≥60 years may receive a single dose of RSV vaccine, using shared clinical decision-making ( 4 ). Guillain-Barré syndrome (GBS) was identified as a potential vaccine safety concern in clinical trials of both RSV vaccines ( 4 ). To characterize early post-marketing vaccine safety findings in adults aged ≥60 years after RSV vaccination, CDC reviewed health surveys and adverse events reported to V-safe, an active U.S. surveillance system that sends web surveys to enrolled participants during the 6 weeks after vaccination, and the Vaccine Adverse Event Reporting System (VAERS), a passive, voluntary surveillance system that monitors adverse events after vaccination, during May 3, 2023–April 14, 2024* ( 5 ). During August 4, 2023–March 30, 2024, approximately 7.2 million adults aged ≥60 years received GSK RSV vaccine, and 3.4 million received Pfizer RSV vaccine. † Among the 16,220 V-safe participants aged ≥60 years who reported receiving an RSV vaccine and completed one or more daily surveys, 39.0% reported at least one symptom after vaccination; 0.4% of participants reported receiving medical care. VAERS received 3,200 reports of adverse events after RSV vaccination among persons aged ≥60 years (including 28 verified reports of GBS); 91.2% of reports were classified as nonserious. Estimated VAERS GBS reporting rates after RSV vaccination were 5.0 and 1.5 reports per million administered doses of Pfizer and GSK vaccines, respectively. CDC and the partnership between FDA and the Centers for Medicare & Medicaid Services are conducting population-based surveillance assessments of RSV vaccine safety.

V-safe ( https://vsafe.cdc.gov ) is a voluntary, active U.S. surveillance system that sends web surveys to enrolled participants on days 0–7 after vaccination, based on the reported vaccination date. § V-safe surveys for adults aged ≥60 years who received an RSV vaccine were available starting October 20, 2023. Daily surveys include questions about local injection site and systemic reactions and health impacts experienced. ¶ Participants reporting medical care for symptoms are also prompted to complete a VAERS report.

VAERS ( https://vaers.hhs.gov ) accepts reports of adverse events from health care providers, vaccine manufacturers, and members of the public ( 5 ). Reports to VAERS generally cannot be used to determine causal associations between adverse events and vaccination. Medical Dictionary for Regulatory Activities Preferred Terms (MedDRA PTs) are assigned by VAERS staff members to signs, symptoms, and diagnostic findings in VAERS reports.** Reports of serious events (including death) to VAERS during May 3, 2023–April 14, 2024, and relevant available medical records were reviewed by CDC experts to form a clinical impression of each reported outcome †† ( 6 ). Using selected MedDRA PTs, a search was performed to identify outcomes of interest, including GBS and immune thrombocytopenia (ITP), multiple cases of which were identified in clinical reviews of serious reports. §§

Symptoms and health impacts reported during the week after RSV vaccination were described for V-safe participants aged ≥60 years who were vaccinated during May 3, 2023–April 14, 2024, and completed one or more daily surveys. Primary VAERS adverse event reports after RSV vaccination for persons aged ≥60 years were described by serious and nonserious classification and MedDRA PTs. ¶¶ All analyses were conducted using SAS software (version 9.4; SAS Institute). Reporting rates for GBS reports that met the Brighton Collaboration case definition ( 6 ) were estimated using available doses administered as the denominator. This activity was reviewed by CDC, deemed not research, and was conducted consistent with applicable federal law and CDC policy.***

Review of V-safe Data

During May 3, 2023–April 14, 2024, a total of 16,220 V-safe participants aged ≥60 years reported receiving an RSV vaccine and completed at least one daily survey ( Table 1 ). The median age of these participants was 70 years (range = 60–94 years), 9,684 (59.7%) were women, 6,402 (39.5%) received GSK vaccine, 3,882 (23.9%) received Pfizer vaccine, and 5,936 (36.6%) did not know the manufacturer of the vaccine they received. Approximately one third (5,043; 31.1%) of participants reported receiving one or more other vaccines during the same visit; those most commonly reported were COVID-19 (3,370; 20.8%) and influenza (2,630; 16.2%) vaccines. During the week after vaccination, 6,328 (39.0%) participants reported symptoms they considered possibly related to RSV vaccination. Injection site symptoms were reported by 2,808 (43.9%) participants who received GSK vaccine and 787 (20.3%) who received Pfizer vaccine. Most injection site symptoms were mild (3,351; 20.7%) or moderate (1,889; 11.6%) ( Table 2 ). Systemic symptoms were reported by 2,344 (36.6%) who received GSK and 839 (21.6%) who received Pfizer. Most systemic symptoms were mild (1,997; 12.3%) or moderate (2,184; 13.5%). The most frequently reported symptoms after RSV vaccination were pain at or near the injection site (5,026; 31.0%), fatigue or tiredness (3,327; 20.5%), and muscle or body aches (2,843; 17.5%). Among those who reported other symptoms, those most commonly reported were sore throat (54; 0.3%), dizziness (38; 0.2%), and runny nose (38; 0.2%).

During the week after vaccination, 1,264 (7.8%) participants reported that they were unable to complete their normal daily activities because of the reported symptoms; 68 (0.4%) reported receiving medical care for the reported symptoms. Among those who reported receiving medical care, five completed a report to VAERS; events reported were chalazion, lower than normal blood pressure, exacerbation of chronic obstructive pulmonary disease, injection site pain, and suspected lichen planus.

Review of VAERS Data

During May 3, 2023–April 14, 2024, VAERS received and processed 3,200 reports of adverse events among persons aged ≥60 years who reported receiving an RSV vaccine ( Table 3 ), ††† including 2,191 (68.5%) for GSK vaccine, 921 (28.8%) for Pfizer, and 88 (2.8%) for which the vaccine manufacturer was unknown. The median age of persons for whom a VAERS report was received was 72 years (range = 60–112 years), and 2,237 (69.9%) reports were for women. At least one other vaccine was received at the same visit for approximately one third (1,050; 32.8%) of reports, with influenza vaccine administered most commonly (625; 19.5%). Among the 3,200 VAERS reports, 346 (10.8%) specified a vaccination error (e.g., product administered at an inappropriate site, extra dose administered, or incorrect route of product administration); 64 (2.0%) reports also indicated that an adverse health event had occurred. Overall, 2,919 (91.2%) reports were classified as nonserious, including 2,026 (92.5%) after receipt of GSK vaccine and 821 (89.1%) after receipt of Pfizer vaccine. Commonly reported events included pain in an extremity (384; 13.2%), headache (376; 12.9%), pain (373; 12.8%), injection site pain (370; 12.7%), and fatigue (355; 12.2%).

Among all VAERS reports, 281 (8.8%) were classified as serious, including 216 (6.8%) for hospitalization, 81 (2.5%) for a life-threatening illness, 66 (2.1%) for a permanent disability, and 34 (1.1%) for death. Clinical impressions of serious reports included stroke or transient ischemic attack (24), GBS (37; 28 met case definition), §§§ atrial fibrillation (14), other thromboembolic event (13), encephalitis or aseptic meningitis (11), immune thrombocytopenia (11), sepsis, bacteremia, or both (11), and shoulder pain (11). Among the 28 reports of GBS after vaccination that met case definition, 11 (39.3%) were after GSK vaccine (1.5 reports per 1 million doses administered), and 17 (60.3%) were after Pfizer vaccine (5.0 reports per 1 million doses administered). For the 18 reports of death with sufficient information for review, reported causes of death were acute respiratory distress syndrome, bronchopneumonia, cardiac event, cardiopulmonary arrest, ehrlichiosis, GBS (two), hepatic encephalopathy, hypoxic respiratory failure, multifocal leukoencephalopathy, respiratory failure, rhabdomyolysis, RSV infection, sepsis, sepsis secondary to pneumonia, Pseudomonas bacteremia, varicella-zoster virus meningoencephalitis, and vascular dementia.

This review provides early findings from V-safe and VAERS surveillance systems during the first months of GSK and Pfizer RSV vaccine administration among U.S. adults aged ≥60 years. The findings in this report are generally consistent with those from safety data collected in prelicensure clinical trials, including the observance of GBS cases ¶¶¶ ( 2 , 3 ).

In V-safe, injection site and systemic reactions were more frequently reported among those who received GSK than among those who received Pfizer vaccine; few participants reported receiving medical care ( 2 , 3 ). Expected vaccination reactions (e.g., pain in extremity, headache, and fatigue) were among the most frequently reported events among nonserious VAERS reports. Using VAERS data, estimated GBS reporting rates after RSV vaccination among persons aged ≥60 years were 5.0 and 1.5 reports per million doses of Pfizer and GSK vaccine administered, respectively.

VAERS reporting rates of GBS after mRNA COVID-19 vaccination were used to estimate expected background rates of GBS in this study population; no excess risk for GBS was observed after mRNA COVID-19 vaccinations in active Vaccine Safety Datalink surveillance ( 7 ). VAERS reporting rates for GBS among adults aged ≥65 years were 0.43 and 0.54 per million doses of Pfizer-BioNTech and Moderna COVID-19 vaccines, respectively**** ( 8 ). Thus, using the reporting rate for mRNA COVID-19 vaccines as an estimate of background rate, reports of GBS after RSV vaccination were more common than expected. Two deaths among vaccine recipients who had been diagnosed with GBS were reported.

Limitations

The findings in this report are subject to at least four limitations. First, V-safe is a voluntary program, and data might not be representative of the vaccinated population. Second, VAERS is a passive surveillance system and is subject to reporting biases, underreporting (especially of nonserious events), and incomplete data reporting. Third, VAERS generally cannot determine causal associations between adverse events and vaccination ( 5 ). Finally, because these data do not include a comparison group of unvaccinated persons with a similar likelihood of receiving an RSV vaccine, estimating the magnitude of risk for serious but rare outcomes (e.g., GBS) after vaccination is not possible.

Implications for Public Health Practice

On February 29, 2024, ACIP announced that, based on a thorough review of currently available data, the estimated benefits of RSV vaccination continued to outweigh potential risks. RSV vaccination continues to be recommended for adults aged ≥60 years using shared clinical decision-making ( 9 ). CDC and FDA are conducting active safety evaluations to assess risks for GBS and other adverse events of special interest after RSV vaccination. Results of these studies will help guide future CDC RSV vaccine recommendations.

Acknowledgments

Charles Licata, Isaac McCullum, Seth Meador, Amna Mehmood, Narayan Nair, Carmen Ng, Suchita Patel, Tom Shimabukuro, Jonathan Tewodros, Peter Van Ameyden Van Duym, Jared Woo.

Corresponding author: Anne M. Hause, [email protected] .

1 Division of Healthcare Quality Promotion, National Center for Emerging and Zoonotic Infectious Diseases, CDC; 2 Coronavirus and Other Respiratory Diseases Division, National Center for Immunization and Respiratory Diseases, CDC; 3 Food and Drug Administration, Silver Spring, Maryland.

All authors have completed and submitted the International Committee of Medical Journal Editors form for disclosure of potential conflicts of interest. Phillip G. Blanc reports ownership of Community Health Systems, Inc. stock, sold in 2022. No other potential conflicts of interest were disclosed.

* This review includes V-safe data collected during October 20, 2023–April 14, 2024, for persons vaccinated during May 3, 2023–April 1, 2024. This review includes VAERS reports collected during May 3, 2023–April 14, 2024, for persons vaccinated during May 3, 2023–April 2, 2024, and reports that are missing a date of vaccination.

† Projected doses administered during August 4, 2023–March 30, 2024, at physician medical offices (data source: IQVIA Custom Medical Claims [Dx]; data current through April 6, 2024) and during August 12, 2023–March 29, 2024, at retail pharmacies (data source: IQVIA Custom Longitudinal Prescription Claims [LRx]); data are current through April 5, 2024). IQVIA data do not include vaccinations administered at other medical settings such as public health clinics, including workplaces and community locations. IQVIA uses a proprietary methodology to project doses administered in all retail pharmacies and all office-based physicians based on a sample of retail pharmacies and a sample of office-based physicians. The projection to office-based physicians uses a list of U.S.-licensed office-based physicians maintained by the American Medical Association.

§ Registered account holders can add dependents to their accounts and complete surveys on their behalf.

¶ Symptom severity is self-reported as mild (symptoms noticeable, but not problematic), moderate (symptoms limit normal daily activities), or severe (symptoms make daily activities difficult or impossible); some symptoms have additional, specific severity definitions. Participants who report “other” systemic reactions can select signs, symptoms, and health conditions from a dropdown menu. The dropdown menu of 814 common signs, symptoms, and health conditions consists of the most common self-reported MedDRA PTs among VAERS reports.

** Each VAERS report might be assigned more than one MedDRA PT. A MedDRA-coded event does not indicate a medically confirmed diagnosis. https://www.meddra.org/how-to-use/basics/hierarchy

†† VAERS reports are classified as serious ( http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?fr5600.80 ) if any of the following events are reported: hospitalization, prolongation of hospitalization, life-threatening illness, permanent disability, congenital anomaly or birth defect, or death. Medical records are requested for reports of serious adverse events, including autopsy findings and death certificates for reported deaths.

§§ CDC experts reviewed primary reports of GBS to VAERS. Reports of GBS within 42 days of RSV vaccination that met the Brighton Collaboration case definition for GBS levels 1–3 were included. Clinical reviews of serious reports identified multiple cases of ITP. To identify other potential ITP cases, a group of MedDRA PTs for thrombocytopenia were used; two additional reports of ITP were detected with this search.

¶¶ A primary VAERS report is the first report of an event after vaccination for a particular patient; subsequent reports pertaining to the same patient and event (from the same reporter or other reporters) are termed secondary reports. Excluded from analysis were reports with missing age (1,064) or age <60 years (388), including reports indicating pregnancy at time of vaccination (401).

*** 45 C.F.R. part 46.102(l)(2), 21 C.F.R. part 56; 42 U.S.C. Sect. 241(d); 5 U.S.C. Sect. 552a; 44 U.S.C. Sect. 3501 et seq.

††† Processed VAERS reports are those that have been coded using MedDRA, deduplicated, and undergone standard quality assurance and quality control review.

§§§ This review of reports to VAERS includes 21 of the 23 verified GBS reports after RSV vaccination presented at the February 29, 2024, ACIP meeting. Excluded from this review were one report for a person aged 50 years and one report that did not include RSV vaccine in the primary report.

¶¶¶ All but one verified VAERS report of GBS indicated that symptoms occurred within 21 days of RSV vaccination. The other report indicated onset of GBS symptoms 22 days after RSV vaccination. In GSK RSV vaccine clinical trials in older adults (18,304 vaccine recipients aged ≥60 years), one case of GBS was reported within 42 days after receipt of the GSK vaccine. In Pfizer RSV vaccine clinical trials in older adults (20,255 vaccine recipients aged ≥60 years), two cases of GBS were reported within 42 days after vaccination.

**** VAERS reports of GBS within 21 days of Pfizer-BioNTech and Moderna COVID-19 vaccination that met the Brighton Collaboration case definition for GBS levels 1–3 were included in reporting rate estimations.

• McLaughlin JM, Khan F, Begier E, et al. Rates of medically attended RSV among US adults: a systematic review and meta-analysis. Open Forum Infect Dis 2022;9:ofac300. https://doi.org/10.1093/ofid/ofac300 PMID:35873302
• Food and Drug Administration. Approval letter: Arexvy. Silver Spring, MD: US Department of Health and Human Services, Food and Drug Administration; 2023. https://www.fda.gov/media/167806/download
• Food and Drug Administration. Approval letter: Abrysvo. Silver Spring, MD: US Department of Health and Human Services, Food and Drug Administration; 2023. https://www.fda.gov/media/168890/download
• Melgar M, Britton A, Roper LE, et al. Use of respiratory syncytial virus vaccines in older adults: recommendations of the Advisory Committee on Immunization Practices—United States, 2023. MMWR Morb Mortal Wkly Rep 2023;72:793–801. https://doi.org/10.15585/mmwr.mm7229a4 PMID:37471262
• Shimabukuro TT, Nguyen M, Martin D, DeStefano F. Safety monitoring in the Vaccine Adverse Event Reporting System (VAERS). Vaccine 2015;33:4398–405. https://doi.org/10.1016/j.vaccine.2015.07.035 PMID:26209838
• Sejvar JJ, Kohl KS, Gidudu J, et al.; Brighton Collaboration GBS Working Group. Guillain-Barré syndrome and Fisher syndrome: case definitions and guidelines for collection, analysis, and presentation of immunization safety data. Vaccine 2011;29:599–612. https://doi.org/10.1016/j.vaccine.2010.06.003 PMID:20600491
• Hanson KE, Goddard K, Lewis N, et al. Incidence of Guillain-Barré syndrome after COVID-19 vaccination in the Vaccine Safety Datalink. JAMA Netw Open 2022;5:e228879. https://doi.org/10.1001/jamanetworkopen.2022.8879 PMID:35471572
• Abara WE, Gee J, Marquez P, et al. Reports of Guillain-Barré syndrome after COVID-19 vaccination in the United States. JAMA Netw Open 2023;6:e2253845. https://doi.org/10.1001/jamanetworkopen.2022.53845 PMID:36723942
• Britton A. RSV vaccination in older adults: work group interpretations. Presented at the Advisory Committee on Immunization Practices meeting, Atlanta, GA; February 29, 2024. https://www.cdc.gov/vaccines/acip/meetings/downloads/slides-2024-02-28-29/08-RSV-Adults-Britton-508.pdf

* Percentage of participants who reported a symptom or health impact at least once during days 0–7 postvaccination. † Fever is a self-reported symptom and might not reflect the clinical definition of fever. § Among those who reported “Other” symptoms, 409 selected additional symptoms from a dropdown menu; most commonly selected were sore throat (54), dizziness (38), runny nose (38), cough (27), dizziness upon standing (17), and congestion (16). ¶ Participants can select from more than one type of care received from a health professional, including doctor appointment or urgent care clinic visit, telehealth, virtual health, or email health consultation, emergency department or emergency department visit, hospitalization, and other.

* Symptom severity was self-reported. The following definitions describe the severity of symptoms: mild = symptoms noticeable, but not problematic; moderate = symptoms limit normal daily activities; or severe = symptoms make daily activities difficult or impossible. Some symptoms have specific severity definitions. † Percentage of participants who reported a symptom or health impact at least once during days 0–7 postvaccination. § The symptom severity total differs from the total systemic symptoms reported because severity is not collected for other symptoms. ¶ “Fever or feverish” is a self-reported symptom and might not correspond to a clinical definition of fever. The number of registrants (1,765) who reported having a fever or feeling feverish differs from the total who entered information about temperature (643). Severity of fever was defined as follows: normal or subfebrile = 96.0°–100.3°F (35.6°–37.9°C); mild = 100.4°–102.2°F (38.0°–39.0°C); moderate = 102.3°–103.9°F (39.1°–39.9°C); and severe = 104.0°–107.0°F (40.0°–41.7°C).

Abbreviations: GBS = Guillain Barré syndrome; MedDRA PT = Medical Dictionary for Regulatory Activities Preferred Term; RSV = respiratory syncytial virus; VAERS = Vaccine Adverse Event Reporting System. * Each event is a sign or symptom in a VAERS report, coded by a MedDRA PT. MedDRA PTs are assigned by VAERS staff members after review of available data. Each VAERS report might be assigned more than one MedDRA PT, which can include normal diagnostic findings. A MedDRA PT does not necessarily indicate a medically confirmed diagnosis. † Includes the top 10 most frequently coded MedDRA PTs among nonserious reports. § VAERS reports are classified as serious if any of the following are reported: hospitalization (216), prolongation of hospitalization (three), life-threatening illness (81), permanent disability (66), congenital anomaly or birth defect (zero), or death (34). ¶ Serious reports to VAERS were reviewed by CDC physicians and experts to form preliminary clinical impressions. Includes 20 most common events from preliminary clinical impressions; a report might include more than one event. Because of the small number of serious reports, percentages are not provided for serious report events. Other clinical impressions included acute appendicitis, acute hepatitis, acute on chronic renal failure, acute on chronic respiratory failure, acute renal failure, altered mental status (three), anti-neutrophil cytoplasmic antibody vasculitis, angina, autoimmune hemolytic anemia, body temperature fluctuation, cellulitis in leg, chest pain (three), choked, chronic pulmonary fibrosis, chronic obstructive pulmonary disease (two), COVID-19 infection, duodenal ulcer, epidural abscess, episodic memory loss, fall, fever (two), generalized weakness (three), headache (two), hypertension (two), hypoglycemia, laryngospasm, lower extremity ischemia, myalgia (two), myocarditis, nausea and vomiting, osteoarthritis (three), pancreatitis, pancytopenia, polymyalgia rheumatica, posterior reversible encephalopathy syndrome, reactive arthritis, receptive aphasia, respiratory distress, rhabdomyolysis, subdural hematoma after fall, seizure, spinal stenosis post laminectomy, stress cardiomyopathy, systemic inflammatory response syndrome, third cranial nerve palsy, tinnitus and hearing loss (three), unevaluable (four), urinary tract infection, vaccination related anxiety, viral illness with delirium, and visual impairment. ** Includes two reports of angioedema. †† Includes eight reports of new-onset atrial fibrillation. §§ This review of reports to VAERS includes 21 of the 23 verified GBS reports after RSV vaccination (including one report for a person who died) presented at the February 29, 2024, Advisory Committee on Immunization Practices meeting. Excluded from this report were one report for a person aged 50 years and one report that did not include RSV vaccine in the primary report. In addition, seven reports did not meet the case definition or were unverified because of a lack of records, and two reports remain under review. Three additional unverified reports were identified using the selected MedDRA PTs search and are not included in the table. ¶¶ Includes one report of pyoderma. *** Two additional nonserious reports were identified using the selected MedDRA PTs search, which are not included in the table. ††† Includes reports of pulmonary embolism (10), deep vein thrombosis (two), and retinal artery occlusion (one). §§§ For reports of death, the following reported causes of death was available for 18 reports: acute respiratory distress syndrome, bronchopneumonia, cardiac event, cardiopulmonary arrest, ehrlichiosis, GBS (two), hepatic encephalopathy, hypoxic respiratory failure, multifocal leukoencephalopathy, respiratory failure, rhabdomyolysis, RSV infection, sepsis, sepsis secondary to pneumonia, septic shock ( Pseudomonas bacteremia), varicella-zoster virus meningoencephalitis, and vascular dementia.

Suggested citation for this article: Hause AM, Moro PL, Baggs J, et al. Early Safety Findings Among Persons Aged ≥60 Years Who Received a Respiratory Syncytial Virus Vaccine — United States, May 3, 2023–April 14, 2024. MMWR Morb Mortal Wkly Rep 2024;73:489–494. DOI: http://dx.doi.org/10.15585/mmwr.mm7321a3 .

MMWR and Morbidity and Mortality Weekly Report are service marks of the U.S. Department of Health and Human Services. Use of trade names and commercial sources is for identification only and does not imply endorsement by the U.S. Department of Health and Human Services. References to non-CDC sites on the Internet are provided as a service to MMWR readers and do not constitute or imply endorsement of these organizations or their programs by CDC or the U.S. Department of Health and Human Services. CDC is not responsible for the content of pages found at these sites. URL addresses listed in MMWR were current as of the date of publication.

All HTML versions of MMWR articles are generated from final proofs through an automated process. This conversion might result in character translation or format errors in the HTML version. Users are referred to the electronic PDF version ( https://www.cdc.gov/mmwr ) and/or the original MMWR paper copy for printable versions of official text, figures, and tables.

Exit Notification / Disclaimer Policy

• The Centers for Disease Control and Prevention (CDC) cannot attest to the accuracy of a non-federal website.
• Linking to a non-federal website does not constitute an endorsement by CDC or any of its employees of the sponsors or the information and products presented on the website.
• You will be subject to the destination website's privacy policy when you follow the link.
• CDC is not responsible for Section 508 compliance (accessibility) on other federal or private website.

Why musicians die in poverty – South African study reveals battle to make ends meet

Associate of the Gordon Institute for Business Science, University of Pretoria

Disclosure statement

Gwen Ansell conducted this research as part of a team commissioned by Pro Helvetia Johannesburg, who had been responsible for disbursing the Fund concerned.

University of Pretoria provides funding as a partner of The Conversation AFRICA.

View all partners

Benefits and appeals were recently launched to support the medical costs of veteran South African guitarist Madala Kunene . It’s not the first time such initiatives have been necessary, nor the first time that media and politicians have expressed astonishment that a renowned musician “died in poverty”.

Musicians’ dire financial circumstances are sometimes wrongly blamed on irresponsible spending; a musicians’ pension plan is often proposed as a solution. But until very recently, no data existed about what musicians in South Africa really earn, what costs they carry, and the tough trade-offs between professional and household costs they often need to make in order to survive.

Read more: Music streaming in South Africa – new survey reveals musicians get a raw deal

As a music researcher who has written frequently on the economics of music work, I welcomed the opportunity to conduct long-term research with a team at IKS Cultural Consulting into a relief fund supporting SA jazz musicians during the COVID-19 period. That work has now given us concrete information to better understand the picture. Surprisingly, far fewer research projects anywhere in the world have looked at individual musicians than at how “the industry” (major music businesses) fares.

What we found confirms what has been understood anecdotally for a long time. Even “star” musicians are not nearly as well paid as the public believes. In fact, their working costs are sometimes high enough to drive them away from music as a profession.

And mining the fund’s administrative data affirmed how valuable such data is for far more than accounting. It can inform policy too.

The research

The main purpose of the study was to evaluate the impact of the Swiss Jazz Income Relief Fund, administered by Pro Helvetia Johannesburg during the pandemic. Pro Helvetia took on the relief fund as a special project – with finance from a Swiss donor – in the context of its longstanding commitment to sustaining exchange and collaboration in the jazz field between Switzerland and South Africa. The fund paid regular monthly disbursements based on self-reported household income shortfall for 6- or 12-month periods between July 2020 and October 2021.

Identifying details of the fund’s 700-plus recipients were removed before we analysed the income and spending data they submitted. A voluntary anonymous survey followed in 2022, which 311 of those completed. More extended retrospective reflections followed late in 2023 with seven on-the-record interviews.

Musicians’ experience of the fund itself was overwhelmingly positive. And there are significant potential learnings for grant-makers and policy-makers about what a difference it can make when accessible processes are tailored to the patterns of musicians’ earning and spending.

When we asked recipients to say how the funding helped, “Saved me” and “I/we survived” were the words that came back most often. Diverse, innovative ways of using the support to sustain and grow careers were also reported. We heard many heartening stories of community initiatives: food parcels; opening homes to serve as rehearsal spaces; educating the young.

Although our research population was jazz musicians, and we don’t claim their situation is identical to that of musicians making other kinds of music, we’re confident their circumstances apply far more widely. More than 90% of them, like their counterparts worldwide, work in other genres too. Studies across multiple genres from elsewhere concur about the patterns of earnings and working costs we found.

The findings

Our research was skewed towards experienced musicians, some at the height of their careers. They averaged earnings similar to the South African average wage : not what a “star” might earn, but what a formally employed mid-level South African motor mechanic or payroll clerk does – around R26,000 (US$1,392) a month.

However, musicians have no formal employment contracts or packages, or a guaranteed monthly wage. Work is irregular and project-based, with no regulated 35-hour week or any tax concession for the tools of their trade – in fact many instruments are taxed as “luxuries”.

Musicians earn from multiple “ income streams ”: they’re paid from many sources. However, many other COVID-19 relief grants demanded formal proofs of only one consistent wage. Said vocalist and music educator Zodwa Mabena:

You’re running around looking for company numbers, things like that … things most of us don’t have.

Musicians carry significant opportunity costs in terms of the long and usually unpaid rehearsal and practice hours they have to put in to stay skilled. During that time, they cannot do other work. They carry significant working costs: instrument and equipment purchase and upkeep; studio and venue hire; and – most significantly for many – transport. Some reported turning down engagements because travel costs would be higher than the fee offered.

Lasting impact of COVID

Worldwide, musicians have been experiencing a “ cost of working crisis ” since the pandemic. So are South Africans.

There was a 31% drop-off in the number of people who reported earning their main income from music between 2019 and 2021. And 45% of our respondents expressed pessimism about their future in music. All costs have risen. Bassist Shane Cooper told us:

We have a band member living in another city … We used to be able to make it work … (but now) it’s just not financially viable. We don’t play.

Studios and other service providers also have to compensate for the devastating losses incurred during the pandemic. Because some venues have closed and events ceased, work is scarcer, and the fees offered are static or have even dropped.

Meanwhile, musicians have their own COVID-period debts to repay. Some have to repurchase instruments and equipment they were forced to sell. Many lost old colleagues to COVID, and report now having to forge new working relationships with younger freelance players who may expect higher fees. Professional development costs, such as upgrading equipment or making better recordings and promotional videos, fall most heavily on younger, less established musicians.

All this happens in a context where musicians constantly face a trade-off dilemma between spending on sustaining or growing their music and putting food on the table, paying school fees and everything else that must be budgeted for.

So, as we found, it’s not surprising that musicians spend much less on retirement planning than other earners, and proportionately more on dependants, communication and insurance (instrument insurance costs are high).

What needs to happen

But a national artists’ pension scheme might not offer an answer. Musicians might not be able to contribute regularly, because of intermittent work. Should they make the pension scheme payment – or use that money to travel to a gig that might be the only work in sight that month?

Read more: Nearly half of South Africa’s live music workers may quit the industry for good

Our findings suggest that policy and grant-making need to be rooted far more deeply in understanding the real circumstances of musicians’ working lives. Correcting opaque bureaucratic processes, implementing an income streams perspective that understands musicians may have multiple sources of earnings, and giving consideration to musicians’ costs of working, for example, might better help to sustain this important creative industry.

• Music industry
• Cultural economy
• South African culture
• Music in Africa
• Arts industry
• African musicians
• South African musicians

Head of School, School of Arts & Social Sciences, Monash University Malaysia

Chief Operating Officer (COO)

Clinical Teaching Fellow

Data Manager

Director, Social Policy

A systematic literature review of empirical research on ChatGPT in education

• Open access
• Published: 26 May 2024
• Volume 3 , article number  60 , ( 2024 )

Cite this article

You have full access to this open access article

• Yazid Albadarin   ORCID: orcid.org/0009-0005-8068-8902 1 ,
• Mohammed Saqr 1 ,
• Nicolas Pope 1 &
• Markku Tukiainen 1  

555 Accesses

Explore all metrics

Over the last four decades, studies have investigated the incorporation of Artificial Intelligence (AI) into education. A recent prominent AI-powered technology that has impacted the education sector is ChatGPT. This article provides a systematic review of 14 empirical studies incorporating ChatGPT into various educational settings, published in 2022 and before the 10th of April 2023—the date of conducting the search process. It carefully followed the essential steps outlined in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA 2020) guidelines, as well as Okoli’s (Okoli in Commun Assoc Inf Syst, 2015) steps for conducting a rigorous and transparent systematic review. In this review, we aimed to explore how students and teachers have utilized ChatGPT in various educational settings, as well as the primary findings of those studies. By employing Creswell’s (Creswell in Educational research: planning, conducting, and evaluating quantitative and qualitative research [Ebook], Pearson Education, London, 2015) coding techniques for data extraction and interpretation, we sought to gain insight into their initial attempts at ChatGPT incorporation into education. This approach also enabled us to extract insights and considerations that can facilitate its effective and responsible use in future educational contexts. The results of this review show that learners have utilized ChatGPT as a virtual intelligent assistant, where it offered instant feedback, on-demand answers, and explanations of complex topics. Additionally, learners have used it to enhance their writing and language skills by generating ideas, composing essays, summarizing, translating, paraphrasing texts, or checking grammar. Moreover, learners turned to it as an aiding tool to facilitate their directed and personalized learning by assisting in understanding concepts and homework, providing structured learning plans, and clarifying assignments and tasks. However, the results of specific studies (n = 3, 21.4%) show that overuse of ChatGPT may negatively impact innovative capacities and collaborative learning competencies among learners. Educators, on the other hand, have utilized ChatGPT to create lesson plans, generate quizzes, and provide additional resources, which helped them enhance their productivity and efficiency and promote different teaching methodologies. Despite these benefits, the majority of the reviewed studies recommend the importance of conducting structured training, support, and clear guidelines for both learners and educators to mitigate the drawbacks. This includes developing critical evaluation skills to assess the accuracy and relevance of information provided by ChatGPT, as well as strategies for integrating human interaction and collaboration into learning activities that involve AI tools. Furthermore, they also recommend ongoing research and proactive dialogue with policymakers, stakeholders, and educational practitioners to refine and enhance the use of AI in learning environments. This review could serve as an insightful resource for practitioners who seek to integrate ChatGPT into education and stimulate further research in the field.

Similar content being viewed by others

Empowering learners with ChatGPT: insights from a systematic literature exploration

Incorporating AI in foreign language education: An investigation into ChatGPT’s effect on foreign language learners

Large language models in education: A focus on the complementary relationship between human teachers and ChatGPT

Avoid common mistakes on your manuscript.

1 Introduction

Educational technology, a rapidly evolving field, plays a crucial role in reshaping the landscape of teaching and learning [ 82 ]. One of the most transformative technological innovations of our era that has influenced the field of education is Artificial Intelligence (AI) [ 50 ]. Over the last four decades, AI in education (AIEd) has gained remarkable attention for its potential to make significant advancements in learning, instructional methods, and administrative tasks within educational settings [ 11 ]. In particular, a large language model (LLM), a type of AI algorithm that applies artificial neural networks (ANNs) and uses massively large data sets to understand, summarize, generate, and predict new content that is almost difficult to differentiate from human creations [ 79 ], has opened up novel possibilities for enhancing various aspects of education, from content creation to personalized instruction [ 35 ]. Chatbots that leverage the capabilities of LLMs to understand and generate human-like responses have also presented the capacity to enhance student learning and educational outcomes by engaging students, offering timely support, and fostering interactive learning experiences [ 46 ].

The ongoing and remarkable technological advancements in chatbots have made their use more convenient, increasingly natural and effortless, and have expanded their potential for deployment across various domains [ 70 ]. One prominent example of chatbot applications is the Chat Generative Pre-Trained Transformer, known as ChatGPT, which was introduced by OpenAI, a leading AI research lab, on November 30th, 2022. ChatGPT employs a variety of deep learning techniques to generate human-like text, with a particular focus on recurrent neural networks (RNNs). Long short-term memory (LSTM) allows it to grasp the context of the text being processed and retain information from previous inputs. Also, the transformer architecture, a neural network architecture based on the self-attention mechanism, allows it to analyze specific parts of the input, thereby enabling it to produce more natural-sounding and coherent output. Additionally, the unsupervised generative pre-training and the fine-tuning methods allow ChatGPT to generate more relevant and accurate text for specific tasks [ 31 , 62 ]. Furthermore, reinforcement learning from human feedback (RLHF), a machine learning approach that combines reinforcement learning techniques with human-provided feedback, has helped improve ChatGPT’s model by accelerating the learning process and making it significantly more efficient.

This cutting-edge natural language processing (NLP) tool is widely recognized as one of today's most advanced LLMs-based chatbots [ 70 ], allowing users to ask questions and receive detailed, coherent, systematic, personalized, convincing, and informative human-like responses [ 55 ], even within complex and ambiguous contexts [ 63 , 77 ]. ChatGPT is considered the fastest-growing technology in history: in just three months following its public launch, it amassed an estimated 120 million monthly active users [ 16 ] with an estimated 13 million daily queries [ 49 ], surpassing all other applications [ 64 ]. This remarkable growth can be attributed to the unique features and user-friendly interface that ChatGPT offers. Its intuitive design allows users to interact seamlessly with the technology, making it accessible to a diverse range of individuals, regardless of their technical expertise [ 78 ]. Additionally, its exceptional performance results from a combination of advanced algorithms, continuous enhancements, and extensive training on a diverse dataset that includes various text sources such as books, articles, websites, and online forums [ 63 ], have contributed to a more engaging and satisfying user experience [ 62 ]. These factors collectively explain its remarkable global growth and set it apart from predecessors like Bard, Bing Chat, ERNIE, and others.

In this context, several studies have explored the technological advancements of chatbots. One noteworthy recent research effort, conducted by Schöbel et al. [ 70 ], stands out for its comprehensive analysis of more than 5,000 studies on communication agents. This study offered a comprehensive overview of the historical progression and future prospects of communication agents, including ChatGPT. Moreover, other studies have focused on making comparisons, particularly between ChatGPT and alternative chatbots like Bard, Bing Chat, ERNIE, LaMDA, BlenderBot, and various others. For example, O’Leary [ 53 ] compared two chatbots, LaMDA and BlenderBot, with ChatGPT and revealed that ChatGPT outperformed both. This superiority arises from ChatGPT’s capacity to handle a wider range of questions and generate slightly varied perspectives within specific contexts. Similarly, ChatGPT exhibited an impressive ability to formulate interpretable responses that were easily understood when compared with Google's feature snippet [ 34 ]. Additionally, ChatGPT was compared to other LLMs-based chatbots, including Bard and BERT, as well as ERNIE. The findings indicated that ChatGPT exhibited strong performance in the given tasks, often outperforming the other models [ 59 ].

Furthermore, in the education context, a comprehensive study systematically compared a range of the most promising chatbots, including Bard, Bing Chat, ChatGPT, and Ernie across a multidisciplinary test that required higher-order thinking. The study revealed that ChatGPT achieved the highest score, surpassing Bing Chat and Bard [ 64 ]. Similarly, a comparative analysis was conducted to compare ChatGPT with Bard in answering a set of 30 mathematical questions and logic problems, grouped into two question sets. Set (A) is unavailable online, while Set (B) is available online. The results revealed ChatGPT's superiority in Set (A) over Bard. Nevertheless, Bard's advantage emerged in Set (B) due to its capacity to access the internet directly and retrieve answers, a capability that ChatGPT does not possess [ 57 ]. However, through these varied assessments, ChatGPT consistently highlights its exceptional prowess compared to various alternatives in the ever-evolving chatbot technology.

The widespread adoption of chatbots, especially ChatGPT, by millions of students and educators, has sparked extensive discussions regarding its incorporation into the education sector [ 64 ]. Accordingly, many scholars have contributed to the discourse, expressing both optimism and pessimism regarding the incorporation of ChatGPT into education. For example, ChatGPT has been highlighted for its capabilities in enriching the learning and teaching experience through its ability to support different learning approaches, including adaptive learning, personalized learning, and self-directed learning [ 58 , 60 , 91 ]), deliver summative and formative feedback to students and provide real-time responses to questions, increase the accessibility of information [ 22 , 40 , 43 ], foster students’ performance, engagement and motivation [ 14 , 44 , 58 ], and enhance teaching practices [ 17 , 18 , 64 , 74 ].

On the other hand, concerns have been also raised regarding its potential negative effects on learning and teaching. These include the dissemination of false information and references [ 12 , 23 , 61 , 85 ], biased reinforcement [ 47 , 50 ], compromised academic integrity [ 18 , 40 , 66 , 74 ], and the potential decline in students' skills [ 43 , 61 , 64 , 74 ]. As a result, ChatGPT has been banned in multiple countries, including Russia, China, Venezuela, Belarus, and Iran, as well as in various educational institutions in India, Italy, Western Australia, France, and the United States [ 52 , 90 ].

Clearly, the advent of chatbots, especially ChatGPT, has provoked significant controversy due to their potential impact on learning and teaching. This indicates the necessity for further exploration to gain a deeper understanding of this technology and carefully evaluate its potential benefits, limitations, challenges, and threats to education [ 79 ]. Therefore, conducting a systematic literature review will provide valuable insights into the potential prospects and obstacles linked to its incorporation into education. This systematic literature review will primarily focus on ChatGPT, driven by the aforementioned key factors outlined above.

However, the existing literature lacks a systematic literature review of empirical studies. Thus, this systematic literature review aims to address this gap by synthesizing the existing empirical studies conducted on chatbots, particularly ChatGPT, in the field of education, highlighting how ChatGPT has been utilized in educational settings, and identifying any existing gaps. This review may be particularly useful for researchers in the field and educators who are contemplating the integration of ChatGPT or any chatbot into education. The following research questions will guide this study:

What are students' and teachers' initial attempts at utilizing ChatGPT in education?

What are the main findings derived from empirical studies that have incorporated ChatGPT into learning and teaching?

2 Methodology

To conduct this study, the authors followed the essential steps of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA 2020) and Okoli’s [ 54 ] steps for conducting a systematic review. These included identifying the study’s purpose, drafting a protocol, applying a practical screening process, searching the literature, extracting relevant data, evaluating the quality of the included studies, synthesizing the studies, and ultimately writing the review. The subsequent section provides an extensive explanation of how these steps were carried out in this study.

2.1 Identify the purpose

Given the widespread adoption of ChatGPT by students and teachers for various educational purposes, often without a thorough understanding of responsible and effective use or a clear recognition of its potential impact on learning and teaching, the authors recognized the need for further exploration of ChatGPT's impact on education in this early stage. Therefore, they have chosen to conduct a systematic literature review of existing empirical studies that incorporate ChatGPT into educational settings. Despite the limited number of empirical studies due to the novelty of the topic, their goal is to gain a deeper understanding of this technology and proactively evaluate its potential benefits, limitations, challenges, and threats to education. This effort could help to understand initial reactions and attempts at incorporating ChatGPT into education and bring out insights and considerations that can inform the future development of education.

2.2 Draft the protocol

The next step is formulating the protocol. This protocol serves to outline the study process in a rigorous and transparent manner, mitigating researcher bias in study selection and data extraction [ 88 ]. The protocol will include the following steps: generating the research question, predefining a literature search strategy, identifying search locations, establishing selection criteria, assessing the studies, developing a data extraction strategy, and creating a timeline.

2.3 Apply practical screen

The screening step aims to accurately filter the articles resulting from the searching step and select the empirical studies that have incorporated ChatGPT into educational contexts, which will guide us in answering the research questions and achieving the objectives of this study. To ensure the rigorous execution of this step, our inclusion and exclusion criteria were determined based on the authors' experience and informed by previous successful systematic reviews [ 21 ]. Table 1 summarizes the inclusion and exclusion criteria for study selection.

2.4 Literature search

We conducted a thorough literature search to identify articles that explored, examined, and addressed the use of ChatGPT in Educational contexts. We utilized two research databases: Dimensions.ai, which provides access to a large number of research publications, and lens.org, which offers access to over 300 million articles, patents, and other research outputs from diverse sources. Additionally, we included three databases, Scopus, Web of Knowledge, and ERIC, which contain relevant research on the topic that addresses our research questions. To browse and identify relevant articles, we used the following search formula: ("ChatGPT" AND "Education"), which included the Boolean operator "AND" to get more specific results. The subject area in the Scopus and ERIC databases were narrowed to "ChatGPT" and "Education" keywords, and in the WoS database was limited to the "Education" category. The search was conducted between the 3rd and 10th of April 2023, which resulted in 276 articles from all selected databases (111 articles from Dimensions.ai, 65 from Scopus, 28 from Web of Science, 14 from ERIC, and 58 from Lens.org). These articles were imported into the Rayyan web-based system for analysis. The duplicates were identified automatically by the system. Subsequently, the first author manually reviewed the duplicated articles ensured that they had the same content, and then removed them, leaving us with 135 unique articles. Afterward, the titles, abstracts, and keywords of the first 40 manuscripts were scanned and reviewed by the first author and were discussed with the second and third authors to resolve any disagreements. Subsequently, the first author proceeded with the filtering process for all articles and carefully applied the inclusion and exclusion criteria as presented in Table  1 . Articles that met any one of the exclusion criteria were eliminated, resulting in 26 articles. Afterward, the authors met to carefully scan and discuss them. The authors agreed to eliminate any empirical studies solely focused on checking ChatGPT capabilities, as these studies do not guide us in addressing the research questions and achieving the study's objectives. This resulted in 14 articles eligible for analysis.

2.5 Quality appraisal

The examination and evaluation of the quality of the extracted articles is a vital step [ 9 ]. Therefore, the extracted articles were carefully evaluated for quality using Fink’s [ 24 ] standards, which emphasize the necessity for detailed descriptions of methodology, results, conclusions, strengths, and limitations. The process began with a thorough assessment of each study's design, data collection, and analysis methods to ensure their appropriateness and comprehensive execution. The clarity, consistency, and logical progression from data to results and conclusions were also critically examined. Potential biases and recognized limitations within the studies were also scrutinized. Ultimately, two articles were excluded for failing to meet Fink’s criteria, particularly in providing sufficient detail on methodology, results, conclusions, strengths, or limitations. The review process is illustrated in Fig.  1 .

The study selection process

2.6 Data extraction

The next step is data extraction, the process of capturing the key information and categories from the included studies. To improve efficiency, reduce variation among authors, and minimize errors in data analysis, the coding categories were constructed using Creswell's [ 15 ] coding techniques for data extraction and interpretation. The coding process involves three sequential steps. The initial stage encompasses open coding , where the researcher examines the data, generates codes to describe and categorize it, and gains a deeper understanding without preconceived ideas. Following open coding is axial coding , where the interrelationships between codes from open coding are analyzed to establish more comprehensive categories or themes. The process concludes with selective coding , refining and integrating categories or themes to identify core concepts emerging from the data. The first coder performed the coding process, then engaged in discussions with the second and third authors to finalize the coding categories for the first five articles. The first coder then proceeded to code all studies and engaged again in discussions with the other authors to ensure the finalization of the coding process. After a comprehensive analysis and capturing of the key information from the included studies, the data extraction and interpretation process yielded several themes. These themes have been categorized and are presented in Table  2 . It is important to note that open coding results were removed from Table  2 for aesthetic reasons, as it included many generic aspects, such as words, short phrases, or sentences mentioned in the studies.

2.7 Synthesize studies

In this stage, we will gather, discuss, and analyze the key findings that emerged from the selected studies. The synthesis stage is considered a transition from an author-centric to a concept-centric focus, enabling us to map all the provided information to achieve the most effective evaluation of the data [ 87 ]. Initially, the authors extracted data that included general information about the selected studies, including the author(s)' names, study titles, years of publication, educational levels, research methodologies, sample sizes, participants, main aims or objectives, raw data sources, and analysis methods. Following that, all key information and significant results from the selected studies were compiled using Creswell’s [ 15 ] coding techniques for data extraction and interpretation to identify core concepts and themes emerging from the data, focusing on those that directly contributed to our research questions and objectives, such as the initial utilization of ChatGPT in learning and teaching, learners' and educators' familiarity with ChatGPT, and the main findings of each study. Finally, the data related to each selected study were extracted into an Excel spreadsheet for data processing. The Excel spreadsheet was reviewed by the authors, including a series of discussions to ensure the finalization of this process and prepare it for further analysis. Afterward, the final result being analyzed and presented in various types of charts and graphs. Table 4 presents the extracted data from the selected studies, with each study labeled with a capital 'S' followed by a number.

This section consists of two main parts. The first part provides a descriptive analysis of the data compiled from the reviewed studies. The second part presents the answers to the research questions and the main findings of these studies.

3.1 Part 1: descriptive analysis

This section will provide a descriptive analysis of the reviewed studies, including educational levels and fields, participants distribution, country contribution, research methodologies, study sample size, study population, publication year, list of journals, familiarity with ChatGPT, source of data, and the main aims and objectives of the studies. Table 4 presents a comprehensive overview of the extracted data from the selected studies.

3.1.1 The number of the reviewed studies and publication years

The total number of the reviewed studies was 14. All studies were empirical studies and published in different journals focusing on Education and Technology. One study was published in 2022 [S1], while the remaining were published in 2023 [S2]-[S14]. Table 3 illustrates the year of publication, the names of the journals, and the number of reviewed studies published in each journal for the studies reviewed.

3.1.2 Educational levels and fields

The majority of the reviewed studies, 11 studies, were conducted in higher education institutions [S1]-[S10] and [S13]. Two studies did not specify the educational level of the population [S12] and [S14], while one study focused on elementary education [S11]. However, the reviewed studies covered various fields of education. Three studies focused on Arts and Humanities Education [S8], [S11], and [S14], specifically English Education. Two studies focused on Engineering Education, with one in Computer Engineering [S2] and the other in Construction Education [S3]. Two studies focused on Mathematics Education [S5] and [S12]. One study focused on Social Science Education [S13]. One study focused on Early Education [S4]. One study focused on Journalism Education [S9]. Finally, three studies did not specify the field of education [S1], [S6], and [S7]. Figure  2 represents the educational levels in the reviewed studies, while Fig.  3 represents the context of the reviewed studies.

Educational levels in the reviewed studies

Context of the reviewed studies

3.1.3 Participants distribution and countries contribution

The reviewed studies have been conducted across different geographic regions, providing a diverse representation of the studies. The majority of the studies, 10 in total, [S1]-[S3], [S5]-[S9], [S11], and [S14], primarily focused on participants from single countries such as Pakistan, the United Arab Emirates, China, Indonesia, Poland, Saudi Arabia, South Korea, Spain, Tajikistan, and the United States. In contrast, four studies, [S4], [S10], [S12], and [S13], involved participants from multiple countries, including China and the United States [S4], China, the United Kingdom, and the United States [S10], the United Arab Emirates, Oman, Saudi Arabia, and Jordan [S12], Turkey, Sweden, Canada, and Australia [ 13 ]. Figures  4 and 5 illustrate the distribution of participants, whether from single or multiple countries, and the contribution of each country in the reviewed studies, respectively.

The reviewed studies conducted in single or multiple countries

The Contribution of each country in the studies

3.1.4 Study population and sample size

Four study populations were included: university students, university teachers, university teachers and students, and elementary school teachers. Six studies involved university students [S2], [S3], [S5] and [S6]-[S8]. Three studies focused on university teachers [S1], [S4], and [S6], while one study specifically targeted elementary school teachers [S11]. Additionally, four studies included both university teachers and students [S10] and [ 12 , 13 , 14 ], and among them, study [S13] specifically included postgraduate students. In terms of the sample size of the reviewed studies, nine studies included a small sample size of less than 50 participants [S1], [S3], [S6], [S8], and [S10]-[S13]. Three studies had 50–100 participants [S2], [S9], and [S14]. Only one study had more than 100 participants [S7]. It is worth mentioning that study [S4] adopted a mixed methods approach, including 10 participants for qualitative analysis and 110 participants for quantitative analysis.

3.1.5 Participants’ familiarity with using ChatGPT

The reviewed studies recruited a diverse range of participants with varying levels of familiarity with ChatGPT. Five studies [S2], [S4], [S6], [S8], and [S12] involved participants already familiar with ChatGPT, while eight studies [S1], [S3], [S5], [S7], [S9], [S10], [S13] and [S14] included individuals with differing levels of familiarity. Notably, one study [S11] had participants who were entirely unfamiliar with ChatGPT. It is important to note that four studies [S3], [S5], [S9], and [S11] provided training or guidance to their participants before conducting their studies, while ten studies [S1], [S2], [S4], [S6]-[S8], [S10], and [S12]-[S14] did not provide training due to the participants' existing familiarity with ChatGPT.

3.1.6 Research methodology approaches and source(S) of data

The reviewed studies adopted various research methodology approaches. Seven studies adopted qualitative research methodology [S1], [S4], [S6], [S8], [S10], [S11], and [S12], while three studies adopted quantitative research methodology [S3], [S7], and [S14], and four studies employed mixed-methods, which involved a combination of both the strengths of qualitative and quantitative methods [S2], [S5], [S9], and [S13].

In terms of the source(s) of data, the reviewed studies obtained their data from various sources, such as interviews, questionnaires, and pre-and post-tests. Six studies relied on interviews as their primary source of data collection [S1], [S4], [S6], [S10], [S11], and [S12], four studies relied on questionnaires [S2], [S7], [S13], and [S14], two studies combined the use of pre-and post-tests and questionnaires for data collection [S3] and [S9], while two studies combined the use of questionnaires and interviews to obtain the data [S5] and [S8]. It is important to note that six of the reviewed studies were quasi-experimental [S3], [S5], [S8], [S9], [S12], and [S14], while the remaining ones were experimental studies [S1], [S2], [S4], [S6], [S7], [S10], [S11], and [S13]. Figures  6 and 7 illustrate the research methodologies and the source (s) of data used in the reviewed studies, respectively.

Research methodologies in the reviewed studies

Source of data in the reviewed studies

3.1.7 The aim and objectives of the studies

The reviewed studies encompassed a diverse set of aims, with several of them incorporating multiple primary objectives. Six studies [S3], [S6], [S7], [S8], [S11], and [S12] examined the integration of ChatGPT in educational contexts, and four studies [S4], [S5], [S13], and [S14] investigated the various implications of its use in education, while three studies [S2], [S9], and [S10] aimed to explore both its integration and implications in education. Additionally, seven studies explicitly explored attitudes and perceptions of students [S2] and [S3], educators [S1] and [S6], or both [S10], [S12], and [S13] regarding the utilization of ChatGPT in educational settings.

3.2 Part 2: research questions and main findings of the reviewed studies

This part will present the answers to the research questions and the main findings of the reviewed studies, classified into two main categories (learning and teaching) according to AI Education classification by [ 36 ]. Figure  8 summarizes the main findings of the reviewed studies in a visually informative diagram. Table 4 provides a detailed list of the key information extracted from the selected studies that led to generating these themes.

The main findings in the reviewed studies

4 Students' initial attempts at utilizing ChatGPT in learning and main findings from students' perspective

4.1 virtual intelligent assistant.

Nine studies demonstrated that ChatGPT has been utilized by students as an intelligent assistant to enhance and support their learning. Students employed it for various purposes, such as answering on-demand questions [S2]-[S5], [S8], [S10], and [S12], providing valuable information and learning resources [S2]-[S5], [S6], and [S8], as well as receiving immediate feedback [S2], [S4], [S9], [S10], and [S12]. In this regard, students generally were confident in the accuracy of ChatGPT's responses, considering them relevant, reliable, and detailed [S3], [S4], [S5], and [S8]. However, some students indicated the need for improvement, as they found that answers are not always accurate [S2], and that misleading information may have been provided or that it may not always align with their expectations [S6] and [S10]. It was also observed by the students that the accuracy of ChatGPT is dependent on several factors, including the quality and specificity of the user's input, the complexity of the question or topic, and the scope and relevance of its training data [S12]. Many students felt that ChatGPT's answers were not always accurate and most of them believed that it requires good background knowledge to work with.

4.2 Writing and language proficiency assistant

Six of the reviewed studies highlighted that ChatGPT has been utilized by students as a valuable assistant tool to improve their academic writing skills and language proficiency. Among these studies, three mainly focused on English education, demonstrating that students showed sufficient mastery in using ChatGPT for generating ideas, summarizing, paraphrasing texts, and completing writing essays [S8], [S11], and [S14]. Furthermore, ChatGPT helped them in writing by making students active investigators rather than passive knowledge recipients and facilitated the development of their writing skills [S11] and [S14]. Similarly, ChatGPT allowed students to generate unique ideas and perspectives, leading to deeper analysis and reflection on their journalism writing [S9]. In terms of language proficiency, ChatGPT allowed participants to translate content into their home languages, making it more accessible and relevant to their context [S4]. It also enabled them to request changes in linguistic tones or flavors [S8]. Moreover, participants used it to check grammar or as a dictionary [S11].

4.3 Valuable resource for learning approaches

Five studies demonstrated that students used ChatGPT as a valuable complementary resource for self-directed learning. It provided learning resources and guidance on diverse educational topics and created a supportive home learning environment [S2] and [S4]. Moreover, it offered step-by-step guidance to grasp concepts at their own pace and enhance their understanding [S5], streamlined task and project completion carried out independently [S7], provided comprehensive and easy-to-understand explanations on various subjects [S10], and assisted in studying geometry operations, thereby empowering them to explore geometry operations at their own pace [S12]. Three studies showed that students used ChatGPT as a valuable learning resource for personalized learning. It delivered age-appropriate conversations and tailored teaching based on a child's interests [S4], acted as a personalized learning assistant, adapted to their needs and pace, which assisted them in understanding mathematical concepts [S12], and enabled personalized learning experiences in social sciences by adapting to students' needs and learning styles [S13]. On the other hand, it is important to note that, according to one study [S5], students suggested that using ChatGPT may negatively affect collaborative learning competencies between students.

4.4 Enhancing students' competencies

Six of the reviewed studies have shown that ChatGPT is a valuable tool for improving a wide range of skills among students. Two studies have provided evidence that ChatGPT led to improvements in students' critical thinking, reasoning skills, and hazard recognition competencies through engaging them in interactive conversations or activities and providing responses related to their disciplines in journalism [S5] and construction education [S9]. Furthermore, two studies focused on mathematical education have shown the positive impact of ChatGPT on students' problem-solving abilities in unraveling problem-solving questions [S12] and enhancing the students' understanding of the problem-solving process [S5]. Lastly, one study indicated that ChatGPT effectively contributed to the enhancement of conversational social skills [S4].

4.5 Supporting students' academic success

Seven of the reviewed studies highlighted that students found ChatGPT to be beneficial for learning as it enhanced learning efficiency and improved the learning experience. It has been observed to improve students' efficiency in computer engineering studies by providing well-structured responses and good explanations [S2]. Additionally, students found it extremely useful for hazard reporting [S3], and it also enhanced their efficiency in solving mathematics problems and capabilities [S5] and [S12]. Furthermore, by finding information, generating ideas, translating texts, and providing alternative questions, ChatGPT aided students in deepening their understanding of various subjects [S6]. It contributed to an increase in students' overall productivity [S7] and improved efficiency in composing written tasks [S8]. Regarding learning experiences, ChatGPT was instrumental in assisting students in identifying hazards that they might have otherwise overlooked [S3]. It also improved students' learning experiences in solving mathematics problems and developing abilities [S5] and [S12]. Moreover, it increased students' successful completion of important tasks in their studies [S7], particularly those involving average difficulty writing tasks [S8]. Additionally, ChatGPT increased the chances of educational success by providing students with baseline knowledge on various topics [S10].

5 Teachers' initial attempts at utilizing ChatGPT in teaching and main findings from teachers' perspective

5.1 valuable resource for teaching.

The reviewed studies showed that teachers have employed ChatGPT to recommend, modify, and generate diverse, creative, organized, and engaging educational contents, teaching materials, and testing resources more rapidly [S4], [S6], [S10] and [S11]. Additionally, teachers experienced increased productivity as ChatGPT facilitated quick and accurate responses to questions, fact-checking, and information searches [S1]. It also proved valuable in constructing new knowledge [S6] and providing timely answers to students' questions in classrooms [S11]. Moreover, ChatGPT enhanced teachers' efficiency by generating new ideas for activities and preplanning activities for their students [S4] and [S6], including interactive language game partners [S11].

5.2 Improving productivity and efficiency

The reviewed studies showed that participants' productivity and work efficiency have been significantly enhanced by using ChatGPT as it enabled them to allocate more time to other tasks and reduce their overall workloads [S6], [S10], [S11], [S13], and [S14]. However, three studies [S1], [S4], and [S11], indicated a negative perception and attitude among teachers toward using ChatGPT. This negativity stemmed from a lack of necessary skills to use it effectively [S1], a limited familiarity with it [S4], and occasional inaccuracies in the content provided by it [S10].

5.3 Catalyzing new teaching methodologies

Five of the reviewed studies highlighted that educators found the necessity of redefining their teaching profession with the assistance of ChatGPT [S11], developing new effective learning strategies [S4], and adapting teaching strategies and methodologies to ensure the development of essential skills for future engineers [S5]. They also emphasized the importance of adopting new educational philosophies and approaches that can evolve with the introduction of ChatGPT into the classroom [S12]. Furthermore, updating curricula to focus on improving human-specific features, such as emotional intelligence, creativity, and philosophical perspectives [S13], was found to be essential.

5.4 Effective utilization of CHATGPT in teaching

According to the reviewed studies, effective utilization of ChatGPT in education requires providing teachers with well-structured training, support, and adequate background on how to use ChatGPT responsibly [S1], [S3], [S11], and [S12]. Establishing clear rules and regulations regarding its usage is essential to ensure it positively impacts the teaching and learning processes, including students' skills [S1], [S4], [S5], [S8], [S9], and [S11]-[S14]. Moreover, conducting further research and engaging in discussions with policymakers and stakeholders is indeed crucial for the successful integration of ChatGPT in education and to maximize the benefits for both educators and students [S1], [S6]-[S10], and [S12]-[S14].

6 Discussion

The purpose of this review is to conduct a systematic review of empirical studies that have explored the utilization of ChatGPT, one of today’s most advanced LLM-based chatbots, in education. The findings of the reviewed studies showed several ways of ChatGPT utilization in different learning and teaching practices as well as it provided insights and considerations that can facilitate its effective and responsible use in future educational contexts. The results of the reviewed studies came from diverse fields of education, which helped us avoid a biased review that is limited to a specific field. Similarly, the reviewed studies have been conducted across different geographic regions. This kind of variety in geographic representation enriched the findings of this review.

In response to RQ1 , "What are students' and teachers' initial attempts at utilizing ChatGPT in education?", the findings from this review provide comprehensive insights. Chatbots, including ChatGPT, play a crucial role in supporting student learning, enhancing their learning experiences, and facilitating diverse learning approaches [ 42 , 43 ]. This review found that this tool, ChatGPT, has been instrumental in enhancing students' learning experiences by serving as a virtual intelligent assistant, providing immediate feedback, on-demand answers, and engaging in educational conversations. Additionally, students have benefited from ChatGPT’s ability to generate ideas, compose essays, and perform tasks like summarizing, translating, paraphrasing texts, or checking grammar, thereby enhancing their writing and language competencies. Furthermore, students have turned to ChatGPT for assistance in understanding concepts and homework, providing structured learning plans, and clarifying assignments and tasks, which fosters a supportive home learning environment, allowing them to take responsibility for their own learning and cultivate the skills and approaches essential for supportive home learning environment [ 26 , 27 , 28 ]. This finding aligns with the study of Saqr et al. [ 68 , 69 ] who highlighted that, when students actively engage in their own learning process, it yields additional advantages, such as heightened motivation, enhanced achievement, and the cultivation of enthusiasm, turning them into advocates for their own learning.

Moreover, students have utilized ChatGPT for tailored teaching and step-by-step guidance on diverse educational topics, streamlining task and project completion, and generating and recommending educational content. This personalization enhances the learning environment, leading to increased academic success. This finding aligns with other recent studies [ 26 , 27 , 28 , 60 , 66 ] which revealed that ChatGPT has the potential to offer personalized learning experiences and support an effective learning process by providing students with customized feedback and explanations tailored to their needs and abilities. Ultimately, fostering students' performance, engagement, and motivation, leading to increase students' academic success [ 14 , 44 , 58 ]. This ultimate outcome is in line with the findings of Saqr et al. [ 68 , 69 ], which emphasized that learning strategies are important catalysts of students' learning, as students who utilize effective learning strategies are more likely to have better academic achievement.

Teachers, too, have capitalized on ChatGPT's capabilities to enhance productivity and efficiency, using it for creating lesson plans, generating quizzes, providing additional resources, generating and preplanning new ideas for activities, and aiding in answering students’ questions. This adoption of technology introduces new opportunities to support teaching and learning practices, enhancing teacher productivity. This finding aligns with those of Day [ 17 ], De Castro [ 18 ], and Su and Yang [ 74 ] as well as with those of Valtonen et al. [ 82 ], who revealed that emerging technological advancements have opened up novel opportunities and means to support teaching and learning practices, and enhance teachers’ productivity.

In response to RQ2 , "What are the main findings derived from empirical studies that have incorporated ChatGPT into learning and teaching?", the findings from this review provide profound insights and raise significant concerns. Starting with the insights, chatbots, including ChatGPT, have demonstrated the potential to reshape and revolutionize education, creating new, novel opportunities for enhancing the learning process and outcomes [ 83 ], facilitating different learning approaches, and offering a range of pedagogical benefits [ 19 , 43 , 72 ]. In this context, this review found that ChatGPT could open avenues for educators to adopt or develop new effective learning and teaching strategies that can evolve with the introduction of ChatGPT into the classroom. Nonetheless, there is an evident lack of research understanding regarding the potential impact of generative machine learning models within diverse educational settings [ 83 ]. This necessitates teachers to attain a high level of proficiency in incorporating chatbots, such as ChatGPT, into their classrooms to create inventive, well-structured, and captivating learning strategies. In the same vein, the review also found that teachers without the requisite skills to utilize ChatGPT realized that it did not contribute positively to their work and could potentially have adverse effects [ 37 ]. This concern could lead to inequity of access to the benefits of chatbots, including ChatGPT, as individuals who lack the necessary expertise may not be able to harness their full potential, resulting in disparities in educational outcomes and opportunities. Therefore, immediate action is needed to address these potential issues. A potential solution is offering training, support, and competency development for teachers to ensure that all of them can leverage chatbots, including ChatGPT, effectively and equitably in their educational practices [ 5 , 28 , 80 ], which could enhance accessibility and inclusivity, and potentially result in innovative outcomes [ 82 , 83 ].

Additionally, chatbots, including ChatGPT, have the potential to significantly impact students' thinking abilities, including retention, reasoning, analysis skills [ 19 , 45 ], and foster innovation and creativity capabilities [ 83 ]. This review found that ChatGPT could contribute to improving a wide range of skills among students. However, it found that frequent use of ChatGPT may result in a decrease in innovative capacities, collaborative skills and cognitive capacities, and students' motivation to attend classes, as well as could lead to reduced higher-order thinking skills among students [ 22 , 29 ]. Therefore, immediate action is needed to carefully examine the long-term impact of chatbots such as ChatGPT, on learning outcomes as well as to explore its incorporation into educational settings as a supportive tool without compromising students' cognitive development and critical thinking abilities. In the same vein, the review also found that it is challenging to draw a consistent conclusion regarding the potential of ChatGPT to aid self-directed learning approach. This finding aligns with the recent study of Baskara [ 8 ]. Therefore, further research is needed to explore the potential of ChatGPT for self-directed learning. One potential solution involves utilizing learning analytics as a novel approach to examine various aspects of students' learning and support them in their individual endeavors [ 32 ]. This approach can bridge this gap by facilitating an in-depth analysis of how learners engage with ChatGPT, identifying trends in self-directed learning behavior, and assessing its influence on their outcomes.

Turning to the significant concerns, on the other hand, a fundamental challenge with LLM-based chatbots, including ChatGPT, is the accuracy and quality of the provided information and responses, as they provide false information as truth—a phenomenon often referred to as "hallucination" [ 3 , 49 ]. In this context, this review found that the provided information was not entirely satisfactory. Consequently, the utilization of chatbots presents potential concerns, such as generating and providing inaccurate or misleading information, especially for students who utilize it to support their learning. This finding aligns with other findings [ 6 , 30 , 35 , 40 ] which revealed that incorporating chatbots such as ChatGPT, into education presents challenges related to its accuracy and reliability due to its training on a large corpus of data, which may contain inaccuracies and the way users formulate or ask ChatGPT. Therefore, immediate action is needed to address these potential issues. One possible solution is to equip students with the necessary skills and competencies, which include a background understanding of how to use it effectively and the ability to assess and evaluate the information it generates, as the accuracy and the quality of the provided information depend on the input, its complexity, the topic, and the relevance of its training data [ 28 , 49 , 86 ]. However, it's also essential to examine how learners can be educated about how these models operate, the data used in their training, and how to recognize their limitations, challenges, and issues [ 79 ].

Furthermore, chatbots present a substantial challenge concerning maintaining academic integrity [ 20 , 56 ] and copyright violations [ 83 ], which are significant concerns in education. The review found that the potential misuse of ChatGPT might foster cheating, facilitate plagiarism, and threaten academic integrity. This issue is also affirmed by the research conducted by Basic et al. [ 7 ], who presented evidence that students who utilized ChatGPT in their writing assignments had more plagiarism cases than those who did not. These findings align with the conclusions drawn by Cotton et al. [ 13 ], Hisan and Amri [ 33 ] and Sullivan et al. [ 75 ], who revealed that the integration of chatbots such as ChatGPT into education poses a significant challenge to the preservation of academic integrity. Moreover, chatbots, including ChatGPT, have increased the difficulty in identifying plagiarism [ 47 , 67 , 76 ]. The findings from previous studies [ 1 , 84 ] indicate that AI-generated text often went undetected by plagiarism software, such as Turnitin. However, Turnitin and other similar plagiarism detection tools, such as ZeroGPT, GPTZero, and Copyleaks, have since evolved, incorporating enhanced techniques to detect AI-generated text, despite the possibility of false positives, as noted in different studies that have found these tools still not yet fully ready to accurately and reliably identify AI-generated text [ 10 , 51 ], and new novel detection methods may need to be created and implemented for AI-generated text detection [ 4 ]. This potential issue could lead to another concern, which is the difficulty of accurately evaluating student performance when they utilize chatbots such as ChatGPT assistance in their assignments. Consequently, the most LLM-driven chatbots present a substantial challenge to traditional assessments [ 64 ]. The findings from previous studies indicate the importance of rethinking, improving, and redesigning innovative assessment methods in the era of chatbots [ 14 , 20 , 64 , 75 ]. These methods should prioritize the process of evaluating students' ability to apply knowledge to complex cases and demonstrate comprehension, rather than solely focusing on the final product for assessment. Therefore, immediate action is needed to address these potential issues. One possible solution would be the development of clear guidelines, regulatory policies, and pedagogical guidance. These measures would help regulate the proper and ethical utilization of chatbots, such as ChatGPT, and must be established before their introduction to students [ 35 , 38 , 39 , 41 , 89 ].

In summary, our review has delved into the utilization of ChatGPT, a prominent example of chatbots, in education, addressing the question of how ChatGPT has been utilized in education. However, there remain significant gaps, which necessitate further research to shed light on this area.

7 Conclusions

This systematic review has shed light on the varied initial attempts at incorporating ChatGPT into education by both learners and educators, while also offering insights and considerations that can facilitate its effective and responsible use in future educational contexts. From the analysis of 14 selected studies, the review revealed the dual-edged impact of ChatGPT in educational settings. On the positive side, ChatGPT significantly aided the learning process in various ways. Learners have used it as a virtual intelligent assistant, benefiting from its ability to provide immediate feedback, on-demand answers, and easy access to educational resources. Additionally, it was clear that learners have used it to enhance their writing and language skills, engaging in practices such as generating ideas, composing essays, and performing tasks like summarizing, translating, paraphrasing texts, or checking grammar. Importantly, other learners have utilized it in supporting and facilitating their directed and personalized learning on a broad range of educational topics, assisting in understanding concepts and homework, providing structured learning plans, and clarifying assignments and tasks. Educators, on the other hand, found ChatGPT beneficial for enhancing productivity and efficiency. They used it for creating lesson plans, generating quizzes, providing additional resources, and answers learners' questions, which saved time and allowed for more dynamic and engaging teaching strategies and methodologies.

However, the review also pointed out negative impacts. The results revealed that overuse of ChatGPT could decrease innovative capacities and collaborative learning among learners. Specifically, relying too much on ChatGPT for quick answers can inhibit learners' critical thinking and problem-solving skills. Learners might not engage deeply with the material or consider multiple solutions to a problem. This tendency was particularly evident in group projects, where learners preferred consulting ChatGPT individually for solutions over brainstorming and collaborating with peers, which negatively affected their teamwork abilities. On a broader level, integrating ChatGPT into education has also raised several concerns, including the potential for providing inaccurate or misleading information, issues of inequity in access, challenges related to academic integrity, and the possibility of misusing the technology.

Accordingly, this review emphasizes the urgency of developing clear rules, policies, and regulations to ensure ChatGPT's effective and responsible use in educational settings, alongside other chatbots, by both learners and educators. This requires providing well-structured training to educate them on responsible usage and understanding its limitations, along with offering sufficient background information. Moreover, it highlights the importance of rethinking, improving, and redesigning innovative teaching and assessment methods in the era of ChatGPT. Furthermore, conducting further research and engaging in discussions with policymakers and stakeholders are essential steps to maximize the benefits for both educators and learners and ensure academic integrity.

It is important to acknowledge that this review has certain limitations. Firstly, the limited inclusion of reviewed studies can be attributed to several reasons, including the novelty of the technology, as new technologies often face initial skepticism and cautious adoption; the lack of clear guidelines or best practices for leveraging this technology for educational purposes; and institutional or governmental policies affecting the utilization of this technology in educational contexts. These factors, in turn, have affected the number of studies available for review. Secondly, the utilization of the original version of ChatGPT, based on GPT-3 or GPT-3.5, implies that new studies utilizing the updated version, GPT-4 may lead to different findings. Therefore, conducting follow-up systematic reviews is essential once more empirical studies on ChatGPT are published. Additionally, long-term studies are necessary to thoroughly examine and assess the impact of ChatGPT on various educational practices.

Despite these limitations, this systematic review has highlighted the transformative potential of ChatGPT in education, revealing its diverse utilization by learners and educators alike and summarized the benefits of incorporating it into education, as well as the forefront critical concerns and challenges that must be addressed to facilitate its effective and responsible use in future educational contexts. This review could serve as an insightful resource for practitioners who seek to integrate ChatGPT into education and stimulate further research in the field.

Data availability

The data supporting our findings are available upon request.

Abbreviations

• Artificial intelligence

AI in education

Large language model

Artificial neural networks

Chat Generative Pre-Trained Transformer

Recurrent neural networks

Long short-term memory

Reinforcement learning from human feedback

Natural language processing

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

AlAfnan MA, Dishari S, Jovic M, Lomidze K. ChatGPT as an educational tool: opportunities, challenges, and recommendations for communication, business writing, and composition courses. J Artif Intell Technol. 2023. https://doi.org/10.37965/jait.2023.0184 .

Article   Google Scholar  

Ali JKM, Shamsan MAA, Hezam TA, Mohammed AAQ. Impact of ChatGPT on learning motivation. J Engl Stud Arabia Felix. 2023;2(1):41–9. https://doi.org/10.56540/jesaf.v2i1.51 .

Alkaissi H, McFarlane SI. Artificial hallucinations in ChatGPT: implications in scientific writing. Cureus. 2023. https://doi.org/10.7759/cureus.35179 .

Anderson N, Belavý DL, Perle SM, Hendricks S, Hespanhol L, Verhagen E, Memon AR. AI did not write this manuscript, or did it? Can we trick the AI text detector into generated texts? The potential future of ChatGPT and AI in sports & exercise medicine manuscript generation. BMJ Open Sport Exerc Med. 2023;9(1): e001568. https://doi.org/10.1136/bmjsem-2023-001568 .

Ausat AMA, Massang B, Efendi M, Nofirman N, Riady Y. Can chat GPT replace the role of the teacher in the classroom: a fundamental analysis. J Educ. 2023;5(4):16100–6.

Google Scholar  

Baidoo-Anu D, Ansah L. Education in the Era of generative artificial intelligence (AI): understanding the potential benefits of ChatGPT in promoting teaching and learning. Soc Sci Res Netw. 2023. https://doi.org/10.2139/ssrn.4337484 .

Basic Z, Banovac A, Kruzic I, Jerkovic I. Better by you, better than me, chatgpt3 as writing assistance in students essays. 2023. arXiv preprint arXiv:2302.04536 .‏

Baskara FR. The promises and pitfalls of using chat GPT for self-determined learning in higher education: an argumentative review. Prosiding Seminar Nasional Fakultas Tarbiyah dan Ilmu Keguruan IAIM Sinjai. 2023;2:95–101. https://doi.org/10.47435/sentikjar.v2i0.1825 .

Behera RK, Bala PK, Dhir A. The emerging role of cognitive computing in healthcare: a systematic literature review. Int J Med Inform. 2019;129:154–66. https://doi.org/10.1016/j.ijmedinf.2019.04.024 .

Chaka C. Detecting AI content in responses generated by ChatGPT, YouChat, and Chatsonic: the case of five AI content detection tools. J Appl Learn Teach. 2023. https://doi.org/10.37074/jalt.2023.6.2.12 .

Chiu TKF, Xia Q, Zhou X, Chai CS, Cheng M. Systematic literature review on opportunities, challenges, and future research recommendations of artificial intelligence in education. Comput Educ Artif Intell. 2023;4:100118. https://doi.org/10.1016/j.caeai.2022.100118 .

Choi EPH, Lee JJ, Ho M, Kwok JYY, Lok KYW. Chatting or cheating? The impacts of ChatGPT and other artificial intelligence language models on nurse education. Nurse Educ Today. 2023;125:105796. https://doi.org/10.1016/j.nedt.2023.105796 .

Cotton D, Cotton PA, Shipway JR. Chatting and cheating: ensuring academic integrity in the era of ChatGPT. Innov Educ Teach Int. 2023. https://doi.org/10.1080/14703297.2023.2190148 .

Crawford J, Cowling M, Allen K. Leadership is needed for ethical ChatGPT: Character, assessment, and learning using artificial intelligence (AI). J Univ Teach Learn Pract. 2023. https://doi.org/10.53761/1.20.3.02 .

Creswell JW. Educational research: planning, conducting, and evaluating quantitative and qualitative research [Ebook]. 4th ed. London: Pearson Education; 2015.

Curry D. ChatGPT Revenue and Usage Statistics (2023)—Business of Apps. 2023. https://www.businessofapps.com/data/chatgpt-statistics/

Day T. A preliminary investigation of fake peer-reviewed citations and references generated by ChatGPT. Prof Geogr. 2023. https://doi.org/10.1080/00330124.2023.2190373 .

De Castro CA. A Discussion about the Impact of ChatGPT in education: benefits and concerns. J Bus Theor Pract. 2023;11(2):p28. https://doi.org/10.22158/jbtp.v11n2p28 .

Deng X, Yu Z. A meta-analysis and systematic review of the effect of Chatbot technology use in sustainable education. Sustainability. 2023;15(4):2940. https://doi.org/10.3390/su15042940 .

Eke DO. ChatGPT and the rise of generative AI: threat to academic integrity? J Responsib Technol. 2023;13:100060. https://doi.org/10.1016/j.jrt.2023.100060 .

Elmoazen R, Saqr M, Tedre M, Hirsto L. A systematic literature review of empirical research on epistemic network analysis in education. IEEE Access. 2022;10:17330–48. https://doi.org/10.1109/access.2022.3149812 .

Farrokhnia M, Banihashem SK, Noroozi O, Wals AEJ. A SWOT analysis of ChatGPT: implications for educational practice and research. Innov Educ Teach Int. 2023. https://doi.org/10.1080/14703297.2023.2195846 .

Fergus S, Botha M, Ostovar M. Evaluating academic answers generated using ChatGPT. J Chem Educ. 2023;100(4):1672–5. https://doi.org/10.1021/acs.jchemed.3c00087 .

Fink A. Conducting research literature reviews: from the Internet to Paper. Incorporated: SAGE Publications; 2010.

Firaina R, Sulisworo D. Exploring the usage of ChatGPT in higher education: frequency and impact on productivity. Buletin Edukasi Indonesia (BEI). 2023;2(01):39–46. https://doi.org/10.56741/bei.v2i01.310 .

Firat, M. (2023). How chat GPT can transform autodidactic experiences and open education.  Department of Distance Education, Open Education Faculty, Anadolu Unive .‏ https://orcid.org/0000-0001-8707-5918

Firat M. What ChatGPT means for universities: perceptions of scholars and students. J Appl Learn Teach. 2023. https://doi.org/10.37074/jalt.2023.6.1.22 .

Fuchs K. Exploring the opportunities and challenges of NLP models in higher education: is Chat GPT a blessing or a curse? Front Educ. 2023. https://doi.org/10.3389/feduc.2023.1166682 .

García-Peñalvo FJ. La percepción de la inteligencia artificial en contextos educativos tras el lanzamiento de ChatGPT: disrupción o pánico. Educ Knowl Soc. 2023;24: e31279. https://doi.org/10.14201/eks.31279 .

Gilson A, Safranek CW, Huang T, Socrates V, Chi L, Taylor A, Chartash D. How does ChatGPT perform on the United States medical Licensing examination? The implications of large language models for medical education and knowledge assessment. JMIR Med Educ. 2023;9: e45312. https://doi.org/10.2196/45312 .

Hashana AJ, Brundha P, Ayoobkhan MUA, Fazila S. Deep Learning in ChatGPT—A Survey. In   2023 7th international conference on trends in electronics and informatics (ICOEI) . 2023. (pp. 1001–1005). IEEE. https://doi.org/10.1109/icoei56765.2023.10125852

Hirsto L, Saqr M, López-Pernas S, Valtonen T. (2022). A systematic narrative review of learning analytics research in K-12 and schools.  Proceedings . https://ceur-ws.org/Vol-3383/FLAIEC22_paper_9536.pdf

Hisan UK, Amri MM. ChatGPT and medical education: a double-edged sword. J Pedag Educ Sci. 2023;2(01):71–89. https://doi.org/10.13140/RG.2.2.31280.23043/1 .

Hopkins AM, Logan JM, Kichenadasse G, Sorich MJ. Artificial intelligence chatbots will revolutionize how cancer patients access information: ChatGPT represents a paradigm-shift. JNCI Cancer Spectr. 2023. https://doi.org/10.1093/jncics/pkad010 .

Househ M, AlSaad R, Alhuwail D, Ahmed A, Healy MG, Latifi S, Sheikh J. Large Language models in medical education: opportunities, challenges, and future directions. JMIR Med Educ. 2023;9: e48291. https://doi.org/10.2196/48291 .

Ilkka T. The impact of artificial intelligence on learning, teaching, and education. Minist de Educ. 2018. https://doi.org/10.2760/12297 .

Iqbal N, Ahmed H, Azhar KA. Exploring teachers’ attitudes towards using CHATGPT. Globa J Manag Adm Sci. 2022;3(4):97–111. https://doi.org/10.46568/gjmas.v3i4.163 .

Irfan M, Murray L, Ali S. Integration of Artificial intelligence in academia: a case study of critical teaching and learning in Higher education. Globa Soc Sci Rev. 2023;8(1):352–64. https://doi.org/10.31703/gssr.2023(viii-i).32 .

Jeon JH, Lee S. Large language models in education: a focus on the complementary relationship between human teachers and ChatGPT. Educ Inf Technol. 2023. https://doi.org/10.1007/s10639-023-11834-1 .

Khan RA, Jawaid M, Khan AR, Sajjad M. ChatGPT—Reshaping medical education and clinical management. Pak J Med Sci. 2023. https://doi.org/10.12669/pjms.39.2.7653 .

King MR. A conversation on artificial intelligence, Chatbots, and plagiarism in higher education. Cell Mol Bioeng. 2023;16(1):1–2. https://doi.org/10.1007/s12195-022-00754-8 .

Kooli C. Chatbots in education and research: a critical examination of ethical implications and solutions. Sustainability. 2023;15(7):5614. https://doi.org/10.3390/su15075614 .

Kuhail MA, Alturki N, Alramlawi S, Alhejori K. Interacting with educational chatbots: a systematic review. Educ Inf Technol. 2022;28(1):973–1018. https://doi.org/10.1007/s10639-022-11177-3 .

Lee H. The rise of ChatGPT: exploring its potential in medical education. Anat Sci Educ. 2023. https://doi.org/10.1002/ase.2270 .

Li L, Subbareddy R, Raghavendra CG. AI intelligence Chatbot to improve students learning in the higher education platform. J Interconnect Netw. 2022. https://doi.org/10.1142/s0219265921430325 .

Limna P. A Review of Artificial Intelligence (AI) in Education during the Digital Era. 2022. https://ssrn.com/abstract=4160798

Lo CK. What is the impact of ChatGPT on education? A rapid review of the literature. Educ Sci. 2023;13(4):410. https://doi.org/10.3390/educsci13040410 .

Luo W, He H, Liu J, Berson IR, Berson MJ, Zhou Y, Li H. Aladdin’s genie or pandora’s box For early childhood education? Experts chat on the roles, challenges, and developments of ChatGPT. Early Educ Dev. 2023. https://doi.org/10.1080/10409289.2023.2214181 .

Meyer JG, Urbanowicz RJ, Martin P, O’Connor K, Li R, Peng P, Moore JH. ChatGPT and large language models in academia: opportunities and challenges. Biodata Min. 2023. https://doi.org/10.1186/s13040-023-00339-9 .

Mhlanga D. Open AI in education, the responsible and ethical use of ChatGPT towards lifelong learning. Soc Sci Res Netw. 2023. https://doi.org/10.2139/ssrn.4354422 .

Neumann, M., Rauschenberger, M., & Schön, E. M. (2023). “We Need To Talk About ChatGPT”: The Future of AI and Higher Education.‏ https://doi.org/10.1109/seeng59157.2023.00010

Nolan B. Here are the schools and colleges that have banned the use of ChatGPT over plagiarism and misinformation fears. Business Insider . 2023. https://www.businessinsider.com

O’Leary DE. An analysis of three chatbots: BlenderBot, ChatGPT and LaMDA. Int J Intell Syst Account, Financ Manag. 2023;30(1):41–54. https://doi.org/10.1002/isaf.1531 .

Okoli C. A guide to conducting a standalone systematic literature review. Commun Assoc Inf Syst. 2015. https://doi.org/10.17705/1cais.03743 .

OpenAI. (2023). https://openai.com/blog/chatgpt

Perkins M. Academic integrity considerations of AI large language models in the post-pandemic era: ChatGPT and beyond. J Univ Teach Learn Pract. 2023. https://doi.org/10.53761/1.20.02.07 .

Plevris V, Papazafeiropoulos G, Rios AJ. Chatbots put to the test in math and logic problems: A preliminary comparison and assessment of ChatGPT-3.5, ChatGPT-4, and Google Bard. arXiv (Cornell University) . 2023. https://doi.org/10.48550/arxiv.2305.18618

Rahman MM, Watanobe Y (2023) ChatGPT for education and research: opportunities, threats, and strategies. Appl Sci 13(9):5783. https://doi.org/10.3390/app13095783

Ram B, Verma P. Artificial intelligence AI-based Chatbot study of ChatGPT, google AI bard and baidu AI. World J Adv Eng Technol Sci. 2023;8(1):258–61. https://doi.org/10.30574/wjaets.2023.8.1.0045 .

Rasul T, Nair S, Kalendra D, Robin M, de Oliveira Santini F, Ladeira WJ, Heathcote L. The role of ChatGPT in higher education: benefits, challenges, and future research directions. J Appl Learn Teach. 2023. https://doi.org/10.37074/jalt.2023.6.1.29 .

Ratnam M, Sharm B, Tomer A. ChatGPT: educational artificial intelligence. Int J Adv Trends Comput Sci Eng. 2023;12(2):84–91. https://doi.org/10.30534/ijatcse/2023/091222023 .

Ray PP. ChatGPT: a comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet Things Cyber-Phys Syst. 2023;3:121–54. https://doi.org/10.1016/j.iotcps.2023.04.003 .

Roumeliotis KI, Tselikas ND. ChatGPT and Open-AI models: a preliminary review. Future Internet. 2023;15(6):192. https://doi.org/10.3390/fi15060192 .

Rudolph J, Tan S, Tan S. War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education. J Appl Learn Teach. 2023. https://doi.org/10.37074/jalt.2023.6.1.23 .

Ruiz LMS, Moll-López S, Nuñez-Pérez A, Moraño J, Vega-Fleitas E. ChatGPT challenges blended learning methodologies in engineering education: a case study in mathematics. Appl Sci. 2023;13(10):6039. https://doi.org/10.3390/app13106039 .

Sallam M, Salim NA, Barakat M, Al-Tammemi AB. ChatGPT applications in medical, dental, pharmacy, and public health education: a descriptive study highlighting the advantages and limitations. Narra J. 2023;3(1): e103. https://doi.org/10.52225/narra.v3i1.103 .

Salvagno M, Taccone FS, Gerli AG. Can artificial intelligence help for scientific writing? Crit Care. 2023. https://doi.org/10.1186/s13054-023-04380-2 .

Saqr M, López-Pernas S, Helske S, Hrastinski S. The longitudinal association between engagement and achievement varies by time, students’ profiles, and achievement state: a full program study. Comput Educ. 2023;199:104787. https://doi.org/10.1016/j.compedu.2023.104787 .

Saqr M, Matcha W, Uzir N, Jovanović J, Gašević D, López-Pernas S. Transferring effective learning strategies across learning contexts matters: a study in problem-based learning. Australas J Educ Technol. 2023;39(3):9.

Schöbel S, Schmitt A, Benner D, Saqr M, Janson A, Leimeister JM. Charting the evolution and future of conversational agents: a research agenda along five waves and new frontiers. Inf Syst Front. 2023. https://doi.org/10.1007/s10796-023-10375-9 .

Shoufan A. Exploring students’ perceptions of CHATGPT: thematic analysis and follow-up survey. IEEE Access. 2023. https://doi.org/10.1109/access.2023.3268224 .

Sonderegger S, Seufert S. Chatbot-mediated learning: conceptual framework for the design of Chatbot use cases in education. Gallen: Institute for Educational Management and Technologies, University of St; 2022. https://doi.org/10.5220/0010999200003182 .

Book   Google Scholar  

Strzelecki A. To use or not to use ChatGPT in higher education? A study of students’ acceptance and use of technology. Interact Learn Environ. 2023. https://doi.org/10.1080/10494820.2023.2209881 .

Su J, Yang W. Unlocking the power of ChatGPT: a framework for applying generative AI in education. ECNU Rev Educ. 2023. https://doi.org/10.1177/20965311231168423 .

Sullivan M, Kelly A, McLaughlan P. ChatGPT in higher education: Considerations for academic integrity and student learning. J ApplLearn Teach. 2023;6(1):1–10. https://doi.org/10.37074/jalt.2023.6.1.17 .

Szabo A. ChatGPT is a breakthrough in science and education but fails a test in sports and exercise psychology. Balt J Sport Health Sci. 2023;1(128):25–40. https://doi.org/10.33607/bjshs.v127i4.1233 .

Taecharungroj V. “What can ChatGPT do?” analyzing early reactions to the innovative AI chatbot on Twitter. Big Data Cognit Comput. 2023;7(1):35. https://doi.org/10.3390/bdcc7010035 .

Tam S, Said RB. User preferences for ChatGPT-powered conversational interfaces versus traditional methods. Biomed Eng Soc. 2023. https://doi.org/10.58496/mjcsc/2023/004 .

Tedre M, Kahila J, Vartiainen H. (2023). Exploration on how co-designing with AI facilitates critical evaluation of ethics of AI in craft education. In: Langran E, Christensen P, Sanson J (Eds).  Proceedings of Society for Information Technology and Teacher Education International Conference . 2023. pp. 2289–2296.

Tlili A, Shehata B, Adarkwah MA, Bozkurt A, Hickey DT, Huang R, Agyemang B. What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learn Environ. 2023. https://doi.org/10.1186/s40561-023-00237-x .

Uddin SMJ, Albert A, Ovid A, Alsharef A. Leveraging CHATGPT to aid construction hazard recognition and support safety education and training. Sustainability. 2023;15(9):7121. https://doi.org/10.3390/su15097121 .

Valtonen T, López-Pernas S, Saqr M, Vartiainen H, Sointu E, Tedre M. The nature and building blocks of educational technology research. Comput Hum Behav. 2022;128:107123. https://doi.org/10.1016/j.chb.2021.107123 .

Vartiainen H, Tedre M. Using artificial intelligence in craft education: crafting with text-to-image generative models. Digit Creat. 2023;34(1):1–21. https://doi.org/10.1080/14626268.2023.2174557 .

Ventayen RJM. OpenAI ChatGPT generated results: similarity index of artificial intelligence-based contents. Soc Sci Res Netw. 2023. https://doi.org/10.2139/ssrn.4332664 .

Wagner MW, Ertl-Wagner BB. Accuracy of information and references using ChatGPT-3 for retrieval of clinical radiological information. Can Assoc Radiol J. 2023. https://doi.org/10.1177/08465371231171125 .

Wardat Y, Tashtoush MA, AlAli R, Jarrah AM. ChatGPT: a revolutionary tool for teaching and learning mathematics. Eurasia J Math, Sci Technol Educ. 2023;19(7):em2286. https://doi.org/10.29333/ejmste/13272 .

Webster J, Watson RT. Analyzing the past to prepare for the future: writing a literature review. Manag Inf Syst Quart. 2002;26(2):3.

Xiao Y, Watson ME. Guidance on conducting a systematic literature review. J Plan Educ Res. 2017;39(1):93–112. https://doi.org/10.1177/0739456x17723971 .

Yan D. Impact of ChatGPT on learners in a L2 writing practicum: an exploratory investigation. Educ Inf Technol. 2023. https://doi.org/10.1007/s10639-023-11742-4 .

Yu H. Reflection on whether Chat GPT should be banned by academia from the perspective of education and teaching. Front Psychol. 2023;14:1181712. https://doi.org/10.3389/fpsyg.2023.1181712 .

Zhu C, Sun M, Luo J, Li T, Wang M. How to harness the potential of ChatGPT in education? Knowl Manag ELearn. 2023;15(2):133–52. https://doi.org/10.34105/j.kmel.2023.15.008 .

Download references

The paper is co-funded by the Academy of Finland (Suomen Akatemia) Research Council for Natural Sciences and Engineering for the project Towards precision education: Idiographic learning analytics (TOPEILA), Decision Number 350560.

Author information

Authors and affiliations.

School of Computing, University of Eastern Finland, 80100, Joensuu, Finland

Yazid Albadarin, Mohammed Saqr, Nicolas Pope & Markku Tukiainen

You can also search for this author in PubMed   Google Scholar

Contributions

YA contributed to the literature search, data analysis, discussion, and conclusion. Additionally, YA contributed to the manuscript’s writing, editing, and finalization. MS contributed to the study’s design, conceptualization, acquisition of funding, project administration, allocation of resources, supervision, validation, literature search, and analysis of results. Furthermore, MS contributed to the manuscript's writing, revising, and approving it in its finalized state. NP contributed to the results, and discussions, and provided supervision. NP also contributed to the writing process, revisions, and the final approval of the manuscript in its finalized state. MT contributed to the study's conceptualization, resource management, supervision, writing, revising the manuscript, and approving it.

Corresponding author

Correspondence to Yazid Albadarin .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

See Table  4

The process of synthesizing the data presented in Table  4 involved identifying the relevant studies through a search process of databases (ERIC, Scopus, Web of Knowledge, Dimensions.ai, and lens.org) using specific keywords "ChatGPT" and "education". Following this, inclusion/exclusion criteria were applied, and data extraction was performed using Creswell's [ 15 ] coding techniques to capture key information and identify common themes across the included studies.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Albadarin, Y., Saqr, M., Pope, N. et al. A systematic literature review of empirical research on ChatGPT in education. Discov Educ 3 , 60 (2024). https://doi.org/10.1007/s44217-024-00138-2

Download citation

Received : 22 October 2023

Accepted : 10 May 2024

Published : 26 May 2024

DOI : https://doi.org/10.1007/s44217-024-00138-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Large language models
• Educational technology
• Systematic review

Advertisement

• Find a journal
• Publish with us
• Track your research