scholarly articles about research in education

On The Site

Harvard educational review.

Edited by Maya Alkateb-Chami, Jane Choi, Jeannette Garcia Coppersmith, Ron Grady, Phoebe A. Grant-Robinson, Pennie M. Gregory, Jennifer Ha, Woohee Kim, Catherine E. Pitcher, Elizabeth Salinas, Caroline Tucker, Kemeyawi Q. Wahpepah

Individuals

Institutions.

• Read the journal here

Journal Information

• ISSN: 0017-8055
• eISSN: 1943-5045
• Keywords: scholarly journal, education research
• First Issue: 1930
• Frequency: Quarterly

Description

The Harvard Educational Review (HER) is a scholarly journal of opinion and research in education. The Editorial Board aims to publish pieces from interdisciplinary and wide-ranging fields that advance our understanding of educational theory, equity, and practice. HER encourages submissions from established and emerging scholars, as well as from practitioners working in the field of education. Since its founding in 1930, HER has been central to elevating pieces and debates that tackle various dimensions of educational justice, with circulation to researchers, policymakers, teachers, and administrators.

Our Editorial Board is composed entirely of doctoral students from the Harvard Graduate School of Education who review all manuscripts considered for publication. For more information on the current Editorial Board, please see here.

A subscription to the Review includes access to the full-text electronic archives at our Subscribers-Only-Website .

Editorial Board

2023-2024 Harvard Educational Review Editorial Board Members

Maya Alkateb-Chami Development and Partnerships Editor, 2023-2024 Editor, 2022-2024 [email protected]

Maya Alkateb-Chami is a PhD student at the Harvard Graduate School of Education. Her research focuses on the role of schooling in fostering just futures—specifically in relation to language of instruction policies in multilingual contexts and with a focus on epistemic injustice. Prior to starting doctoral studies, she was the Managing Director of Columbia University’s Human Rights Institute, where she supported and co-led a team of lawyers working to advance human rights through research, education, and advocacy. Prior to that, she was the Executive Director of Jusoor, a nonprofit organization that helps conflict-affected Syrian youth and children pursue their education in four countries. Alkateb-Chami is a Fulbright Scholar and UNESCO cultural heritage expert. She holds an MEd in Language and Literacy from Harvard University; an MSc in Education from Indiana University, Bloomington; and a BA in Political Science from Damascus University, and her research on arts-based youth empowerment won the annual Master’s Thesis Award of the U.S. Society for Education Through Art.

Jane Choi Editor, 2023-2025

Jane Choi is a second-year PhD student in Sociology with broad interests in culture, education, and inequality. Her research examines intra-racial and interracial boundaries in US educational contexts. She has researched legacy and first-generation students at Ivy League colleges, families served by Head Start and Early Head Start programs, and parents of pre-K and kindergarten-age children in the New York City School District. Previously, Jane worked as a Research Assistant in the Family Well-Being and Children’s Development policy area at MDRC and received a BA in Sociology from Columbia University.

Jeannette Garcia Coppersmith Content Editor, 2023-2024 Editor, 2022-2024 [email protected]

Jeannette Garcia Coppersmith is a fourth-year Education PhD student in the Human Development, Learning and Teaching concentration at the Harvard Graduate School of Education. A former public middle and high school mathematics teacher and department chair, she is interested in understanding the mechanisms that contribute to disparities in secondary mathematics education, particularly how teacher beliefs and biases intersect with the social-psychological processes and pedagogical choices involved in math teaching. Jeannette holds an EdM in Learning and Teaching from the Harvard Graduate School of Education where she studied as an Urban Scholar and a BA in Environmental Sciences from the University of California, Berkeley.

Ron Grady Editor, 2023-2025

Ron Grady is a second-year doctoral student in the Human Development, Learning, and Teaching concentration at the Harvard Graduate School of Education. His central curiosities involve the social worlds and peer cultures of young children, wondering how lived experience is both constructed within and revealed throughout play, the creation of art and narrative, and through interaction with/production of visual artifacts such as photography and film. Ron also works extensively with educators interested in developing and deepening practices rooted in reflection on, inquiry into, and translation of the social, emotional, and aesthetic aspects of their classroom ecosystems. Prior to his doctoral studies, Ron worked as a preschool teacher in New Orleans. He holds a MS in Early Childhood Education from the Erikson Institute and a BA in Psychology with Honors in Education from Stanford University.

Phoebe A. Grant-Robinson Editor, 2023-2024

Phoebe A. Grant-Robinson is a first year student in the Doctor of Education Leadership(EdLD) program at the Harvard Graduate School of Education. Her ultimate quest is to position all students as drivers of their destiny. Phoebe is passionate about early learning and literacy. She is committed to ensuring that districts and school leaders, have the necessary tools to create equitable learning organizations that facilitate the academic and social well-being of all students. Phoebe is particularly interested in the intersection of homeless students and literacy. Prior to her doctoral studies, Phoebe was a Special Education Instructional Specialist. Supporting a portfolio of more than thirty schools, she facilitated the rollout of New York City’s Special Education Reform. Phoebe also served as an elementary school principal. She holds a BS in Inclusive Education from Syracuse University, and an MS in Curriculum and Instruction from Pace University.

Pennie M. Gregory Editor, 2023-2024

Pennie M. Gregory is a second-year student in the Doctor of Education Leadership (EdLD) program at the Harvard Graduate School of Education. Pennie was born in Incheon, South Korea and raised in Gary, Indiana. She has decades of experience leading efforts to improve outcomes for students with disabilities first as a special education teacher and then as a school district special education administrator. Prior to her doctoral studies, Pennie helped to create Indiana’s first Aspiring Special Education Leadership Institute (ASELI) and served as its Director. She was also the Capacity Events Director for MelanatED Leaders, an organization created to support educational leaders of color in Indianapolis. Pennie has a unique perspective, having worked with members of the school community, with advocacy organizations, and supporting state special education leaders. Pennie holds an EdM in Education Leadership from Marian University.

Jennifer Ha Editor, 2023-2025

Jen Ha is a second-year PhD student in the Culture, Institutions, and Society concentration at the Harvard Graduate School of Education. Her research explores how high school students learn to write personal narratives for school applications, scholarships, and professional opportunities amidst changing landscapes in college access and admissions. Prior to doctoral studies, Jen served as the Coordinator of Public Humanities at Bard Graduate Center and worked in several roles organizing academic enrichment opportunities and supporting postsecondary planning for students in New Haven and New York City. Jen holds a BA in Humanities from Yale University, where she was an Education Studies Scholar.

Woohee Kim Editor, 2023-2025

Woohee Kim is a PhD student studying youth activists’ civic and pedagogical practices. She is a scholar-activist dedicated to creating spaces for pedagogies of resistance and transformative possibilities. Shaped by her activism and research across South Korea, the US, and the UK, Woohee seeks to interrogate how educational spaces are shaped as cultural and political sites and reshaped by activists as sites of struggle. She hopes to continue exploring the intersections of education, knowledge, power, and resistance.

Catherine E. Pitcher Editor, 2023-2025

Catherine is a second-year doctoral student at Harvard Graduate School of Education in the Culture, Institutions, and Society program. She has over 10 years of experience in education in the US in roles that range from special education teacher to instructional coach to department head to educational game designer. She started working in Palestine in 2017, first teaching, and then designing and implementing educational programming. Currently, she is working on research to understand how Palestinian youth think about and build their futures and continues to lead programming in the West Bank, Gaza, and East Jerusalem. She holds an EdM from Harvard in International Education Policy.

Elizabeth Salinas Editor, 2023-2025

Elizabeth Salinas is a doctoral student in the Education Policy and Program Evaluation concentration at HGSE. She is interested in the intersection of higher education and the social safety net and hopes to examine policies that address basic needs insecurity among college students. Before her doctoral studies, Liz was a research director at a public policy consulting firm. There, she supported government, education, and philanthropy leaders by conducting and translating research into clear and actionable information. Previously, Liz served as a high school physics teacher in her hometown in Texas and as a STEM outreach program director at her alma mater. She currently sits on the Board of Directors at Leadership Enterprise for a Diverse America, a nonprofit organization working to diversify the leadership pipeline in the United States. Liz holds a bachelor’s degree in civil engineering from the Massachusetts Institute of Technology and a master’s degree in higher education from the Harvard Graduate School of Education.

Caroline Tucker Co-Chair, 2023-2024 Editor, 2022-2024 [email protected]

Caroline Tucker is a fourth-year doctoral student in the Culture, Institutions, and Society concentration at the Harvard Graduate School of Education. Her research focuses on the history and organizational dynamics of women’s colleges as women gained entry into the professions and coeducation took root in the United States. She is also a research assistant for the Harvard and the Legacy of Slavery Initiative’s Subcommittee on Curriculum and the editorial assistant for Into Practice, the pedagogy newsletter distributed by Harvard University’s Office of the Vice Provost for Advances in Learning. Prior to her doctoral studies, Caroline served as an American politics and English teaching fellow in London and worked in college advising. Caroline holds a BA in History from Princeton University, an MA in the Social Sciences from the University of Chicago, and an EdM in Higher Education from the Harvard Graduate School of Education.

Kemeyawi Q. Wahpepah Co-Chair, 2023-2024 Editor, 2022-2024 [email protected]

Kemeyawi Q. Wahpepah (Kickapoo, Sac & Fox) is a fourth-year doctoral student in the Culture, Institutions, and Society concentration at the Harvard Graduate School of Education. Their research explores how settler colonialism is addressed in K-12 history and social studies classrooms in the United States. Prior to their doctoral studies, Kemeyawi taught middle and high school English and history for eleven years in Boston and New York City. They hold an MS in Middle Childhood Education from Hunter College and an AB in Social Studies from Harvard University.

Submission Information

Click here to view submission guidelines .

Contact Information

Click here to view contact information for the editorial board and customer service .

Subscriber Support

Individual subscriptions must have an individual name in the given address for shipment. Individual copies are not for multiple readers or libraries. Individual accounts come with a personal username and password for access to online archives. Online access instructions will be attached to your order confirmation e-mail.

Institutional rates apply to libraries and organizations with multiple readers. Institutions receive digital access to content on Meridian from IP addresses via theIPregistry.org (by sending HER your PSI Org ID).

Online access instructions will be attached to your order confirmation e-mail. If you have questions about using theIPregistry.org you may find the answers in their FAQs. Otherwise please let us know at [email protected] .

How to Subscribe

To order online via credit card, please use the subscribe button at the top of this page.

To order by phone, please call 888-437-1437.

Checks can be mailed to Harvard Educational Review C/O Fulco, 30 Broad Street, Suite 6, Denville, NJ 07834. (Please include reference to your subscriber number if you are renewing. Institutions must include their PSI Org ID or follow up with this information via email to [email protected] .)

Permissions

Click here to view permissions information.

Article Submission FAQ

Closing the open call, question: “i have already submitted an article to her and i am awaiting a decision, what can i expect”.

Answer: First, any manuscripts already submitted through the open call and acknowledged by HER, as well as all invited manuscripts, R&R’d manuscripts, and manuscripts currently in production are NOT affected in any way by our pause in open calls. Editors are working to move through all current submissions and you can expect to receive any updates or decisions as we move through each step of our production process. If you have any questions, please contact the Co-Chairs, Caroline Tucker and Kemeyawi Wahpepah at [email protected] .

Question: “Can you share more about why you are closing the open call?”

Answer: As a graduate student run journal, we perform our editorial tasks in addition to our daily lives as doctoral students. We have been (and continue to be) incredibly grateful for the authors who share their work with us. In closing the open call, we hope to give ourselves time to review each manuscript in the best manner possible.

Submissions

Question: “what manuscripts are a good fit for her ”.

Answer: As a generalist scholarly journal, HER publishes on a wide range of topics within the field of education and related disciplines. We receive many articles that deserve publication, but due to the restrictions of print publication, we are only able to publish very few in the journal. The originality and import of the findings, as well as the accessibility of a piece to HER’s interdisciplinary, international audience which includes education practitioners, are key criteria in determining if an article will be selected for publication.

We strongly recommend that prospective authors review the current and past issues of HER to see the types of articles we have published recently. If you are unsure whether your manuscript is a good fit, please reach out to the Content Editor at [email protected] .

Question: “What makes HER a developmental journal?”

Answer: Supporting the development of high-quality education research is a key tenet of HER’s mission. HER promotes this development through offering comprehensive feedback to authors. All manuscripts that pass the first stage of our review process (see below) receive detailed feedback. For accepted manuscripts, HER also has a unique feedback process called casting whereby two editors carefully read a manuscript and offer overarching suggestions to strengthen and clarify the argument.

Question: “What is a Voices piece and how does it differ from an essay?”

Answer: Voices pieces are first-person reflections about an education-related topic rather than empirical or theoretical essays. Our strongest pieces have often come from educators and policy makers who draw on their personal experiences in the education field. Although they may not present data or generate theory, Voices pieces should still advance a cogent argument, drawing on appropriate literature to support any claims asserted. For examples of Voices pieces, please see Alvarez et al. (2021) and Snow (2021).

Question: “Does HER accept Book Note or book review submissions?”

Answer: No, all Book Notes are written internally by members of the Editorial Board.

Question: “If I want to submit a book for review consideration, who do I contact?”

Answer: Please send details about your book to the Content Editor at [email protected].

Manuscript Formatting

Question: “the submission guidelines state that manuscripts should be a maximum of 9,000 words – including abstract, appendices, and references. is this applicable only for research articles, or should the word count limit be followed for other manuscripts, such as essays”.

Answer: The 9,000-word limit is the same for all categories of manuscripts.

Question: “We are trying to figure out the best way to mask our names in the references. Is it OK if we do not cite any of our references in the reference list? Our names have been removed in the in-text citations. We just cite Author (date).”

Answer: Any references that identify the author/s in the text must be masked or made anonymous (e.g., instead of citing “Field & Bloom, 2007,” cite “Author/s, 2007”). For the reference list, place the citations alphabetically as “Author/s. (2007)” You can also indicate that details are omitted for blind review. Articles can also be blinded effectively by use of the third person in the manuscript. For example, rather than “in an earlier article, we showed that” substitute something like “as has been shown in Field & Bloom, 2007.” In this case, there is no need to mask the reference in the list. Please do not submit a title page as part of your manuscript. We will capture the contact information and any author statement about the fit and scope of the work in the submission form. Finally, please save the uploaded manuscript as the title of the manuscript and do not include the author/s name/s.

Invitations

Question: “can i be invited to submit a manuscript how”.

Answer: If you think your manuscript is a strong fit for HER, we welcome your request for invitation. Invited manuscripts receive one round of feedback from Editors before the piece enters the formal review process. To submit information about your manuscript for the Board to consider for invitation, please fill out the Invitation Request Form. Please provide as many details as possible. Whether we could invite your manuscript depends on the interest and availability of the current Board. Once you submit the form, we will give you an update in about 2–3 weeks on whether there are Editors who are interested in inviting your manuscript.

Review Timeline

Question: “who reviews manuscripts”.

Answer: All manuscripts are reviewed by the Editorial Board composed of doctoral students at Harvard University.

Question: “What is the HER evaluation process as a student-run journal?”

Answer: HER does not utilize the traditional external peer review process and instead has an internal, two-stage review procedure.

Upon submission, every manuscript receives a preliminary assessment by the Content Editor to confirm that the formatting requirements have been carefully followed in preparation of the manuscript, and that the manuscript is in accord with the scope and aim of the journal. The manuscript then formally enters the review process.

In the first stage of review, all manuscripts are read by a minimum of two Editorial Board members. During the second stage of review, manuscripts are read by the full Editorial Board at a weekly meeting.

Question: “How long after submission can I expect a decision on my manuscript?”

Answer: It usually takes 6 to 10 weeks for a manuscript to complete the first stage of review and an additional 12 weeks for a manuscript to complete the second stage. Due to time constraints and the large volume of manuscripts received, HER only provides detailed comments on manuscripts that complete the second stage of review.

Question: “How soon are accepted pieces published?”

Answer: The date of publication depends entirely on how many manuscripts are already in the queue for an issue. Typically, however, it takes about 6 months post-acceptance for a piece to be published.

Submission Process

Question: “how do i submit a manuscript for publication in her”.

Answer: Manuscripts are submitted through HER’s Submittable platform, accessible here. All first-time submitters must create an account to access the platform. You can find details on our submission guidelines on our Submissions page.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Review Article
• Open access
• Published: 12 February 2024

Education reform and change driven by digital technology: a bibliometric study from a global perspective

• Chengliang Wang 1 ,
• Xiaojiao Chen 1 ,
• Teng Yu   ORCID: orcid.org/0000-0001-5198-7261 2 , 3 ,
• Yidan Liu 1 , 4 &
• Yuhui Jing 1  

Humanities and Social Sciences Communications volume  11 , Article number:  256 ( 2024 ) Cite this article

2577 Accesses

1 Altmetric

Metrics details

• Development studies
• Science, technology and society

Amidst the global digital transformation of educational institutions, digital technology has emerged as a significant area of interest among scholars. Such technologies have played an instrumental role in enhancing learner performance and improving the effectiveness of teaching and learning. These digital technologies also ensure the sustainability and stability of education during the epidemic. Despite this, a dearth of systematic reviews exists regarding the current state of digital technology application in education. To address this gap, this study utilized the Web of Science Core Collection as a data source (specifically selecting the high-quality SSCI and SCIE) and implemented a topic search by setting keywords, yielding 1849 initial publications. Furthermore, following the PRISMA guidelines, we refined the selection to 588 high-quality articles. Using software tools such as CiteSpace, VOSviewer, and Charticulator, we reviewed these 588 publications to identify core authors (such as Selwyn, Henderson, Edwards), highly productive countries/regions (England, Australia, USA), key institutions (Monash University, Australian Catholic University), and crucial journals in the field ( Education and Information Technologies , Computers & Education , British Journal of Educational Technology ). Evolutionary analysis reveals four developmental periods in the research field of digital technology education application: the embryonic period, the preliminary development period, the key exploration, and the acceleration period of change. The study highlights the dual influence of technological factors and historical context on the research topic. Technology is a key factor in enabling education to transform and upgrade, and the context of the times is an important driving force in promoting the adoption of new technologies in the education system and the transformation and upgrading of education. Additionally, the study identifies three frontier hotspots in the field: physical education, digital transformation, and professional development under the promotion of digital technology. This study presents a clear framework for digital technology application in education, which can serve as a valuable reference for researchers and educational practitioners concerned with digital technology education application in theory and practice.

Similar content being viewed by others

Digital transformation and digital literacy in the context of complexity within higher education institutions: a systematic literature review

Silvia Farias-Gaytan, Ignacio Aguaded & Maria-Soledad Ramirez-Montoya

Education big data and learning analytics: a bibliometric analysis

Shaza Arissa Samsul, Noraffandy Yahaya & Hassan Abuhassna

Knowledge mapping of vocational education and training research (2004–2020): a visual analysis based on CiteSpace

Yumi Tian, Jiayun Liu, … Xueshi Wu

Introduction

Digital technology has become an essential component of modern education, facilitating the extension of temporal and spatial boundaries and enriching the pedagogical contexts (Selwyn and Facer, 2014 ). The advent of mobile communication technology has enabled learning through social media platforms (Szeto et al. 2015 ; Pires et al. 2022 ), while the advancement of augmented reality technology has disrupted traditional conceptions of learning environments and spaces (Perez-Sanagustin et al., 2014 ; Kyza and Georgiou, 2018 ). A wide range of digital technologies has enabled learning to become a norm in various settings, including the workplace (Sjöberg and Holmgren, 2021 ), home (Nazare et al. 2022 ), and online communities (Tang and Lam, 2014 ). Education is no longer limited to fixed locations and schedules, but has permeated all aspects of life, allowing learning to continue at any time and any place (Camilleri and Camilleri, 2016 ; Selwyn and Facer, 2014 ).

The advent of digital technology has led to the creation of several informal learning environments (Greenhow and Lewin, 2015 ) that exhibit divergent form, function, features, and patterns in comparison to conventional learning environments (Nygren et al. 2019 ). Consequently, the associated teaching and learning processes, as well as the strategies for the creation, dissemination, and acquisition of learning resources, have undergone a complete overhaul. The ensuing transformations have posed a myriad of novel issues, such as the optimal structuring of teaching methods by instructors and the adoption of appropriate learning strategies by students in the new digital technology environment. Consequently, an examination of the principles that underpin effective teaching and learning in this environment is a topic of significant interest to numerous scholars engaged in digital technology education research.

Over the course of the last two decades, digital technology has made significant strides in the field of education, notably in extending education time and space and creating novel educational contexts with sustainability. Despite research attempts to consolidate the application of digital technology in education, previous studies have only focused on specific aspects of digital technology, such as Pinto and Leite’s ( 2020 ) investigation into digital technology in higher education and Mustapha et al.’s ( 2021 ) examination of the role and value of digital technology in education during the pandemic. While these studies have provided valuable insights into the practical applications of digital technology in particular educational domains, they have not comprehensively explored the macro-mechanisms and internal logic of digital technology implementation in education. Additionally, these studies were conducted over a relatively brief period, making it challenging to gain a comprehensive understanding of the macro-dynamics and evolutionary process of digital technology in education. Some studies have provided an overview of digital education from an educational perspective but lack a precise understanding of technological advancement and change (Yang et al. 2022 ). Therefore, this study seeks to employ a systematic scientific approach to collate relevant research from 2000 to 2022, comprehend the internal logic and development trends of digital technology in education, and grasp the outstanding contribution of digital technology in promoting the sustainability of education in time and space. In summary, this study aims to address the following questions:

RQ1: Since the turn of the century, what is the productivity distribution of the field of digital technology education application research in terms of authorship, country/region, institutional and journal level?

RQ2: What is the development trend of research on the application of digital technology in education in the past two decades?

RQ3: What are the current frontiers of research on the application of digital technology in education?

Literature review

Although the term “digital technology” has become ubiquitous, a unified definition has yet to be agreed upon by scholars. Because the meaning of the word digital technology is closely related to the specific context. Within the educational research domain, Selwyn’s ( 2016 ) definition is widely favored by scholars (Pinto and Leite, 2020 ). Selwyn ( 2016 ) provides a comprehensive view of various concrete digital technologies and their applications in education through ten specific cases, such as immediate feedback in classes, orchestrating teaching, and community learning. Through these specific application scenarios, Selwyn ( 2016 ) argues that digital technology encompasses technologies associated with digital devices, including but not limited to tablets, smartphones, computers, and social media platforms (such as Facebook and YouTube). Furthermore, Further, the behavior of accessing the internet at any location through portable devices can be taken as an extension of the behavior of applying digital technology.

The evolving nature of digital technology has significant implications in the field of education. In the 1890s, the focus of digital technology in education was on comprehending the nuances of digital space, digital culture, and educational methodologies, with its connotations aligned more towards the idea of e-learning. The advent and subsequent widespread usage of mobile devices since the dawn of the new millennium have been instrumental in the rapid expansion of the concept of digital technology. Notably, mobile learning devices such as smartphones and tablets, along with social media platforms, have become integral components of digital technology (Conole and Alevizou, 2010 ; Batista et al. 2016 ). In recent times, the burgeoning application of AI technology in the education sector has played a vital role in enriching the digital technology lexicon (Banerjee et al. 2021 ). ChatGPT, for instance, is identified as a novel educational technology that has immense potential to revolutionize future education (Rospigliosi, 2023 ; Arif, Munaf and Ul-Haque, 2023 ).

Pinto and Leite ( 2020 ) conducted a comprehensive macroscopic survey of the use of digital technologies in the education sector and identified three distinct categories, namely technologies for assessment and feedback, mobile technologies, and Information Communication Technologies (ICT). This classification criterion is both macroscopic and highly condensed. In light of the established concept definitions of digital technology in the educational research literature, this study has adopted the characterizations of digital technology proposed by Selwyn ( 2016 ) and Pinto and Leite ( 2020 ) as crucial criteria for analysis and research inclusion. Specifically, this criterion encompasses several distinct types of digital technologies, including Information and Communication Technologies (ICT), Mobile tools, eXtended Reality (XR) Technologies, Assessment and Feedback systems, Learning Management Systems (LMS), Publish and Share tools, Collaborative systems, Social media, Interpersonal Communication tools, and Content Aggregation tools.

Methodology and materials

Research method: bibliometric.

The research on econometric properties has been present in various aspects of human production and life, yet systematic scientific theoretical guidance has been lacking, resulting in disorganization. In 1969, British scholar Pritchard ( 1969 ) proposed “bibliometrics,” which subsequently emerged as an independent discipline in scientific quantification research. Initially, Pritchard defined bibliometrics as “the application of mathematical and statistical methods to books and other media of communication,” however, the definition was not entirely rigorous. To remedy this, Hawkins ( 2001 ) expanded Pritchard’s definition to “the quantitative analysis of the bibliographic features of a body of literature.” De Bellis further clarified the objectives of bibliometrics, stating that it aims to analyze and identify patterns in literature, such as the most productive authors, institutions, countries, and journals in scientific disciplines, trends in literary production over time, and collaboration networks (De Bellis, 2009 ). According to Garfield ( 2006 ), bibliometric research enables the examination of the history and structure of a field, the flow of information within the field, the impact of journals, and the citation status of publications over a longer time scale. All of these definitions illustrate the unique role of bibliometrics as a research method for evaluating specific research fields.

This study uses CiteSpace, VOSviewer, and Charticulator to analyze data and create visualizations. Each of these three tools has its own strengths and can complement each other. CiteSpace and VOSviewer use set theory and probability theory to provide various visualization views in fields such as keywords, co-occurrence, and co-authors. They are easy to use and produce visually appealing graphics (Chen, 2006 ; van Eck and Waltman, 2009 ) and are currently the two most widely used bibliometric tools in the field of visualization (Pan et al. 2018 ). In this study, VOSviewer provided the data necessary for the Performance Analysis; Charticulator was then used to redraw using the tabular data exported from VOSviewer (for creating the chord diagram of country collaboration); this was to complement the mapping process, while CiteSpace was primarily utilized to generate keyword maps and conduct burst word analysis.

Data retrieval

This study selected documents from the Science Citation Index Expanded (SCIE) and Social Science Citation Index (SSCI) in the Web of Science Core Collection as the data source, for the following reasons:

(1) The Web of Science Core Collection, as a high-quality digital literature resource database, has been widely accepted by many researchers and is currently considered the most suitable database for bibliometric analysis (Jing et al. 2023a ). Compared to other databases, Web of Science provides more comprehensive data information (Chen et al. 2022a ), and also provides data formats suitable for analysis using VOSviewer and CiteSpace (Gaviria-Marin et al. 2019 ).

(2) The application of digital technology in the field of education is an interdisciplinary research topic, involving technical knowledge literature belonging to the natural sciences and education-related literature belonging to the social sciences. Therefore, it is necessary to select Science Citation Index Expanded (SCIE) and Social Science Citation Index (SSCI) as the sources of research data, ensuring the comprehensiveness of data while ensuring the reliability and persuasiveness of bibliometric research (Hwang and Tsai, 2011 ; Wang et al. 2022 ).

After establishing the source of research data, it is necessary to determine a retrieval strategy (Jing et al. 2023b ). The choice of a retrieval strategy should consider a balance between the breadth and precision of the search formula. That is to say, it should encompass all the literature pertaining to the research topic while excluding irrelevant documents as much as possible. In light of this, this study has set a retrieval strategy informed by multiple related papers (Mustapha et al. 2021 ; Luo et al. 2021 ). The research by Mustapha et al. ( 2021 ) guided us in selecting keywords (“digital” AND “technolog*”) to target digital technology, while Luo et al. ( 2021 ) informed the selection of terms (such as “instruct*,” “teach*,” and “education”) to establish links with the field of education. Then, based on the current application of digital technology in the educational domain and the scope of selection criteria, we constructed the final retrieval strategy. Following the general patterns of past research (Jing et al. 2023a , 2023b ), we conducted a specific screening using the topic search (Topics, TS) function in Web of Science. For the specific criteria used in the screening for this study, please refer to Table 1 .

Literature screening

Literature acquired through keyword searches may contain ostensibly related yet actually unrelated works. Therefore, to ensure the close relevance of literature included in the analysis to the research topic, it is often necessary to perform a manual screening process to identify the final literature to be analyzed, subsequent to completing the initial literature search.

The manual screening process consists of two steps. Initially, irrelevant literature is weeded out based on the title and abstract, with two members of the research team involved in this phase. This stage lasted about one week, resulting in 1106 articles being retained. Subsequently, a comprehensive review of the full text is conducted to accurately identify the literature required for the study. To carry out the second phase of manual screening effectively and scientifically, and to minimize the potential for researcher bias, the research team established the inclusion criteria presented in Table 2 . Three members were engaged in this phase, which took approximately 2 weeks, culminating in the retention of 588 articles after meticulous screening. The entire screening process is depicted in Fig. 1 , adhering to the PRISMA guidelines (Page et al. 2021 ).

The process of obtaining and filtering the necessary literature data for research.

Data standardization

Nguyen and Hallinger ( 2020 ) pointed out that raw data extracted from scientific databases often contains multiple expressions of the same term, and not addressing these synonymous expressions could affect research results in bibliometric analysis. For instance, in the original data, the author list may include “Tsai, C. C.” and “Tsai, C.-C.”, while the keyword list may include “professional-development” and “professional development,” which often require merging. Therefore, before analyzing the selected literature, a data disambiguation process is necessary to standardize the data (Strotmann and Zhao, 2012 ; Van Eck and Waltman, 2019 ). This study adopted the data standardization process proposed by Taskin and Al ( 2019 ), mainly including the following standardization operations:

Firstly, the author and source fields in the data are corrected and standardized to differentiate authors with similar names.

Secondly, the study checks whether the journals to which the literature belongs have been renamed in the past over 20 years, so as to avoid the influence of periodical name change on the analysis results.

Finally, the keyword field is standardized by unifying parts of speech and singular/plural forms of keywords, which can help eliminate redundant entries in the knowledge graph.

Performance analysis (RQ1)

This section offers a thorough and detailed analysis of the state of research in the field of digital technology education. By utilizing descriptive statistics and visual maps, it provides a comprehensive overview of the development trends, authors, countries, institutions, and journal distribution within the field. The insights presented in this section are of great significance in advancing our understanding of the current state of research in this field and identifying areas for further investigation. The use of visual aids to display inter-country cooperation and the evolution of the field adds to the clarity and coherence of the analysis.

Time trend of the publications

To understand a research field, it is first necessary to understand the most basic quantitative information, among which the change in the number of publications per year best reflects the development trend of a research field. Figure 2 shows the distribution of publication dates.

Time trend of the publications on application of digital technology in education.

From the Fig. 2 , it can be seen that the development of this field over the past over 20 years can be roughly divided into three stages. The first stage was from 2000 to 2007, during which the number of publications was relatively low. Due to various factors such as technological maturity, the academic community did not pay widespread attention to the role of digital technology in expanding the scope of teaching and learning. The second stage was from 2008 to 2019, during which the overall number of publications showed an upward trend, and the development of the field entered an accelerated period, attracting more and more scholars’ attention. The third stage was from 2020 to 2022, during which the number of publications stabilized at around 100. During this period, the impact of the pandemic led to a large number of scholars focusing on the role of digital technology in education during the pandemic, and research on the application of digital technology in education became a core topic in social science research.

Analysis of authors

An analysis of the author’s publication volume provides information about the representative scholars and core research strengths of a research area. Table 3 presents information on the core authors in adaptive learning research, including name, publication number, and average number of citations per article (based on the analysis and statistics from VOSviewer).

Variations in research foci among scholars abound. Within the field of digital technology education application research over the past two decades, Neil Selwyn stands as the most productive author, having published 15 papers garnering a total of 1027 citations, resulting in an average of 68.47 citations per paper. As a Professor at the Faculty of Education at Monash University, Selwyn concentrates on exploring the application of digital technology in higher education contexts (Selwyn et al. 2021 ), as well as related products in higher education such as Coursera, edX, and Udacity MOOC platforms (Bulfin et al. 2014 ). Selwyn’s contributions to the educational sociology perspective include extensive research on the impact of digital technology on education, highlighting the spatiotemporal extension of educational processes and practices through technological means as the greatest value of educational technology (Selwyn, 2012 ; Selwyn and Facer, 2014 ). In addition, he provides a blueprint for the development of future schools in 2030 based on the present impact of digital technology on education (Selwyn et al. 2019 ). The second most productive author in this field, Henderson, also offers significant contributions to the understanding of the important value of digital technology in education, specifically in the higher education setting, with a focus on the impact of the pandemic (Henderson et al. 2015 ; Cohen et al. 2022 ). In contrast, Edwards’ research interests focus on early childhood education, particularly the application of digital technology in this context (Edwards, 2013 ; Bird and Edwards, 2015 ). Additionally, on the technical level, Edwards also mainly prefers digital game technology, because it is a digital technology that children are relatively easy to accept (Edwards, 2015 ).

Analysis of countries/regions and organization

The present study aimed to ascertain the leading countries in digital technology education application research by analyzing 75 countries related to 558 works of literature. Table 4 depicts the top ten countries that have contributed significantly to this field in terms of publication count (based on the analysis and statistics from VOSviewer). Our analysis of Table 4 data shows that England emerged as the most influential country/region, with 92 published papers and 2401 citations. Australia and the United States secured the second and third ranks, respectively, with 90 papers (2187 citations) and 70 papers (1331 citations) published. Geographically, most of the countries featured in the top ten publication volumes are situated in Australia, North America, and Europe, with China being the only exception. Notably, all these countries, except China, belong to the group of developed nations, suggesting that economic strength is a prerequisite for fostering research in the digital technology education application field.

This study presents a visual representation of the publication output and cooperation relationships among different countries in the field of digital technology education application research. Specifically, a chord diagram is employed to display the top 30 countries in terms of publication output, as depicted in Fig. 3 . The chord diagram is composed of nodes and chords, where the nodes are positioned as scattered points along the circumference, and the length of each node corresponds to the publication output, with longer lengths indicating higher publication output. The chords, on the other hand, represent the cooperation relationships between any two countries, and are weighted based on the degree of closeness of the cooperation, with wider chords indicating closer cooperation. Through the analysis of the cooperation relationships, the findings suggest that the main publishing countries in this field are engaged in cooperative relationships with each other, indicating a relatively high level of international academic exchange and research internationalization.

In the diagram, nodes are scattered along the circumference of a circle, with the length of each node representing the volume of publications. The weighted arcs connecting any two points on the circle are known as chords, representing the collaborative relationship between the two, with the width of the arc indicating the closeness of the collaboration.

Further analyzing Fig. 3 , we can extract more valuable information, enabling a deeper understanding of the connections between countries in the research field of digital technology in educational applications. It is evident that certain countries, such as the United States, China, and England, display thicker connections, indicating robust collaborative relationships in terms of productivity. These thicker lines signify substantial mutual contributions and shared objectives in certain sectors or fields, highlighting the interconnectedness and global integration in these areas. By delving deeper, we can also explore potential future collaboration opportunities through the chord diagram, identifying possible partners to propel research and development in this field. In essence, the chord diagram successfully encapsulates and conveys the multi-dimensionality of global productivity and cooperation, allowing for a comprehensive understanding of the intricate inter-country relationships and networks in a global context, providing valuable guidance and insights for future research and collaborations.

An in-depth examination of the publishing institutions is provided in Table 5 , showcasing the foremost 10 institutions ranked by their publication volume. Notably, Monash University and Australian Catholic University, situated in Australia, have recorded the most prolific publications within the digital technology education application realm, with 22 and 10 publications respectively. Moreover, the University of Oslo from Norway is featured among the top 10 publishing institutions, with an impressive average citation count of 64 per publication. It is worth highlighting that six institutions based in the United Kingdom were also ranked within the top 10 publishing institutions, signifying their leading position in this area of research.

Analysis of journals

Journals are the main carriers for publishing high-quality papers. Some scholars point out that the two key factors to measure the influence of journals in the specified field are the number of articles published and the number of citations. The more papers published in a magazine and the more citations, the greater its influence (Dzikowski, 2018 ). Therefore, this study utilized VOSviewer to statistically analyze the top 10 journals with the most publications in the field of digital technology in education and calculated the average citations per article (see Table 6 ).

Based on Table 6 , it is apparent that the highest number of articles in the domain of digital technology in education research were published in Education and Information Technologies (47 articles), Computers & Education (34 articles), and British Journal of Educational Technology (32 articles), indicating a higher article output compared to other journals. This underscores the fact that these three journals concentrate more on the application of digital technology in education. Furthermore, several other journals, such as Technology Pedagogy and Education and Sustainability, have published more than 15 articles in this domain. Sustainability represents the open access movement, which has notably facilitated research progress in this field, indicating that the development of open access journals in recent years has had a significant impact. Although there is still considerable disagreement among scholars on the optimal approach to achieve open access, the notion that research outcomes should be accessible to all is widely recognized (Huang et al. 2020 ). On further analysis of the research fields to which these journals belong, except for Sustainability, it is evident that they all pertain to educational technology, thus providing a qualitative definition of the research area of digital technology education from the perspective of journals.

Temporal keyword analysis: thematic evolution (RQ2)

The evolution of research themes is a dynamic process, and previous studies have attempted to present the developmental trajectory of fields by drawing keyword networks in phases (Kumar et al. 2021 ; Chen et al. 2022b ). To understand the shifts in research topics across different periods, this study follows past research and, based on the significant changes in the research field and corresponding technological advancements during the outlined periods, divides the timeline into four stages (the first stage from January 2000 to December 2005, the second stage from January 2006 to December 2011, the third stage from January 2012 to December 2017; and the fourth stage from January 2018 to December 2022). The division into these four stages was determined through a combination of bibliometric analysis and literature review, which presented a clear trajectory of the field’s development. The research analyzes the keyword networks for each time period (as there are only three articles in the first stage, it was not possible to generate an appropriate keyword co-occurrence map, hence only the keyword co-occurrence maps from the second to the fourth stages are provided), to understand the evolutionary track of the digital technology education application research field over time.

2000.1–2005.12: germination period

From January 2000 to December 2005, digital technology education application research was in its infancy. Only three studies focused on digital technology, all of which were related to computers. Due to the popularity of computers, the home became a new learning environment, highlighting the important role of digital technology in expanding the scope of learning spaces (Sutherland et al. 2000 ). In specific disciplines and contexts, digital technology was first favored in medical clinical practice, becoming an important tool for supporting the learning of clinical knowledge and practice (Tegtmeyer et al. 2001 ; Durfee et al. 2003 ).

2006.1–2011.12: initial development period

Between January 2006 and December 2011, it was the initial development period of digital technology education research. Significant growth was observed in research related to digital technology, and discussions and theoretical analyses about “digital natives” emerged. During this phase, scholars focused on the debate about “how to use digital technology reasonably” and “whether current educational models and school curriculum design need to be adjusted on a large scale” (Bennett and Maton, 2010 ; Selwyn, 2009 ; Margaryan et al. 2011 ). These theoretical and speculative arguments provided a unique perspective on the impact of cognitive digital technology on education and teaching. As can be seen from the vocabulary such as “rethinking”, “disruptive pedagogy”, and “attitude” in Fig. 4 , many scholars joined the calm reflection and analysis under the trend of digital technology (Laurillard, 2008 ; Vratulis et al. 2011 ). During this phase, technology was still undergoing dramatic changes. The development of mobile technology had already caught the attention of many scholars (Wong et al. 2011 ), but digital technology represented by computers was still very active (Selwyn et al. 2011 ). The change in technological form would inevitably lead to educational transformation. Collins and Halverson ( 2010 ) summarized the prospects and challenges of using digital technology for learning and educational practices, believing that digital technology would bring a disruptive revolution to the education field and bring about a new educational system. In addition, the term “teacher education” in Fig. 4 reflects the impact of digital technology development on teachers. The rapid development of technology has widened the generation gap between teachers and students. To ensure smooth communication between teachers and students, teachers must keep up with the trend of technological development and establish a lifelong learning concept (Donnison, 2009 ).

In the diagram, each node represents a keyword, with the size of the node indicating the frequency of occurrence of the keyword. The connections represent the co-occurrence relationships between keywords, with a higher frequency of co-occurrence resulting in tighter connections.

2012.1–2017.12: critical exploration period

During the period spanning January 2012 to December 2017, the application of digital technology in education research underwent a significant exploration phase. As can be seen from Fig. 5 , different from the previous stage, the specific elements of specific digital technology have started to increase significantly, including the enrichment of technological contexts, the greater variety of research methods, and the diversification of learning modes. Moreover, the temporal and spatial dimensions of the learning environment were further de-emphasized, as noted in previous literature (Za et al. 2014 ). Given the rapidly accelerating pace of technological development, the education system in the digital era is in urgent need of collaborative evolution and reconstruction, as argued by Davis, Eickelmann, and Zaka ( 2013 ).

In the domain of digital technology, social media has garnered substantial scholarly attention as a promising avenue for learning, as noted by Pasquini and Evangelopoulos ( 2016 ). The implementation of social media in education presents several benefits, including the liberation of education from the restrictions of physical distance and time, as well as the erasure of conventional educational boundaries. The user-generated content (UGC) model in social media has emerged as a crucial source for knowledge creation and distribution, with the widespread adoption of mobile devices. Moreover, social networks have become an integral component of ubiquitous learning environments (Hwang et al. 2013 ). The utilization of social media allows individuals to function as both knowledge producers and recipients, which leads to a blurring of the conventional roles of learners and teachers. On mobile platforms, the roles of learners and teachers are not fixed, but instead interchangeable.

In terms of research methodology, the prevalence of empirical studies with survey designs in the field of educational technology during this period is evident from the vocabulary used, such as “achievement,” “acceptance,” “attitude,” and “ict.” in Fig. 5 . These studies aim to understand learners’ willingness to adopt and attitudes towards new technologies, and some seek to investigate the impact of digital technologies on learning outcomes through quasi-experimental designs (Domínguez et al. 2013 ). Among these empirical studies, mobile learning emerged as a hot topic, and this is not surprising. First, the advantages of mobile learning environments over traditional ones have been empirically demonstrated (Hwang et al. 2013 ). Second, learners born around the turn of the century have been heavily influenced by digital technologies and have developed their own learning styles that are more open to mobile devices as a means of learning. Consequently, analyzing mobile learning as a relatively novel mode of learning has become an important issue for scholars in the field of educational technology.

The intervention of technology has led to the emergence of several novel learning modes, with the blended learning model being the most representative one in the current phase. Blended learning, a novel concept introduced in the information age, emphasizes the integration of the benefits of traditional learning methods and online learning. This learning mode not only highlights the prominent role of teachers in guiding, inspiring, and monitoring the learning process but also underlines the importance of learners’ initiative, enthusiasm, and creativity in the learning process. Despite being an early conceptualization, blended learning’s meaning has been expanded by the widespread use of mobile technology and social media in education. The implementation of new technologies, particularly mobile devices, has resulted in the transformation of curriculum design and increased flexibility and autonomy in students’ learning processes (Trujillo Maza et al. 2016 ), rekindling scholarly attention to this learning mode. However, some scholars have raised concerns about the potential drawbacks of the blended learning model, such as its significant impact on the traditional teaching system, the lack of systematic coping strategies and relevant policies in several schools and regions (Moskal et al. 2013 ).

2018.1–2022.12: accelerated transformation period

The period spanning from January 2018 to December 2022 witnessed a rapid transformation in the application of digital technology in education research. The field of digital technology education research reached a peak period of publication, largely influenced by factors such as the COVID-19 pandemic (Yu et al. 2023 ). Research during this period was built upon the achievements, attitudes, and social media of the previous phase, and included more elements that reflect the characteristics of this research field, such as digital literacy, digital competence, and professional development, as depicted in Fig. 6 . Alongside this, scholars’ expectations for the value of digital technology have expanded, and the pursuit of improving learning efficiency and performance is no longer the sole focus. Some research now aims to cultivate learners’ motivation and enhance their self-efficacy by applying digital technology in a reasonable manner, as demonstrated by recent studies (Beardsley et al. 2021 ; Creely et al. 2021 ).

The COVID-19 pandemic has emerged as a crucial backdrop for the digital technology’s role in sustaining global education, as highlighted by recent scholarly research (Zhou et al. 2022 ; Pan and Zhang, 2020 ; Mo et al. 2022 ). The online learning environment, which is supported by digital technology, has become the primary battleground for global education (Yu, 2022 ). This social context has led to various studies being conducted, with some scholars positing that the pandemic has impacted the traditional teaching order while also expanding learning possibilities in terms of patterns and forms (Alabdulaziz, 2021 ). Furthermore, the pandemic has acted as a catalyst for teacher teaching and technological innovation, and this viewpoint has been empirically substantiated (Moorhouse and Wong, 2021 ). Additionally, some scholars believe that the pandemic’s push is a crucial driving force for the digital transformation of the education system, serving as an essential mechanism for overcoming the system’s inertia (Romero et al. 2021 ).

The rapid outbreak of the pandemic posed a challenge to the large-scale implementation of digital technologies, which was influenced by a complex interplay of subjective and objective factors. Objective constraints included the lack of infrastructure in some regions to support digital technologies, while subjective obstacles included psychological resistance among certain students and teachers (Moorhouse, 2021 ). These factors greatly impacted the progress of online learning during the pandemic. Additionally, Timotheou et al. ( 2023 ) conducted a comprehensive systematic review of existing research on digital technology use during the pandemic, highlighting the critical role played by various factors such as learners’ and teachers’ digital skills, teachers’ personal attributes and professional development, school leadership and management, and administration in facilitating the digitalization and transformation of schools.

The current stage of research is characterized by the pivotal term “digital literacy,” denoting a growing interest in learners’ attitudes and adoption of emerging technologies. Initially, the term “literacy” was restricted to fundamental abilities and knowledge associated with books and print materials (McMillan, 1996 ). However, with the swift advancement of computers and digital technology, there have been various attempts to broaden the scope of literacy beyond its traditional meaning, including game literacy (Buckingham and Burn, 2007 ), information literacy (Eisenberg, 2008 ), and media literacy (Turin and Friesem, 2020 ). Similarly, digital literacy has emerged as a crucial concept, and Gilster and Glister ( 1997 ) were the first to introduce this concept, referring to the proficiency in utilizing technology and processing digital information in academic, professional, and daily life settings. In practical educational settings, learners who possess higher digital literacy often exhibit an aptitude for quickly mastering digital devices and applying them intelligently to education and teaching (Yu, 2022 ).

The utilization of digital technology in education has undergone significant changes over the past two decades, and has been a crucial driver of educational reform with each new technological revolution. The impact of these changes on the underlying logic of digital technology education applications has been noticeable. From computer technology to more recent developments such as virtual reality (VR), augmented reality (AR), and artificial intelligence (AI), the acceleration in digital technology development has been ongoing. Educational reforms spurred by digital technology development continue to be dynamic, as each new digital innovation presents new possibilities and models for teaching practice. This is especially relevant in the post-pandemic era, where the importance of technological progress in supporting teaching cannot be overstated (Mughal et al. 2022 ). Existing digital technologies have already greatly expanded the dimensions of education in both time and space, while future digital technologies aim to expand learners’ perceptions. Researchers have highlighted the potential of integrated technology and immersive technology in the development of the educational metaverse, which is highly anticipated to create a new dimension for the teaching and learning environment, foster a new value system for the discipline of educational technology, and more effectively and efficiently achieve the grand educational blueprint of the United Nations’ Sustainable Development Goals (Zhang et al. 2022 ; Li and Yu, 2023 ).

Hotspot evolution analysis (RQ3)

The examination of keyword evolution reveals a consistent trend in the advancement of digital technology education application research. The emergence and transformation of keywords serve as indicators of the varying research interests in this field. Thus, the utilization of the burst detection function available in CiteSpace allowed for the identification of the top 10 burst words that exhibited a high level of burst strength. This outcome is illustrated in Table 7 .

According to the results presented in Table 7 , the explosive terminology within the realm of digital technology education research has exhibited a concentration mainly between the years 2018 and 2022. Prior to this time frame, the emerging keywords were limited to “information technology” and “computer”. Notably, among them, computer, as an emergent keyword, has always had a high explosive intensity from 2008 to 2018, which reflects the important position of computer in digital technology and is the main carrier of many digital technologies such as Learning Management Systems (LMS) and Assessment and Feedback systems (Barlovits et al. 2022 ).

Since 2018, an increasing number of research studies have focused on evaluating the capabilities of learners to accept, apply, and comprehend digital technologies. As indicated by the use of terms such as “digital literacy” and “digital skill,” the assessment of learners’ digital literacy has become a critical task. Scholarly efforts have been directed towards the development of literacy assessment tools and the implementation of empirical assessments. Furthermore, enhancing the digital literacy of both learners and educators has garnered significant attention. (Nagle, 2018 ; Yu, 2022 ). Simultaneously, given the widespread use of various digital technologies in different formal and informal learning settings, promoting learners’ digital skills has become a crucial objective for contemporary schools (Nygren et al. 2019 ; Forde and OBrien, 2022 ).

Since 2020, the field of applied research on digital technology education has witnessed the emergence of three new hotspots, all of which have been affected to some extent by the pandemic. Firstly, digital technology has been widely applied in physical education, which is one of the subjects that has been severely affected by the pandemic (Parris et al. 2022 ; Jiang and Ning, 2022 ). Secondly, digital transformation has become an important measure for most schools, especially higher education institutions, to cope with the impact of the pandemic globally (García-Morales et al. 2021 ). Although the concept of digital transformation was proposed earlier, the COVID-19 pandemic has greatly accelerated this transformation process. Educational institutions must carefully redesign their educational products to face this new situation, providing timely digital learning methods, environments, tools, and support systems that have far-reaching impacts on modern society (Krishnamurthy, 2020 ; Salas-Pilco et al. 2022 ). Moreover, the professional development of teachers has become a key mission of educational institutions in the post-pandemic era. Teachers need to have a certain level of digital literacy and be familiar with the tools and online teaching resources used in online teaching, which has become a research hotspot today. Organizing digital skills training for teachers to cope with the application of emerging technologies in education is an important issue for teacher professional development and lifelong learning (Garzón-Artacho et al. 2021 ). As the main organizers and practitioners of emergency remote teaching (ERT) during the pandemic, teachers must put cognitive effort into their professional development to ensure effective implementation of ERT (Romero-Hall and Jaramillo Cherrez, 2022 ).

The burst word “digital transformation” reveals that we are in the midst of an ongoing digital technology revolution. With the emergence of innovative digital technologies such as ChatGPT and Microsoft 365 Copilot, technology trends will continue to evolve, albeit unpredictably. While the impact of these advancements on school education remains uncertain, it is anticipated that the widespread integration of technology will significantly affect the current education system. Rejecting emerging technologies without careful consideration is unwise. Like any revolution, the technological revolution in the education field has both positive and negative aspects. Detractors argue that digital technology disrupts learning and memory (Baron, 2021 ) or causes learners to become addicted and distracted from learning (Selwyn and Aagaard, 2020 ). On the other hand, the prudent use of digital technology in education offers a glimpse of a golden age of open learning. Educational leaders and practitioners have the opportunity to leverage cutting-edge digital technologies to address current educational challenges and develop a rational path for the sustainable and healthy growth of education.

Discussion on performance analysis (RQ1)

The field of digital technology education application research has experienced substantial growth since the turn of the century, a phenomenon that is quantifiably apparent through an analysis of authorship, country/region contributions, and institutional engagement. This expansion reflects the increased integration of digital technologies in educational settings and the heightened scholarly interest in understanding and optimizing their use.

Discussion on authorship productivity in digital technology education research

The authorship distribution within digital technology education research is indicative of the field’s intellectual structure and depth. A primary figure in this domain is Neil Selwyn, whose substantial citation rate underscores the profound impact of his work. His focus on the implications of digital technology in higher education and educational sociology has proven to be seminal. Selwyn’s research trajectory, especially the exploration of spatiotemporal extensions of education through technology, provides valuable insights into the multifaceted role of digital tools in learning processes (Selwyn et al. 2019 ).

Other notable contributors, like Henderson and Edwards, present diversified research interests, such as the impact of digital technologies during the pandemic and their application in early childhood education, respectively. Their varied focuses highlight the breadth of digital technology education research, encompassing pedagogical innovation, technological adaptation, and policy development.

Discussion on country/region-level productivity and collaboration

At the country/region level, the United Kingdom, specifically England, emerges as a leading contributor with 92 published papers and a significant citation count. This is closely followed by Australia and the United States, indicating a strong English-speaking research axis. Such geographical concentration of scholarly output often correlates with investment in research and development, technological infrastructure, and the prevalence of higher education institutions engaging in cutting-edge research.

China’s notable inclusion as the only non-Western country among the top contributors to the field suggests a growing research capacity and interest in digital technology in education. However, the lower average citation per paper for China could reflect emerging engagement or different research focuses that may not yet have achieved the same international recognition as Western counterparts.

The chord diagram analysis furthers this understanding, revealing dense interconnections between countries like the United States, China, and England, which indicates robust collaborations. Such collaborations are fundamental in addressing global educational challenges and shaping international research agendas.

Discussion on institutional-level contributions to digital technology education

Institutional productivity in digital technology education research reveals a constellation of universities driving the field forward. Monash University and the Australian Catholic University have the highest publication output, signaling Australia’s significant role in advancing digital education research. The University of Oslo’s remarkable average citation count per publication indicates influential research contributions, potentially reflecting high-quality studies that resonate with the broader academic community.

The strong showing of UK institutions, including the University of London, The Open University, and the University of Cambridge, reinforces the UK’s prominence in this research field. Such institutions are often at the forefront of pedagogical innovation, benefiting from established research cultures and funding mechanisms that support sustained inquiry into digital education.

Discussion on journal publication analysis

An examination of journal outputs offers a lens into the communicative channels of the field’s knowledge base. Journals such as Education and Information Technologies , Computers & Education , and the British Journal of Educational Technology not only serve as the primary disseminators of research findings but also as indicators of research quality and relevance. The impact factor (IF) serves as a proxy for the quality and influence of these journals within the academic community.

The high citation counts for articles published in Computers & Education suggest that research disseminated through this medium has a wide-reaching impact and is of particular interest to the field. This is further evidenced by its significant IF of 11.182, indicating that the journal is a pivotal platform for seminal work in the application of digital technology in education.

The authorship, regional, and institutional productivity in the field of digital technology education application research collectively narrate the evolution of this domain since the turn of the century. The prominence of certain authors and countries underscores the importance of socioeconomic factors and existing academic infrastructure in fostering research productivity. Meanwhile, the centrality of specific journals as outlets for high-impact research emphasizes the role of academic publishing in shaping the research landscape.

As the field continues to grow, future research may benefit from leveraging the collaborative networks that have been elucidated through this analysis, perhaps focusing on underrepresented regions to broaden the scope and diversity of research. Furthermore, the stabilization of publication numbers in recent years invites a deeper exploration into potential plateaus in research trends or saturation in certain sub-fields, signaling an opportunity for novel inquiries and methodological innovations.

Discussion on the evolutionary trends (RQ2)

The evolution of the research field concerning the application of digital technology in education over the past two decades is a story of convergence, diversification, and transformation, shaped by rapid technological advancements and shifting educational paradigms.

At the turn of the century, the inception of digital technology in education was largely exploratory, with a focus on how emerging computer technologies could be harnessed to enhance traditional learning environments. Research from this early period was primarily descriptive, reflecting on the potential and challenges of incorporating digital tools into the educational setting. This phase was critical in establishing the fundamental discourse that would guide subsequent research, as it set the stage for understanding the scope and impact of digital technology in learning spaces (Wang et al. 2023 ).

As the first decade progressed, the narrative expanded to encompass the pedagogical implications of digital technologies. This was a period of conceptual debates, where terms like “digital natives” and “disruptive pedagogy” entered the academic lexicon, underscoring the growing acknowledgment of digital technology as a transformative force within education (Bennett and Maton, 2010 ). During this time, the research began to reflect a more nuanced understanding of the integration of technology, considering not only its potential to change where and how learning occurred but also its implications for educational equity and access.

In the second decade, with the maturation of internet connectivity and mobile technology, the focus of research shifted from theoretical speculations to empirical investigations. The proliferation of digital devices and the ubiquity of social media influenced how learners interacted with information and each other, prompting a surge in studies that sought to measure the impact of these tools on learning outcomes. The digital divide and issues related to digital literacy became central concerns, as scholars explored the varying capacities of students and educators to engage with technology effectively.

Throughout this period, there was an increasing emphasis on the individualization of learning experiences, facilitated by adaptive technologies that could cater to the unique needs and pacing of learners (Jing et al. 2023a ). This individualization was coupled with a growing recognition of the importance of collaborative learning, both online and offline, and the role of digital tools in supporting these processes. Blended learning models, which combined face-to-face instruction with online resources, emerged as a significant trend, advocating for a balance between traditional pedagogies and innovative digital strategies.

The later years, particularly marked by the COVID-19 pandemic, accelerated the necessity for digital technology in education, transforming it from a supplementary tool to an essential platform for delivering education globally (Mo et al. 2022 ; Mustapha et al. 2021 ). This era brought about an unprecedented focus on online learning environments, distance education, and virtual classrooms. Research became more granular, examining not just the pedagogical effectiveness of digital tools, but also their role in maintaining continuity of education during crises, their impact on teacher and student well-being, and their implications for the future of educational policy and infrastructure.

Across these two decades, the research field has seen a shift from examining digital technology as an external addition to the educational process, to viewing it as an integral component of curriculum design, instructional strategies, and even assessment methods. The emergent themes have broadened from a narrow focus on specific tools or platforms to include wider considerations such as data privacy, ethical use of technology, and the environmental impact of digital tools.

Moreover, the field has moved from considering the application of digital technology in education as a primarily cognitive endeavor to recognizing its role in facilitating socio-emotional learning, digital citizenship, and global competencies. Researchers have increasingly turned their attention to the ways in which technology can support collaborative skills, cultural understanding, and ethical reasoning within diverse student populations.

In summary, the past over twenty years in the research field of digital technology applications in education have been characterized by a progression from foundational inquiries to complex analyses of digital integration. This evolution has mirrored the trajectory of technology itself, from a facilitative tool to a pervasive ecosystem defining contemporary educational experiences. As we look to the future, the field is poised to delve into the implications of emerging technologies like AI, AR, and VR, and their potential to redefine the educational landscape even further. This ongoing metamorphosis suggests that the application of digital technology in education will continue to be a rich area of inquiry, demanding continual adaptation and forward-thinking from educators and researchers alike.

Discussion on the study of research hotspots (RQ3)

The analysis of keyword evolution in digital technology education application research elucidates the current frontiers in the field, reflecting a trajectory that is in tandem with the rapidly advancing digital age. This landscape is sculpted by emergent technological innovations and shaped by the demands of an increasingly digital society.

Interdisciplinary integration and pedagogical transformation

One of the frontiers identified from recent keyword bursts includes the integration of digital technology into diverse educational contexts, particularly noted with the keyword “physical education.” The digitalization of disciplines traditionally characterized by physical presence illustrates the pervasive reach of technology and signifies a push towards interdisciplinary integration where technology is not only a facilitator but also a transformative agent. This integration challenges educators to reconceptualize curriculum delivery to accommodate digital tools that can enhance or simulate the physical aspects of learning.

Digital literacy and skills acquisition

Another pivotal frontier is the focus on “digital literacy” and “digital skill”, which has intensified in recent years. This suggests a shift from mere access to technology towards a comprehensive understanding and utilization of digital tools. In this realm, the emphasis is not only on the ability to use technology but also on critical thinking, problem-solving, and the ethical use of digital resources (Yu, 2022 ). The acquisition of digital literacy is no longer an additive skill but a fundamental aspect of modern education, essential for navigating and contributing to the digital world.

Educational digital transformation

The keyword “digital transformation” marks a significant research frontier, emphasizing the systemic changes that education institutions must undergo to align with the digital era (Romero et al. 2021 ). This transformation includes the redesigning of learning environments, pedagogical strategies, and assessment methods to harness digital technology’s full potential. Research in this area explores the complexity of institutional change, addressing the infrastructural, cultural, and policy adjustments needed for a seamless digital transition.

Engagement and participation

Further exploration into “engagement” and “participation” underscores the importance of student-centered learning environments that are mediated by technology. The current frontiers examine how digital platforms can foster collaboration, inclusivity, and active learning, potentially leading to more meaningful and personalized educational experiences. Here, the use of technology seeks to support the emotional and cognitive aspects of learning, moving beyond the transactional view of education to one that is relational and interactive.

Professional development and teacher readiness

As the field evolves, “professional development” emerges as a crucial area, particularly in light of the pandemic which necessitated emergency remote teaching. The need for teacher readiness in a digital age is a pressing frontier, with research focusing on the competencies required for educators to effectively integrate technology into their teaching practices. This includes familiarity with digital tools, pedagogical innovation, and an ongoing commitment to personal and professional growth in the digital domain.

Pandemic as a catalyst

The recent pandemic has acted as a catalyst for accelerated research and application in this field, particularly in the domains of “digital transformation,” “professional development,” and “physical education.” This period has been a litmus test for the resilience and adaptability of educational systems to continue their operations in an emergency. Research has thus been directed at understanding how digital technologies can support not only continuity but also enhance the quality and reach of education in such contexts.

Ethical and societal considerations

The frontier of digital technology in education is also expanding to consider broader ethical and societal implications. This includes issues of digital equity, data privacy, and the sociocultural impact of technology on learning communities. The research explores how educational technology can be leveraged to address inequities and create more equitable learning opportunities for all students, regardless of their socioeconomic background.

Innovation and emerging technologies

Looking forward, the frontiers are set to be influenced by ongoing and future technological innovations, such as artificial intelligence (AI) (Wu and Yu, 2023 ; Chen et al. 2022a ). The exploration into how these technologies can be integrated into educational practices to create immersive and adaptive learning experiences represents a bold new chapter for the field.

In conclusion, the current frontiers of research on the application of digital technology in education are multifaceted and dynamic. They reflect an overarching movement towards deeper integration of technology in educational systems and pedagogical practices, where the goals are not only to facilitate learning but to redefine it. As these frontiers continue to expand and evolve, they will shape the educational landscape, requiring a concerted effort from researchers, educators, policymakers, and technologists to navigate the challenges and harness the opportunities presented by the digital revolution in education.

Conclusions and future research

Conclusions.

The utilization of digital technology in education is a research area that cuts across multiple technical and educational domains and continues to experience dynamic growth due to the continuous progress of technology. In this study, a systematic review of this field was conducted through bibliometric techniques to examine its development trajectory. The primary focus of the review was to investigate the leading contributors, productive national institutions, significant publications, and evolving development patterns. The study’s quantitative analysis resulted in several key conclusions that shed light on this research field’s current state and future prospects.

(1) The research field of digital technology education applications has entered a stage of rapid development, particularly in recent years due to the impact of the pandemic, resulting in a peak of publications. Within this field, several key authors (Selwyn, Henderson, Edwards, etc.) and countries/regions (England, Australia, USA, etc.) have emerged, who have made significant contributions. International exchanges in this field have become frequent, with a high degree of internationalization in academic research. Higher education institutions in the UK and Australia are the core productive forces in this field at the institutional level.

(2) Education and Information Technologies , Computers & Education , and the British Journal of Educational Technology are notable journals that publish research related to digital technology education applications. These journals are affiliated with the research field of educational technology and provide effective communication platforms for sharing digital technology education applications.

(3) Over the past two decades, research on digital technology education applications has progressed from its early stages of budding, initial development, and critical exploration to accelerated transformation, and it is currently approaching maturity. Technological progress and changes in the times have been key driving forces for educational transformation and innovation, and both have played important roles in promoting the continuous development of education.

(4) Influenced by the pandemic, three emerging frontiers have emerged in current research on digital technology education applications, which are physical education, digital transformation, and professional development under the promotion of digital technology. These frontier research hotspots reflect the core issues that the education system faces when encountering new technologies. The evolution of research hotspots shows that technology breakthroughs in education’s original boundaries of time and space create new challenges. The continuous self-renewal of education is achieved by solving one hotspot problem after another.

The present study offers significant practical implications for scholars and practitioners in the field of digital technology education applications. Firstly, it presents a well-defined framework of the existing research in this area, serving as a comprehensive guide for new entrants to the field and shedding light on the developmental trajectory of this research domain. Secondly, the study identifies several contemporary research hotspots, thus offering a valuable decision-making resource for scholars aiming to explore potential research directions. Thirdly, the study undertakes an exhaustive analysis of published literature to identify core journals in the field of digital technology education applications, with Sustainability being identified as a promising open access journal that publishes extensively on this topic. This finding can potentially facilitate scholars in selecting appropriate journals for their research outputs.

Limitation and future research

Influenced by some objective factors, this study also has some limitations. First of all, the bibliometrics analysis software has high standards for data. In order to ensure the quality and integrity of the collected data, the research only selects the periodical papers in SCIE and SSCI indexes, which are the core collection of Web of Science database, and excludes other databases, conference papers, editorials and other publications, which may ignore some scientific research and original opinions in the field of digital technology education and application research. In addition, although this study used professional software to carry out bibliometric analysis and obtained more objective quantitative data, the analysis and interpretation of data will inevitably have a certain subjective color, and the influence of subjectivity on data analysis cannot be completely avoided. As such, future research endeavors will broaden the scope of literature screening and proactively engage scholars in the field to gain objective and state-of-the-art insights, while minimizing the adverse impact of personal subjectivity on research analysis.

Data availability

The datasets analyzed during the current study are available in the Dataverse repository: https://doi.org/10.7910/DVN/F9QMHY

Alabdulaziz MS (2021) COVID-19 and the use of digital technology in mathematics education. Educ Inf Technol 26(6):7609–7633. https://doi.org/10.1007/s10639-021-10602-3

Arif TB, Munaf U, Ul-Haque I (2023) The future of medical education and research: is ChatGPT a blessing or blight in disguise? Med Educ Online 28. https://doi.org/10.1080/10872981.2023.2181052

Banerjee M, Chiew D, Patel KT, Johns I, Chappell D, Linton N, Cole GD, Francis DP, Szram J, Ross J, Zaman S (2021) The impact of artificial intelligence on clinical education: perceptions of postgraduate trainee doctors in London (UK) and recommendations for trainers. BMC Med Educ 21. https://doi.org/10.1186/s12909-021-02870-x

Barlovits S, Caldeira A, Fesakis G, Jablonski S, Koutsomanoli Filippaki D, Lázaro C, Ludwig M, Mammana MF, Moura A, Oehler DXK, Recio T, Taranto E, Volika S(2022) Adaptive, synchronous, and mobile online education: developing the ASYMPTOTE learning environment. Mathematics 10:1628. https://doi.org/10.3390/math10101628

Article   Google Scholar  

Baron NS(2021) Know what? How digital technologies undermine learning and remembering J Pragmat 175:27–37. https://doi.org/10.1016/j.pragma.2021.01.011

Batista J, Morais NS, Ramos F (2016) Researching the use of communication technologies in higher education institutions in Portugal. https://doi.org/10.4018/978-1-5225-0571-6.ch057

Beardsley M, Albó L, Aragón P, Hernández-Leo D (2021) Emergency education effects on teacher abilities and motivation to use digital technologies. Br J Educ Technol 52. https://doi.org/10.1111/bjet.13101

Bennett S, Maton K(2010) Beyond the “digital natives” debate: towards a more nuanced understanding of students’ technology experiences J Comput Assist Learn 26:321–331. https://doi.org/10.1111/j.1365-2729.2010.00360.x

Buckingham D, Burn A (2007) Game literacy in theory and practice 16:323–349

Google Scholar  

Bulfin S, Pangrazio L, Selwyn N (2014) Making “MOOCs”: the construction of a new digital higher education within news media discourse. In: The International Review of Research in Open and Distributed Learning 15. https://doi.org/10.19173/irrodl.v15i5.1856

Camilleri MA, Camilleri AC(2016) Digital learning resources and ubiquitous technologies in education Technol Knowl Learn 22:65–82. https://doi.org/10.1007/s10758-016-9287-7

Chen C(2006) CiteSpace II: detecting and visualizing emerging trends and transient patterns in scientific literature J Am Soc Inf Sci Technol 57:359–377. https://doi.org/10.1002/asi.20317

Chen J, Dai J, Zhu K, Xu L(2022) Effects of extended reality on language learning: a meta-analysis Front Psychol 13:1016519. https://doi.org/10.3389/fpsyg.2022.1016519

Article   PubMed   PubMed Central   Google Scholar  

Chen J, Wang CL, Tang Y (2022b) Knowledge mapping of volunteer motivation: a bibliometric analysis and cross-cultural comparative study. Front Psychol 13. https://doi.org/10.3389/fpsyg.2022.883150

Cohen A, Soffer T, Henderson M(2022) Students’ use of technology and their perceptions of its usefulness in higher education: International comparison J Comput Assist Learn 38(5):1321–1331. https://doi.org/10.1111/jcal.12678

Collins A, Halverson R(2010) The second educational revolution: rethinking education in the age of technology J Comput Assist Learn 26:18–27. https://doi.org/10.1111/j.1365-2729.2009.00339.x

Conole G, Alevizou P (2010) A literature review of the use of Web 2.0 tools in higher education. Walton Hall, Milton Keynes, UK: the Open University, retrieved 17 February

Creely E, Henriksen D, Crawford R, Henderson M(2021) Exploring creative risk-taking and productive failure in classroom practice. A case study of the perceived self-efficacy and agency of teachers at one school Think Ski Creat 42:100951. https://doi.org/10.1016/j.tsc.2021.100951

Davis N, Eickelmann B, Zaka P(2013) Restructuring of educational systems in the digital age from a co-evolutionary perspective J Comput Assist Learn 29:438–450. https://doi.org/10.1111/jcal.12032

De Belli N (2009) Bibliometrics and citation analysis: from the science citation index to cybermetrics, Scarecrow Press. https://doi.org/10.1111/jcal.12032

Domínguez A, Saenz-de-Navarrete J, de-Marcos L, Fernández-Sanz L, Pagés C, Martínez-Herráiz JJ(2013) Gamifying learning experiences: practical implications and outcomes Comput Educ 63:380–392. https://doi.org/10.1016/j.compedu.2012.12.020

Donnison S (2009) Discourses in conflict: the relationship between Gen Y pre-service teachers, digital technologies and lifelong learning. Australasian J Educ Technol 25. https://doi.org/10.14742/ajet.1138

Durfee SM, Jain S, Shaffer K (2003) Incorporating electronic media into medical student education. Acad Radiol 10:205–210. https://doi.org/10.1016/s1076-6332(03)80046-6

Dzikowski P(2018) A bibliometric analysis of born global firms J Bus Res 85:281–294. https://doi.org/10.1016/j.jbusres.2017.12.054

van Eck NJ, Waltman L(2009) Software survey: VOSviewer, a computer program for bibliometric mapping Scientometrics 84:523–538 https://doi.org/10.1007/s11192-009-0146-3

Edwards S(2013) Digital play in the early years: a contextual response to the problem of integrating technologies and play-based pedagogies in the early childhood curriculum Eur Early Child Educ Res J 21:199–212. https://doi.org/10.1080/1350293x.2013.789190

Edwards S(2015) New concepts of play and the problem of technology, digital media and popular-culture integration with play-based learning in early childhood education Technol Pedagogy Educ 25:513–532 https://doi.org/10.1080/1475939x.2015.1108929

Article   MathSciNet   Google Scholar  

Eisenberg MB(2008) Information literacy: essential skills for the information age DESIDOC J Libr Inf Technol 28:39–47. https://doi.org/10.14429/djlit.28.2.166

Forde C, OBrien A (2022) A literature review of barriers and opportunities presented by digitally enhanced practical skill teaching and learning in health science education. Med Educ Online 27. https://doi.org/10.1080/10872981.2022.2068210

García-Morales VJ, Garrido-Moreno A, Martín-Rojas R (2021) The transformation of higher education after the COVID disruption: emerging challenges in an online learning scenario. Front Psychol 12. https://doi.org/10.3389/fpsyg.2021.616059

Garfield E(2006) The history and meaning of the journal impact factor JAMA 295:90. https://doi.org/10.1001/jama.295.1.90

Article   PubMed   Google Scholar  

Garzón-Artacho E, Sola-Martínez T, Romero-Rodríguez JM, Gómez-García G(2021) Teachers’ perceptions of digital competence at the lifelong learning stage Heliyon 7:e07513. https://doi.org/10.1016/j.heliyon.2021.e07513

Gaviria-Marin M, Merigó JM, Baier-Fuentes H(2019) Knowledge management: a global examination based on bibliometric analysis Technol Forecast Soc Change 140:194–220. https://doi.org/10.1016/j.techfore.2018.07.006

Gilster P, Glister P (1997) Digital literacy. Wiley Computer Pub, New York

Greenhow C, Lewin C(2015) Social media and education: reconceptualizing the boundaries of formal and informal learning Learn Media Technol 41:6–30. https://doi.org/10.1080/17439884.2015.1064954

Hawkins DT(2001) Bibliometrics of electronic journals in information science Infor Res 7(1):7–1. http://informationr.net/ir/7-1/paper120.html

Henderson M, Selwyn N, Finger G, Aston R(2015) Students’ everyday engagement with digital technology in university: exploring patterns of use and “usefulness J High Educ Policy Manag 37:308–319 https://doi.org/10.1080/1360080x.2015.1034424

Huang CK, Neylon C, Hosking R, Montgomery L, Wilson KS, Ozaygen A, Brookes-Kenworthy C (2020) Evaluating the impact of open access policies on research institutions. eLife 9. https://doi.org/10.7554/elife.57067

Hwang GJ, Tsai CC(2011) Research trends in mobile and ubiquitous learning: a review of publications in selected journals from 2001 to 2010 Br J Educ Technol 42:E65–E70. https://doi.org/10.1111/j.1467-8535.2011.01183.x

Hwang GJ, Wu PH, Zhuang YY, Huang YM(2013) Effects of the inquiry-based mobile learning model on the cognitive load and learning achievement of students Interact Learn Environ 21:338–354. https://doi.org/10.1080/10494820.2011.575789

Jiang S, Ning CF (2022) Interactive communication in the process of physical education: are social media contributing to the improvement of physical training performance. Universal Access Inf Soc, 1–10. https://doi.org/10.1007/s10209-022-00911-w

Jing Y, Zhao L, Zhu KK, Wang H, Wang CL, Xia Q(2023) Research landscape of adaptive learning in education: a bibliometric study on research publications from 2000 to 2022 Sustainability 15:3115–3115. https://doi.org/10.3390/su15043115

Jing Y, Wang CL, Chen Y, Wang H, Yu T, Shadiev R (2023b) Bibliometric mapping techniques in educational technology research: a systematic literature review. Educ Inf Technol 1–29. https://doi.org/10.1007/s10639-023-12178-6

Krishnamurthy S (2020) The future of business education: a commentary in the shadow of the Covid-19 pandemic. J Bus Res. https://doi.org/10.1016/j.jbusres.2020.05.034

Kumar S, Lim WM, Pandey N, Christopher Westland J (2021) 20 years of electronic commerce research. Electron Commer Res 21:1–40

Kyza EA, Georgiou Y(2018) Scaffolding augmented reality inquiry learning: the design and investigation of the TraceReaders location-based, augmented reality platform Interact Learn Environ 27:211–225. https://doi.org/10.1080/10494820.2018.1458039

Laurillard D(2008) Technology enhanced learning as a tool for pedagogical innovation J Philos Educ 42:521–533. https://doi.org/10.1111/j.1467-9752.2008.00658.x

Li M, Yu Z (2023) A systematic review on the metaverse-based blended English learning. Front Psychol 13. https://doi.org/10.3389/fpsyg.2022.1087508

Luo H, Li G, Feng Q, Yang Y, Zuo M (2021) Virtual reality in K-12 and higher education: a systematic review of the literature from 2000 to 2019. J Comput Assist Learn. https://doi.org/10.1111/jcal.12538

Margaryan A, Littlejohn A, Vojt G(2011) Are digital natives a myth or reality? University students’ use of digital technologies Comput Educ 56:429–440. https://doi.org/10.1016/j.compedu.2010.09.004

McMillan S(1996) Literacy and computer literacy: definitions and comparisons Comput Educ 27:161–170. https://doi.org/10.1016/s0360-1315(96)00026-7

Mo CY, Wang CL, Dai J, Jin P (2022) Video playback speed influence on learning effect from the perspective of personalized adaptive learning: a study based on cognitive load theory. Front Psychology 13. https://doi.org/10.3389/fpsyg.2022.839982

Moorhouse BL (2021) Beginning teaching during COVID-19: newly qualified Hong Kong teachers’ preparedness for online teaching. Educ Stud 1–17. https://doi.org/10.1080/03055698.2021.1964939

Moorhouse BL, Wong KM (2021) The COVID-19 Pandemic as a catalyst for teacher pedagogical and technological innovation and development: teachers’ perspectives. Asia Pac J Educ 1–16. https://doi.org/10.1080/02188791.2021.1988511

Moskal P, Dziuban C, Hartman J (2013) Blended learning: a dangerous idea? Internet High Educ 18:15–23

Mughal MY, Andleeb N, Khurram AFA, Ali MY, Aslam MS, Saleem MN (2022) Perceptions of teaching-learning force about Metaverse for education: a qualitative study. J. Positive School Psychol 6:1738–1745

Mustapha I, Thuy Van N, Shahverdi M, Qureshi MI, Khan N (2021) Effectiveness of digital technology in education during COVID-19 pandemic. a bibliometric analysis. Int J Interact Mob Technol 15:136

Nagle J (2018) Twitter, cyber-violence, and the need for a critical social media literacy in teacher education: a review of the literature. Teach Teach Education 76:86–94

Nazare J, Woolf A, Sysoev I, Ballinger S, Saveski M, Walker M, Roy D (2022) Technology-assisted coaching can increase engagement with learning technology at home and caregivers’ awareness of it. Comput Educ 188:104565

Nguyen UP, Hallinger P (2020) Assessing the distinctive contributions of simulation & gaming to the literature, 1970-2019: a bibliometric review. Simul Gaming 104687812094156. https://doi.org/10.1177/1046878120941569

Nygren H, Nissinen K, Hämäläinen R, Wever B(2019) Lifelong learning: formal, non-formal and informal learning in the context of the use of problem-solving skills in technology-rich environments Br J Educ Technol 50:1759–1770. https://doi.org/10.1111/bjet.12807

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Moher D (2021) The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Int J Surg 88:105906

Pan SL, Zhang S(2020) From fighting COVID-19 pandemic to tackling sustainable development goals: an opportunity for responsible information systems research Int J Inf Manage 55:102196. https://doi.org/10.1016/j.ijinfomgt.2020.102196

Pan X, Yan E, Cui M, Hua W(2018) Examining the usage, citation, and diffusion patterns of bibliometric mapping software: a comparative study of three tools J Informetr 12:481–493. https://doi.org/10.1016/j.joi.2018.03.005

Parris Z, Cale L, Harris J, Casey A (2022) Physical activity for health, covid-19 and social media: what, where and why?. Movimento, 28. https://doi.org/10.22456/1982-8918.122533

Pasquini LA, Evangelopoulos N (2016) Sociotechnical stewardship in higher education: a field study of social media policy documents. J Comput High Educ 29:218–239

Pérez-Sanagustín M, Hernández-Leo D, Santos P, Delgado Kloos C, Blat J(2014) Augmenting reality and formality of informal and non-formal settings to enhance blended learning IEEE Trans Learn Technol 7:118–131. https://doi.org/10.1109/TLT.2014.2312719

Pinto M, Leite C (2020) Digital technologies in support of students learning in Higher Education: literature review. Digital Education Review 343–360. https://doi.org/10.1344/der.2020.37.343-360

Pires F, Masanet MJ, Tomasena JM, Scolari CA(2022) Learning with YouTube: beyond formal and informal through new actors, strategies and affordances Convergence 28(3):838–853. https://doi.org/10.1177/1354856521102054

Pritchard A (1969) Statistical bibliography or bibliometrics 25:348

Romero M, Romeu T, Guitert M, Baztán P (2021) Digital transformation in higher education: the UOC case. In ICERI2021 Proceedings (pp. 6695–6703). IATED https://doi.org/10.21125/iceri.2021.1512

Romero-Hall E, Jaramillo Cherrez N (2022) Teaching in times of disruption: faculty digital literacy in higher education during the COVID-19 pandemic. Innovations in Education and Teaching International 1–11. https://doi.org/10.1080/14703297.2022.2030782

Rospigliosi PA(2023) Artificial intelligence in teaching and learning: what questions should we ask of ChatGPT? Interactive Learning Environments 31:1–3. https://doi.org/10.1080/10494820.2023.2180191

Salas-Pilco SZ, Yang Y, Zhang Z(2022) Student engagement in online learning in Latin American higher education during the COVID-19 pandemic: a systematic review. Br J Educ Technol 53(3):593–619. https://doi.org/10.1111/bjet.13190

Selwyn N(2009) The digital native-myth and reality In Aslib proceedings 61(4):364–379. https://doi.org/10.1108/00012530910973776

Selwyn N(2012) Making sense of young people, education and digital technology: the role of sociological theory Oxford Review of Education 38:81–96. https://doi.org/10.1080/03054985.2011.577949

Selwyn N, Facer K(2014) The sociology of education and digital technology: past, present and future Oxford Rev Educ 40:482–496. https://doi.org/10.1080/03054985.2014.933005

Selwyn N, Banaji S, Hadjithoma-Garstka C, Clark W(2011) Providing a platform for parents? Exploring the nature of parental engagement with school Learning Platforms J Comput Assist Learn 27:314–323. https://doi.org/10.1111/j.1365-2729.2011.00428.x

Selwyn N, Aagaard J (2020) Banning mobile phones from classrooms-an opportunity to advance understandings of technology addiction, distraction and cyberbullying. Br J Educ Technol 52. https://doi.org/10.1111/bjet.12943

Selwyn N, O’Neill C, Smith G, Andrejevic M, Gu X (2021) A necessary evil? The rise of online exam proctoring in Australian universities. Media Int Austr 1329878X2110058. https://doi.org/10.1177/1329878x211005862

Selwyn N, Pangrazio L, Nemorin S, Perrotta C (2019) What might the school of 2030 be like? An exercise in social science fiction. Learn, Media Technol 1–17. https://doi.org/10.1080/17439884.2020.1694944

Selwyn, N (2016) What works and why?* Understanding successful technology enabled learning within institutional contexts 2016 Final report Appendices (Part B). Monash University Griffith University

Sjöberg D, Holmgren R (2021) Informal workplace learning in swedish police education-a teacher perspective. Vocations and Learning. https://doi.org/10.1007/s12186-021-09267-3

Strotmann A, Zhao D (2012) Author name disambiguation: what difference does it make in author-based citation analysis? J Am Soc Inf Sci Technol 63:1820–1833

Article   CAS   Google Scholar  

Sutherland R, Facer K, Furlong R, Furlong J(2000) A new environment for education? The computer in the home. Comput Educ 34:195–212. https://doi.org/10.1016/s0360-1315(99)00045-7

Szeto E, Cheng AY-N, Hong J-C(2015) Learning with social media: how do preservice teachers integrate YouTube and Social Media in teaching? Asia-Pac Educ Res 25:35–44. https://doi.org/10.1007/s40299-015-0230-9

Tang E, Lam C(2014) Building an effective online learning community (OLC) in blog-based teaching portfolios Int High Educ 20:79–85. https://doi.org/10.1016/j.iheduc.2012.12.002

Taskin Z, Al U(2019) Natural language processing applications in library and information science Online Inf Rev 43:676–690. https://doi.org/10.1108/oir-07-2018-0217

Tegtmeyer K, Ibsen L, Goldstein B(2001) Computer-assisted learning in critical care: from ENIAC to HAL Crit Care Med 29:N177–N182. https://doi.org/10.1097/00003246-200108001-00006

Article   CAS   PubMed   Google Scholar  

Timotheou S, Miliou O, Dimitriadis Y, Sobrino SV, Giannoutsou N, Cachia R, Moné AM, Ioannou A(2023) Impacts of digital technologies on education and factors influencing schools' digital capacity and transformation: a literature review. Educ Inf Technol 28(6):6695–6726. https://doi.org/10.1007/s10639-022-11431-8

Trujillo Maza EM, Gómez Lozano MT, Cardozo Alarcón AC, Moreno Zuluaga L, Gamba Fadul M (2016) Blended learning supported by digital technology and competency-based medical education: a case study of the social medicine course at the Universidad de los Andes, Colombia. Int J Educ Technol High Educ 13. https://doi.org/10.1186/s41239-016-0027-9

Turin O, Friesem Y(2020) Is that media literacy?: Israeli and US media scholars’ perceptions of the field J Media Lit Educ 12:132–144. https://doi.org/10.1007/s11192-009-0146-3

Van Eck NJ, Waltman L (2019) VOSviewer manual. Universiteit Leiden

Vratulis V, Clarke T, Hoban G, Erickson G(2011) Additive and disruptive pedagogies: the use of slowmation as an example of digital technology implementation Teach Teach Educ 27:1179–1188. https://doi.org/10.1016/j.tate.2011.06.004

Wang CL, Dai J, Xu LJ (2022) Big data and data mining in education: a bibliometrics study from 2010 to 2022. In 2022 7th International Conference on Cloud Computing and Big Data Analytics ( ICCCBDA ) (pp. 507-512). IEEE. https://doi.org/10.1109/icccbda55098.2022.9778874

Wang CL, Dai J, Zhu KK, Yu T, Gu XQ (2023) Understanding the continuance intention of college students toward new E-learning spaces based on an integrated model of the TAM and TTF. Int J Hum-Comput Int 1–14. https://doi.org/10.1080/10447318.2023.2291609

Wong L-H, Boticki I, Sun J, Looi C-K(2011) Improving the scaffolds of a mobile-assisted Chinese character forming game via a design-based research cycle Comput Hum Behav 27:1783–1793. https://doi.org/10.1016/j.chb.2011.03.005

Wu R, Yu Z (2023) Do AI chatbots improve students learning outcomes? Evidence from a meta-analysis. Br J Educ Technol. https://doi.org/10.1111/bjet.13334

Yang D, Zhou J, Shi D, Pan Q, Wang D, Chen X, Liu J (2022) Research status, hotspots, and evolutionary trends of global digital education via knowledge graph analysis. Sustainability 14:15157–15157. https://doi.org/10.3390/su142215157

Yu T, Dai J, Wang CL (2023) Adoption of blended learning: Chinese university students’ perspectives. Humanit Soc Sci Commun 10:390. https://doi.org/10.3390/su142215157

Yu Z (2022) Sustaining student roles, digital literacy, learning achievements, and motivation in online learning environments during the COVID-19 pandemic. Sustainability 14:4388. https://doi.org/10.3390/su14084388

Za S, Spagnoletti P, North-Samardzic A(2014) Organisational learning as an emerging process: the generative role of digital tools in informal learning practices Br J Educ Technol 45:1023–1035. https://doi.org/10.1111/bjet.12211

Zhang X, Chen Y, Hu L, Wang Y (2022) The metaverse in education: definition, framework, features, potential applications, challenges, and future research topics. Front Psychol 13:1016300. https://doi.org/10.3389/fpsyg.2022.1016300

Zhou M, Dzingirai C, Hove K, Chitata T, Mugandani R (2022) Adoption, use and enhancement of virtual learning during COVID-19. Education and Information Technologies. https://doi.org/10.1007/s10639-022-10985-x

Download references

Acknowledgements

This research was supported by the Zhejiang Provincial Social Science Planning Project, “Mechanisms and Pathways for Empowering Classroom Teaching through Learning Spaces under the Strategy of High-Quality Education Development”, the 2022 National Social Science Foundation Education Youth Project “Research on the Strategy of Creating Learning Space Value and Empowering Classroom Teaching under the background of ‘Double Reduction’” (Grant No. CCA220319) and the National College Student Innovation and Entrepreneurship Training Program of China (Grant No. 202310337023).

Author information

Authors and affiliations.

College of Educational Science and Technology, Zhejiang University of Technology, Zhejiang, China

Chengliang Wang, Xiaojiao Chen, Yidan Liu & Yuhui Jing

Graduate School of Business, Universiti Sains Malaysia, Minden, Malaysia

Department of Management, The Chinese University of Hong Kong, Hong Kong, China

College of Humanities and Social Sciences, Beihang University, Beijing, China

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization: Y.J., C.W.; methodology, C.W.; software, C.W., Y.L.; writing-original draft preparation, C.W., Y.L.; writing-review and editing, T.Y., Y.L., C.W.; supervision, X.C., T.Y.; project administration, Y.J.; funding acquisition, X.C., Y.L. All authors read and approved the final manuscript. All authors have read and approved the re-submission of the manuscript.

Corresponding author

Correspondence to Yuhui Jing .

Ethics declarations

Ethical approval.

Ethical approval was not required as the study did not involve human participants.

Informed consent

Informed consent was not required as the study did not involve human participants.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Wang, C., Chen, X., Yu, T. et al. Education reform and change driven by digital technology: a bibliometric study from a global perspective. Humanit Soc Sci Commun 11 , 256 (2024). https://doi.org/10.1057/s41599-024-02717-y

Download citation

Received : 11 July 2023

Accepted : 17 January 2024

Published : 12 February 2024

DOI : https://doi.org/10.1057/s41599-024-02717-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

• Open access
• Published: 10 March 2020

Research and trends in STEM education: a systematic review of journal publications

• Yeping Li 1 ,
• Ke Wang 2 ,
• Yu Xiao 1 &
• Jeffrey E. Froyd 3  

International Journal of STEM Education volume  7 , Article number:  11 ( 2020 ) Cite this article

164k Accesses

149 Citations

5 Altmetric

Metrics details

With the rapid increase in the number of scholarly publications on STEM education in recent years, reviews of the status and trends in STEM education research internationally support the development of the field. For this review, we conducted a systematic analysis of 798 articles in STEM education published between 2000 and the end of 2018 in 36 journals to get an overview about developments in STEM education scholarship. We examined those selected journal publications both quantitatively and qualitatively, including the number of articles published, journals in which the articles were published, authorship nationality, and research topic and methods over the years. The results show that research in STEM education is increasing in importance internationally and that the identity of STEM education journals is becoming clearer over time.

Introduction

A recent review of 144 publications in the International Journal of STEM Education ( IJ - STEM ) showed how scholarship in science, technology, engineering, and mathematics (STEM) education developed between August 2014 and the end of 2018 through the lens of one journal (Li, Froyd, & Wang, 2019 ). The review of articles published in only one journal over a short period of time prompted the need to review the status and trends in STEM education research internationally by analyzing articles published in a wider range of journals over a longer period of time.

With global recognition of the growing importance of STEM education, we have witnessed the urgent need to support research and scholarship in STEM education (Li, 2014 , 2018a ). Researchers and educators have responded to this on-going call and published their scholarly work through many different publication outlets including journals, books, and conference proceedings. A simple Google search with the term “STEM,” “STEM education,” or “STEM education research” all returned more than 450,000,000 items. Such voluminous information shows the rapidly evolving and vibrant field of STEM education and sheds light on the volume of STEM education research. In any field, it is important to know and understand the status and trends in scholarship for the field to develop and be appropriately supported. This applies to STEM education.

Conducting systematic reviews to explore the status and trends in specific disciplines is common in educational research. For example, researchers surveyed the historical development of research in mathematics education (Kilpatrick, 1992 ) and studied patterns in technology usage in mathematics education (Bray & Tangney, 2017 ; Sokolowski, Li, & Willson, 2015 ). In science education, Tsai and his colleagues have conducted a sequence of reviews of journal articles to synthesize research trends in every 5 years since 1998 (i.e., 1998–2002, 2003–2007, 2008–2012, and 2013–2017), based on publications in three main science education journals including, Science Education , the International Journal of Science Education , and the Journal of Research in Science Teaching (e.g., Lin, Lin, Potvin, & Tsai, 2019 ; Tsai & Wen, 2005 ). Erduran, Ozdem, and Park ( 2015 ) reviewed argumentation in science education research from 1998 to 2014 and Minner, Levy, and Century ( 2010 ) reviewed inquiry-based science instruction between 1984 and 2002. There are also many literature reviews and syntheses in engineering and technology education (e.g., Borrego, Foster, & Froyd, 2015 ; Xu, Williams, Gu, & Zhang, 2019 ). All of these reviews have been well received in different fields of traditional disciplinary education as they critically appraise and summarize the state-of-art of relevant research in a field in general or with a specific focus. Both types of reviews have been conducted with different methods for identifying, collecting, and analyzing relevant publications, and they differ in terms of review aim and topic scope, time period, and ways of literature selection. In this review, we systematically analyze journal publications in STEM education research to overview STEM education scholarship development broadly and globally.

The complexity and ambiguity of examining the status and trends in STEM education research

A review of research development in a field is relatively straight forward, when the field is mature and its scope can be well defined. Unlike discipline-based education research (DBER, National Research Council, 2012 ), STEM education is not a well-defined field. Conducting a comprehensive literature review of STEM education research require careful thought and clearly specified scope to tackle the complexity naturally associated with STEM education. In the following sub-sections, we provide some further discussion.

Diverse perspectives about STEM and STEM education

STEM education as explicated by the term does not have a long history. The interest in helping students learn across STEM fields can be traced back to the 1990s when the US National Science Foundation (NSF) formally included engineering and technology with science and mathematics in undergraduate and K-12 school education (e.g., National Science Foundation, 1998 ). It coined the acronym SMET (science, mathematics, engineering, and technology) that was subsequently used by other agencies including the US Congress (e.g., United States Congress House Committee on Science, 1998 ). NSF also coined the acronym STEM to replace SMET (e.g., Christenson, 2011 ; Chute, 2009 ) and it has become the acronym of choice. However, a consensus has not been reached on the disciplines included within STEM.

To clarify its intent, NSF published a list of approved fields it considered under the umbrella of STEM (see http://bit.ly/2Bk1Yp5 ). The list not only includes disciplines widely considered under the STEM tent (called “core” disciplines, such as physics, chemistry, and materials research), but also includes disciplines in psychology and social sciences (e.g., political science, economics). However, NSF’s list of STEM fields is inconsistent with other federal agencies. Gonzalez and Kuenzi ( 2012 ) noted that at least two US agencies, the Department of Homeland Security and Immigration and Customs Enforcement, use a narrower definition that excludes social sciences. Researchers also view integration across different disciplines of STEM differently using various terms such as, multidisciplinary, interdisciplinary, and transdisciplinary (Vasquez, Sneider, & Comer, 2013 ). These are only two examples of the ambiguity and complexity in describing and specifying what constitutes STEM.

Multiple perspectives about the meaning of STEM education adds further complexity to determining the extent to which scholarly activity can be categorized as STEM education. For example, STEM education can be viewed with a broad and inclusive perspective to include education in the individual disciplines of STEM, i.e., science education, technology education, engineering education, and mathematics education, as well as interdisciplinary or cross-disciplinary combinations of the individual STEM disciplines (English, 2016 ; Li, 2014 ). On the other hand, STEM education can be viewed by others as referring only to interdisciplinary or cross-disciplinary combinations of the individual STEM disciplines (Honey, Pearson, & Schweingruber, 2014 ; Johnson, Peters-Burton, & Moore, 2015 ; Kelley & Knowles, 2016 ; Li, 2018a ). These multiple perspectives allow scholars to publish articles in a vast array and diverse journals, as long as journals are willing to take the position as connected with STEM education. At the same time, however, the situation presents considerable challenges for researchers intending to locate, identify, and classify publications as STEM education research. To tackle such challenges, we tried to find out what we can learn from prior reviews related to STEM education.

Guidance from prior reviews related to STEM education

A search for reviews of STEM education research found multiple reviews that could suggest approaches for identifying publications (e.g., Brown, 2012 ; Henderson, Beach, & Finkelstein, 2011 ; Kim, Sinatra, & Seyranian, 2018 ; Margot & Kettler, 2019 ; Minichiello, Hood, & Harkness, 2018 ; Mizell & Brown, 2016 ; Thibaut et al., 2018 ; Wu & Rau, 2019 ). The review conducted by Brown ( 2012 ) examined the research base of STEM education. He addressed the complexity and ambiguity by confining the review with publications in eight journals, two in each individual discipline, one academic research journal (e.g., the Journal of Research in Science Teaching ) and one practitioner journal (e.g., Science Teacher ). Journals were selected based on suggestions from some faculty members and K-12 teachers. Out of 1100 articles published in these eight journals from January 1, 2007, to October 1, 2010, Brown located 60 articles that authors self-identified as connected to STEM education. He found that the vast majority of these 60 articles focused on issues beyond an individual discipline and there was a research base forming for STEM education. In a follow-up study, Mizell and Brown ( 2016 ) reviewed articles published from January 2013 to October 2015 in the same eight journals plus two additional journals. Mizell and Brown used the same criteria to identify and include articles that authors self-identified as connected to STEM education, i.e., if the authors included STEM in the title or author-supplied keywords. In comparison to Brown’s findings, they found that many more STEM articles were published in a shorter time period and by scholars from many more different academic institutions. Taking together, both Brown ( 2012 ) and Mizell and Brown ( 2016 ) tended to suggest that STEM education mainly consists of interdisciplinary or cross-disciplinary combinations of the individual STEM disciplines, but their approach consisted of selecting a limited number of individual discipline-based journals and then selecting articles that authors self-identified as connected to STEM education.

In contrast to reviews on STEM education, in general, other reviews focused on specific issues in STEM education (e.g., Henderson et al., 2011 ; Kim et al., 2018 ; Margot & Kettler, 2019 ; Minichiello et al., 2018 ; Schreffler, Vasquez III, Chini, & James, 2019 ; Thibaut et al., 2018 ; Wu & Rau, 2019 ). For example, the review by Henderson et al. ( 2011 ) focused on instructional change in undergraduate STEM courses based on 191 conceptual and empirical journal articles published between 1995 and 2008. Margot and Kettler ( 2019 ) focused on what is known about teachers’ values, beliefs, perceived barriers, and needed support related to STEM education based on 25 empirical journal articles published between 2000 and 2016. The focus of these reviews allowed the researchers to limit the number of articles considered, and they typically used keyword searches of selected databases to identify articles on STEM education. Some researchers used this approach to identify publications from journals only (e.g., Henderson et al., 2011 ; Margot & Kettler, 2019 ; Schreffler et al., 2019 ), and others selected and reviewed publications beyond journals (e.g., Minichiello et al., 2018 ; Thibaut et al., 2018 ; Wu & Rau, 2019 ).

The discussion in this section suggests possible reasons contributing to the absence of a general literature review of STEM education research and development: (1) diverse perspectives in existence about STEM and STEM education that contribute to the difficulty of specifying a scope of literature review, (2) its short but rapid development history in comparison to other discipline-based education (e.g., science education), and (3) difficulties in deciding how to establish the scope of the literature review. With respect to the third reason, prior reviews have used one of two approaches to identify and select articles: (a) identifying specific journals first and then searching and selecting specific articles from these journals (e.g., Brown, 2012 ; Erduran et al., 2015 ; Mizell & Brown, 2016 ) and (b) conducting selected database searches with keywords based on a specific focus (e.g., Margot & Kettler, 2019 ; Thibaut et al., 2018 ). However, neither the first approach of selecting a limited number of individual discipline-based journals nor the second approach of selecting a specific focus for the review leads to an approach that provides a general overview of STEM education scholarship development based on existing journal publications.

Current review

Two issues were identified in setting the scope for this review.

What time period should be considered?

What publications will be selected for review?

Time period

We start with the easy one first. As discussed above, the acronym STEM did exist until the early 2000s. Although the existence of the acronym does not generate scholarship on student learning in STEM disciplines, it is symbolic and helps focus attention to efforts in STEM education. Since we want to examine the status and trends in STEM education, it is reasonable to start with the year 2000. Then, we can use the acronym of STEM as an identifier in locating specific research articles in a way as done by others (e.g., Brown, 2012 ; Mizell & Brown, 2016 ). We chose the end of 2018 as the end of the time period for our review that began during 2019.

Focusing on publications beyond individual discipline-based journals

As mentioned before, scholars responded to the call for scholarship development in STEM education with publications that appeared in various outlets and diverse languages, including journals, books, and conference proceedings. However, journal publications are typically credited and valued as one of the most important outlets for research exchange (e.g., Erduran et al., 2015 ; Henderson et al., 2011 ; Lin et al., 2019 ; Xu et al., 2019 ). Thus, in this review, we will also focus on articles published in journals in English.

The discourse above on the complexity and ambiguity regarding STEM education suggests that scholars may publish their research in a wide range of journals beyond individual discipline-based journals. To search and select articles from a wide range of journals, we thought about the approach of searching selected databases with keywords as other scholars used in reviewing STEM education with a specific focus. However, existing journals in STEM education do not have a long history. In fact, IJ-STEM is the first journal in STEM education that has just been accepted into the Social Sciences Citation Index (SSCI) (Li, 2019a ). Publications in many STEM education journals are practically not available in several important and popular databases, such as the Web of Science and Scopus. Moreover, some journals in STEM education were not normalized due to a journal’s name change or irregular publication schedule. For example, the Journal of STEM Education was named as Journal of SMET Education when it started in 2000 in a print format, and the journal’s name was not changed until 2003, Vol 4 (3 and 4), and also went fully on-line starting 2004 (Raju & Sankar, 2003 ). A simple Google Scholar search with keywords will not be able to provide accurate information, unless you visit the journal’s website to check all publications over the years. Those added complexities prevented us from taking the database search as a viable approach. Thus, we decided to identify journals first and then search and select articles from these journals. Further details about the approach are provided in the “ Method ” section.

Research questions

Given a broader range of journals and a longer period of time to be covered in this review, we can examine some of the same questions as the IJ-STEM review (Li, Froyd, & Wang, 2019 ), but we do not have access to data on readership, articles accessed, or articles cited for the other journals selected for this review. Specifically, we are interested in addressing the following six research questions:

What were the status and trends in STEM education research from 2000 to the end of 2018 based on journal publications?

What were the patterns of publications in STEM education research across different journals?

Which countries or regions, based on the countries or regions in which authors were located, contributed to journal publications in STEM education?

What were the patterns of single-author and multiple-author publications in STEM education?

What main topics had emerged in STEM education research based on the journal publications?

What research methods did authors tend to use in conducting STEM education research?

Based on the above discussion, we developed the methods for this literature review to follow careful sequential steps to identify journals first and then identify and select STEM education research articles published in these journals from January 2000 to the end of 2018. The methods should allow us to obtain a comprehensive overview about the status and trends of STEM education research based on a systematic analysis of related publications from a broad range of journals and over a longer period of time.

Identifying journals

We used the following three steps to search and identify journals for inclusion:

We assumed articles on research in STEM education have been published in journals that involve more than one traditional discipline. Thus, we used Google to search and identify all education journals with their titles containing either two, three, or all four disciplines of STEM. For example, we did Google search of all the different combinations of three areas of science, mathematics, technology Footnote 1 , and engineering as contained in a journal’s title. In addition, we also searched possible journals containing the word STEAM in the title.

Since STEM education may be viewed as encompassing discipline-based education research, articles on STEM education research may have been published in traditional discipline-based education journals, such as the Journal of Research in Science Teaching . However, there are too many such journals. Yale’s Poorvu Center for Teaching and Learning has listed 16 journals that publish articles spanning across undergraduate STEM education disciplines (see https://poorvucenter.yale.edu/FacultyResources/STEMjournals ). Thus, we selected from the list some individual discipline-based education research journals, and also added a few more common ones such as the Journal of Engineering Education .

Since articles on research in STEM education have appeared in some general education research journals, especially those well-established ones. Thus, we identified and selected a few of those journals that we noticed some publications in STEM education research.

Following the above three steps, we identified 45 journals (see Table  1 ).

Identifying articles

In this review, we will not discuss or define the meaning of STEM education. We used the acronym STEM (or STEAM, or written as the phrase of “science, technology, engineering, and mathematics”) as a term in our search of publication titles and/or abstracts. To identify and select articles for review, we searched all items published in those 45 journals and selected only those articles that author(s) self-identified with the acronym STEM (or STEAM, or written as the phrase of “science, technology, engineering, and mathematics”) in the title and/or abstract. We excluded publications in the sections of practices, letters to editors, corrections, and (guest) editorials. Our search found 798 publications that authors self-identified as in STEM education, identified from 36 journals. The remaining 9 journals either did not have publications that met our search terms or published in another language other than English (see the two separate lists in Table 1 ).

Data analysis

To address research question 3, we analyzed authorship to examine which countries/regions contributed to STEM education research over the years. Because each publication may have either one or multiple authors, we used two different methods to analyze authorship nationality that have been recognized as valuable from our review of IJ-STEM publications (Li, Froyd, & Wang, 2019 ). The first method considers only the corresponding author’s (or the first author, if no specific indication is given about the corresponding author) nationality and his/her first institution affiliation, if multiple institution affiliations are listed. Method 2 considers every author of a publication, using the following formula (Howard, Cole, & Maxwell, 1987 ) to quantitatively assign and estimate each author’s contribution to a publication (and thus associated institution’s productivity), when multiple authors are included in a publication. As an example, each publication is given one credit point. For the publication co-authored by two, the first author would be given 0.6 and the second author 0.4 credit point. For an article contributed jointly by three authors, the three authors would be credited with scores of 0.47, 0.32, and 0.21, respectively.

After calculating all the scores for each author of each paper, we added all the credit scores together in terms of each author’s country/region. For brevity, we present only the top 10 countries/regions in terms of their total credit scores calculated using these two different methods, respectively.

To address research question 5, we used the same seven topic categories identified and used in our review of IJ-STEM publications (Li, Froyd, & Wang, 2019 ). We tested coding 100 articles first to ensure the feasibility. Through test-coding and discussions, we found seven topic categories could be used to examine and classify all 798 items.

K-12 teaching, teacher, and teacher education in STEM (including both pre-service and in-service teacher education)

Post-secondary teacher and teaching in STEM (including faculty development, etc.)

K-12 STEM learner, learning, and learning environment

Post-secondary STEM learner, learning, and learning environments (excluding pre-service teacher education)

Policy, curriculum, evaluation, and assessment in STEM (including literature review about a field in general)

Culture and social and gender issues in STEM education

History, epistemology, and perspectives about STEM and STEM education

To address research question 6, we coded all 798 publications in terms of (1) qualitative methods, (2) quantitative methods, (3) mixed methods, and (4) non-empirical studies (including theoretical or conceptual papers, and literature reviews). We assigned each publication to only one research topic and one method, following the process used in the IJ-STEM review (Li, Froyd, & Wang, 2019 ). When there was more than one topic or method that could have been used for a publication, a decision was made in choosing and assigning a topic or a method. The agreement between two coders for all 798 publications was 89.5%. When topic and method coding discrepancies occurred, a final decision was reached after discussion.

Results and discussion

In the following sections, we report findings as corresponding to each of the six research questions.

The status and trends of journal publications in STEM education research from 2000 to 2018

Figure  1 shows the number of publications per year. As Fig.  1 shows, the number of publications increased each year beginning in 2010. There are noticeable jumps from 2015 to 2016 and from 2017 to 2018. The result shows that research in STEM education had grown significantly since 2010, and the most recent large number of STEM education publications also suggests that STEM education research gained its own recognition by many different journals for publication as a hot and important topic area.

The distribution of STEM education publications over the years

Among the 798 articles, there were 549 articles with the word “STEM” (or STEAM, or written with the phrase of “science, technology, engineering, and mathematics”) included in the article’s title or both title and abstract and 249 articles without such identifiers included in the title but abstract only. The results suggest that many scholars tended to include STEM in the publications’ titles to highlight their research in or about STEM education. Figure  2 shows the number of publications per year where publications are distinguished depending on whether they used the term STEM in the title or only in the abstract. The number of publications in both categories had significant increases since 2010. Use of the acronym STEM in the title was growing at a faster rate than using the acronym only in the abstract.

The trends of STEM education publications with vs. without STEM included in the title

Not all the publications that used the acronym STEM in the title and/or abstract reported on a study involving all four STEM areas. For each publication, we further examined the number of the four areas involved in the reported study.

Figure  3 presents the number of publications categorized by the number of the four areas involved in the study, breaking down the distribution of these 798 publications in terms of the content scope being focused on. Studies involving all four STEM areas are the most numerous with 488 (61.2%) publications, followed by involving one area (141, 17.7%), then studies involving both STEM and non-STEM (84, 10.5%), and finally studies involving two or three areas of STEM (72, 9%; 13, 1.6%; respectively). Publications that used the acronym STEAM in either the title or abstract were classified as involving both STEM and non-STEM. For example, both of the following publications were included in this category.

Dika and D’Amico ( 2016 ). “Early experiences and integration in the persistence of first-generation college students in STEM and non-STEM majors.” Journal of Research in Science Teaching , 53 (3), 368–383. (Note: this article focused on early experience in both STEM and Non-STEM majors.)

Sochacka, Guyotte, and Walther ( 2016 ). “Learning together: A collaborative autoethnographic exploration of STEAM (STEM+ the Arts) education.” Journal of Engineering Education , 105 (1), 15–42. (Note: this article focused on STEAM (both STEM and Arts).)

Publication distribution in terms of content scope being focused on. (Note: 1=single subject of STEM, 2=two subjects of STEM, 3=three subjects of STEM, 4=four subjects of STEM, 5=topics related to both STEM and non-STEM)

Figure  4 presents the number of publications per year in each of the five categories described earlier (category 1, one area of STEM; category 2, two areas of STEM; category 3, three areas of STEM; category 4, four areas of STEM; category 5, STEM and non-STEM). The category that had grown most rapidly since 2010 is the one involving all four areas. Recent growth in the number of publications in category 1 likely reflected growing interest of traditional individual disciplinary based educators in developing and sharing multidisciplinary and interdisciplinary scholarship in STEM education, as what was noted recently by Li and Schoenfeld ( 2019 ) with publications in IJ-STEM.

Publication distribution in terms of content scope being focused on over the years

Patterns of publications across different journals

Among the 36 journals that published STEM education articles, two are general education research journals (referred to as “subject-0”), 12 with their titles containing one discipline of STEM (“subject-1”), eight with journal’s titles covering two disciplines of STEM (“subject-2”), six covering three disciplines of STEM (“subject-3”), seven containing the word STEM (“subject-4”), and one in STEAM education (“subject-5”).

Table  2 shows that both subject-0 and subject-1 journals were usually mature journals with a long history, and they were all traditional subscription-based journals, except the Journal of Pre - College Engineering Education Research , a subject-1 journal established in 2011 that provided open access (OA). In comparison to subject-0 and subject-1 journals, subject-2 and subject-3 journals were relatively newer but still had quite many years of history on average. There are also some more journals in these two categories that provided OA. Subject-4 and subject-5 journals had a short history, and most provided OA. The results show that well-established journals had tended to focus on individual disciplines or education research in general. Multidisciplinary and interdisciplinary education journals were started some years later, followed by the recent establishment of several STEM or STEAM journals.

Table 2 also shows that subject-1, subject-2, and subject-4 journals published approximately a quarter each of the publications. The number of publications in subject-1 journals is interested, because we selected a relatively limited number of journals in this category. There are many other journals in the subject-1 category (as well as subject-0 journals) that we did not select, and thus it is very likely that we did not include some STEM education articles published in subject-0 or subject-1 journals that we did not include in our study.

Figure  5 shows the number of publications per year in each of the five categories described earlier (subject-0 through subject-5). The number of publications per year in subject-5 and subject-0 journals did not change much over the time period of the study. On the other hand, the number of publications per year in subject-4 (all 4 areas), subject-1 (single area), and subject-2 journals were all over 40 by the end of the study period. The number of publications per year in subject-3 journals increased but remained less than 30. At first sight, it may be a bit surprising that the number of publications in STEM education per year in subject-1 journals increased much faster than those in subject-2 journals over the past few years. However, as Table 2 indicates these journals had long been established with great reputations, and scholars would like to publish their research in such journals. In contrast to the trend in subject-1 journals, the trend in subject-4 journals suggests that STEM education journals collectively started to gain its own identity for publishing and sharing STEM education research.

STEM education publication distribution across different journal categories over the years. (Note: 0=subject-0; 1=subject-1; 2=subject-2; 3=subject-3; 4=subject-4; 5=subject-5)

Figure  6 shows the number of STEM education publications in each journal where the bars are color-coded (yellow, subject-0; light blue, subject-1; green, subject-2; purple, subject-3; dark blue, subject-4; and black, subject-5). There is no clear pattern shown in terms of the overall number of STEM education publications across categories or journals, but very much individual journal-based performance. The result indicates that the number of STEM education publications might heavily rely on the individual journal’s willingness and capability of attracting STEM education research work and thus suggests the potential value of examining individual journal’s performance.

Publication distribution across all 36 individual journals across different categories with the same color-coded for journals in the same subject category

The top five journals in terms of the number of STEM education publications are Journal of Science Education and Technology (80 publications, journal number 25 in Fig.  6 ), Journal of STEM Education (65 publications, journal number 26), International Journal of STEM Education (64 publications, journal number 17), International Journal of Engineering Education (54 publications, journal number 12), and School Science and Mathematics (41 publications, journal number 31). Among these five journals, two journals are specifically on STEM education (J26, J17), two on two subjects of STEM (J25, J31), and one on one subject of STEM (J12).

Figure  7 shows the number of STEM education publications per year in each of these top five journals. As expected, based on earlier trends, the number of publications per year increased over the study period. The largest increase was in the International Journal of STEM Education (J17) that was established in 2014. As the other four journals were all established in or before 2000, J17’s short history further suggests its outstanding performance in attracting and publishing STEM education articles since 2014 (Li, 2018b ; Li, Froyd, & Wang, 2019 ). The increase was consistent with the journal’s recognition as the first STEM education journal for inclusion in SSCI starting in 2019 (Li, 2019a ).

Publication distribution of selected five journals over the years. (Note: J12: International Journal of Engineering Education; J17: International Journal of STEM Education; J25: Journal of Science Education and Technology; J26: Journal of STEM Education; J31: School Science and Mathematics)

Top 10 countries/regions where scholars contributed journal publications in STEM education

Table  3 shows top countries/regions in terms of the number of publications, where the country/region was established by the authorship using the two different methods presented above. About 75% (depending on the method) of contributions were made by authors from the USA, followed by Australia, Canada, Taiwan, and UK. Only Africa as a continent was not represented among the top 10 countries/regions. The results are relatively consistent with patterns reported in the IJ-STEM study (Li, Froyd, & Wang, 2019 )

Further examination of Table 3 reveals that the two methods provide not only fairly consistent results but also yield some differences. For example, Israel and Germany had more publication credit if only the corresponding author was considered, but South Korea and Turkey had more publication credit when co-authors were considered. The results in Table 3 show that each method has value when analyzing and comparing publications by country/region or institution based on authorship.

Recognizing that, as shown in Fig. 1 , the number of publications per year increased rapidly since 2010, Table  4 shows the number of publications by country/region over a 10-year period (2009–2018) and Table 5 shows the number of publications by country/region over a 5-year period (2014–2018). The ranks in Tables  3 , 4 , and 5 are fairly consistent, but that would be expected since the larger numbers of publications in STEM education had occurred in recent years. At the same time, it is interesting to note in Table 5 some changes over the recent several years with Malaysia, but not Israel, entering the top 10 list when either method was used to calculate author's credit.

Patterns of single-author and multiple-author publications in STEM education

Since STEM education differs from traditional individual disciplinary education, we are interested in determining how common joint co-authorship with collaborations was in STEM education articles. Figure  8 shows that joint co-authorship was very common among these 798 STEM education publications, with 83.7% publications with two or more co-authors. Publications with two, three, or at least five co-authors were highest, with 204, 181, and 157 publications, respectively.

Number of publications with single or different joint authorship. (Note: 1=single author; 2=two co-authors; 3=three co-authors; 4=four co-authors; 5=five or more co-authors)

Figure  9 shows the number of publications per year using the joint authorship categories in Fig.  8 . Each category shows an increase consistent with the increase shown in Fig. 1 for all 798 publications. By the end of the time period, the number of publications with two, three, or at least five co-authors was the largest, which might suggest an increase in collaborations in STEM education research.

Publication distribution with single or different joint authorship over the years. (Note: 1=single author; 2=two co-authors; 3=three co-authors; 4=four co-authors; 5=five or more co-authors)

Co-authors can be from the same or different countries/regions. Figure  10 shows the number of publications per year by single authors (no collaboration), co-authors from the same country (collaboration in a country/region), and co-authors from different countries (collaboration across countries/regions). Each year the largest number of publications was by co-authors from the same country, and the number increased dramatically during the period of the study. Although the number of publications in the other two categories increased, the numbers of publications were noticeably fewer than the number of publications by co-authors from the same country.

Publication distribution in authorship across different categories in terms of collaboration over the years

Published articles by research topics

Figure  11 shows the number of publications in each of the seven topic categories. The topic category of goals, policy, curriculum, evaluation, and assessment had almost half of publications (375, 47%). Literature reviews were included in this topic category, as providing an overview assessment of education and research development in a topic area or a field. Sample publications included in this category are listed as follows:

DeCoito ( 2016 ). “STEM education in Canada: A knowledge synthesis.” Canadian Journal of Science , Mathematics and Technology Education , 16 (2), 114–128. (Note: this article provides a national overview of STEM initiatives and programs, including success, criteria for effective programs and current research in STEM education.)

Ring-Whalen, Dare, Roehrig, Titu, and Crotty ( 2018 ). “From conception to curricula: The role of science, technology, engineering, and mathematics in integrated STEM units.” International Journal of Education in Mathematics Science and Technology , 6 (4), 343–362. (Note: this article investigates the conceptions of integrated STEM education held by in-service science teachers through the use of photo-elicitation interviews and examines how those conceptions were reflected in teacher-created integrated STEM curricula.)

Schwab et al. ( 2018 ). “A summer STEM outreach program run by graduate students: Successes, challenges, and recommendations for implementation.” Journal of Research in STEM Education , 4 (2), 117–129. (Note: the article details the organization and scope of the Foundation in Science and Mathematics Program and evaluates this program.)

Frequencies of publications’ research topic distributions. (Note: 1=K-12 teaching, teacher and teacher education; 2=Post-secondary teacher and teaching; 3=K-12 STEM learner, learning, and learning environment; 4=Post-secondary STEM learner, learning, and learning environments; 5=Goals and policy, curriculum, evaluation, and assessment (including literature review); 6=Culture, social, and gender issues; 7=History, philosophy, Epistemology, and nature of STEM and STEM education)

The topic with the second most publications was “K-12 teaching, teacher and teacher education” (103, 12.9%), followed closely by “K-12 learner, learning, and learning environment” (97, 12.2%). The results likely suggest the research community had a broad interest in both teaching and learning in K-12 STEM education. The top three topics were the same in the IJ-STEM review (Li, Froyd, & Wang, 2019 ).

Figure  11 also shows there was a virtual tie between two topics with the fourth most cumulative publications, “post-secondary STEM learner & learning” (76, 9.5%) and “culture, social, and gender issues in STEM” (78, 9.8%), such as STEM identity, students’ career choices in STEM, and inclusion. This result is different from the IJ-STEM review (Li, Froyd, & Wang, 2019 ), where “post-secondary STEM teacher & teaching” and “post-secondary STEM learner & learning” were tied as the fourth most common topics. This difference is likely due to the scope of journals and the length of the time period being reviewed.

Figure  12 shows the number of publications per year in each topic category. As expected from the results in Fig.  11 the number of publications in topic category 5 (goals, policy, curriculum, evaluation, and assessment) was the largest each year. The numbers of publications in topic category 3 (K-12 learner, learning, and learning environment), 1 (K-12 teaching, teacher, and teacher education), 6 (culture, social, and gender issues in STEM), and 4 (post-secondary STEM learner and learning) were also increasing. Although Fig.  11 shows the number of publications in topic category 1 was slightly more than the number of publications in topic category 3 (see Fig.  11 ), the number of publications in topic category 3 was increasing more rapidly in recent years than its counterpart in topic category 1. This may suggest a more rapidly growing interest in K-12 STEM learner, learning, and learning environment. The numbers of publications in topic categories 2 and 7 were not increasing, but the number of publications in IJ-STEM in topic category 2 was notable (Li, Froyd, & Wang, 2019 ). It will be interesting to follow trends in the seven topic categories in the future.

Publication distributions in terms of research topics over the years

Published articles by research methods

Figure  13 shows the number of publications per year by research methods in empirical studies. Publications with non-empirical studies are shown in a separate category. Although the number of publications in each of the four categories increased during the study period, there were many more publications presenting empirical studies than those without. For those with empirical studies, the number of publications using quantitative methods increased most rapidly in recent years, followed by qualitative and then mixed methods. Although there were quite many publications with non-empirical studies (e.g., theoretical or conceptual papers, literature reviews) during the study period, the increase of the number of publications in this category was noticeably less than empirical studies.

Publication distributions in terms of research methods over the years. (Note: 1=qualitative, 2=quantitative, 3=mixed, 4=Non-empirical)

Concluding remarks

The systematic analysis of publications that were considered to be in STEM education in 36 selected journals shows tremendous growth in scholarship in this field from 2000 to 2018, especially over the past 10 years. Our analysis indicates that STEM education research has been increasingly recognized as an important topic area and studies were being published across many different journals. Scholars still hold diverse perspectives about how research is designated as STEM education; however, authors have been increasingly distinguishing their articles with STEM, STEAM, or related words in the titles, abstracts, and lists of keywords during the past 10 years. Moreover, our systematic analysis shows a dramatic increase in the number of publications in STEM education journals in recent years, which indicates that these journals have been collectively developing their own professional identity. In addition, the International Journal of STEM Education has become the first STEM education journal to be accepted in SSCI in 2019 (Li, 2019a ). The achievement may mark an important milestone as STEM education journals develop their own identity for publishing and sharing STEM education research.

Consistent with our previous reviews (Li, Froyd, & Wang, 2019 ; Li, Wang, & Xiao, 2019 ), the vast majority of publications in STEM education research were contributed by authors from the USA, where STEM and STEAM education originated, followed by Australia, Canada, and Taiwan. At the same time, authors in some countries/regions in Asia were becoming very active in the field over the past several years. This trend is consistent with findings from the IJ-STEM review (Li, Froyd, & Wang, 2019 ). We certainly hope that STEM education scholarship continues its development across all five continents to support educational initiatives and programs in STEM worldwide.

Our analysis has shown that collaboration, as indicated by publications with multiple authors, has been very common among STEM education scholars, as that is often how STEM education distinguishes itself from the traditional individual disciplinary based education. Currently, most collaborations occurred among authors from the same country/region, although collaborations across cross-countries/regions were slowly increasing.

With the rapid changes in STEM education internationally (Li, 2019b ), it is often difficult for researchers to get an overall sense about possible hot topics in STEM education especially when STEM education publications appeared in a vast array of journals across different fields. Our systematic analysis of publications has shown that studies in the topic category of goals, policy, curriculum, evaluation, and assessment have been the most prevalent, by far. Our analysis also suggests that the research community had a broad interest in both teaching and learning in K-12 STEM education. These top three topic categories are the same as in the IJ-STEM review (Li, Froyd, & Wang, 2019 ). Work in STEM education will continue to evolve and it will be interesting to review the trends in another 5 years.

Encouraged by our recent IJ-STEM review, we began this review with an ambitious goal to provide an overview of the status and trends of STEM education research. In a way, this systematic review allowed us to achieve our initial goal with a larger scope of journal selection over a much longer period of publication time. At the same time, there are still limitations, such as the decision to limit the number of journals from which we would identify publications for analysis. We understand that there are many publications on STEM education research that were not included in our review. Also, we only identified publications in journals. Although this is one of the most important outlets for scholars to share their research work, future reviews could examine publications on STEM education research in other venues such as books, conference proceedings, and grant proposals.

Availability of data and materials

The data and materials used and analyzed for the report are publicly available at the various journal websites.

Journals containing the word "computers" or "ICT" appeared automatically when searching with the word "technology". Thus, the word of "computers" or "ICT" was taken as equivalent to "technology" if appeared in a journal's name.

Abbreviations

Information and Communications Technology

International Journal of STEM Education

Kindergarten–Grade 12

Science, Mathematics, Engineering, and Technology

Science, Technology, Engineering, Arts, and Mathematics

Science, Technology, Engineering, and Mathematics

Borrego, M., Foster, M. J., & Froyd, J. E. (2015). What is the state of the art of systematic review in engineering education? Journal of Engineering Education, 104 (2), 212–242. https://doi.org/10.1002/jee.20069 .

Article   Google Scholar  

Bray, A., & Tangney, B. (2017). Technology usage in mathematics education research – a systematic review of recent trends. Computers & Education, 114 , 255–273.

Brown, J. (2012). The current status of STEM education research. Journal of STEM Education: Innovations & Research, 13 (5), 7–11.

Google Scholar  

Christenson, J. (2011). Ramaley coined STEM term now used nationwide . Winona Daily News Retrieved from http://www.winonadailynews.com/news/local/article_457afe3e-0db3-11e1-abe0-001cc4c03286.html Accessed on 16 Jan 2018.

Chute, E. (2009). STEM education is branching out . Pittsburgh Post-Gazette Feb 9, 2009. https://www.post-gazette.com/news/education/2009/02/10/STEM-education-is-branching-out/stories/200902100165 Accessed on 2 Jan 2020.

DeCoito, I. (2016). STEM education in Canada: A knowledge synthesis. Canadian Journal of Science, Mathematics and Technology Education, 16 (2), 114–128.

Dika, S. L., & D'Amico, M. M. (2016). Early experiences and integration in the persistence of first-generation college students in STEM and non-STEM majors. Journal of Research in Science Teaching, 53 (3), 368–383.

English, L. D. (2016). STEM education K-12: Perspectives on integration. International Journal of STEM Education, 3 , 3. https://doi.org/10.1186/s4059%204-016-0036-1 .

Erduran, S., Ozdem, Y., & Park, J.-Y. (2015). Research trends on argumentation in science education: A journal content analysis from 1998-2014. International Journal of STEM Education, 2 , 5. https://doi.org/10.1186/s40594-015-0020-1 .

Gonzalez, H. B. & Kuenzi, J. J. (2012). Science, technology, engineering, and mathematics (STEM) education: A primer. CRS report for congress, R42642, https://fas.org/sgp/crs/misc/R42642.pdf Accessed on 2 Jan 2020.

Henderson, C., Beach, A., & Finkelstein, N. (2011). Facilitating change in undergraduate STEM instructional practices: An analytic review of the literature. Journal of Research in Science Teaching, 48 (8), 952–984.

Honey, M., Pearson, G., & Schweingruber, A. (2014). STEM integration in K-12 education: Status, prospects, and an agenda for research . Washington: National Academies Press.

Howard, G. S., Cole, D. A., & Maxwell, S. E. (1987). Research productivity in psychology based on publication in the journals of the American Psychological Association. American Psychologist, 42 (11), 975–986.

Johnson, C. C., Peters-Burton, E. E., & Moore, T. J. (2015). STEM roadmap: A framework for integration . London: Taylor & Francis.

Book   Google Scholar  

Kelley, T. R., & Knowles, J. G. (2016). A conceptual framework for integrated STEM education. International Journal of STEM Education, 3 , 11. https://doi.org/10.1186/s40594-016-0046-z .

Kilpatrick, J. (1992). A history of research in mathematics education. In D. A. Grouws (Ed.), Handbook of research on mathematics teaching and learning (pp. 3–38). New York: Macmillan.

Kim, A. Y., Sinatra, G. M., & Seyranian, V. (2018). Developing a STEM identity among young women: A social identity perspective. Review of Educational Research, 88 (4), 589–625.

Li, Y. (2014). International journal of STEM education – a platform to promote STEM education and research worldwide. International Journal of STEM Education, 1 , 1. https://doi.org/10.1186/2196-7822-1-1 .

Li, Y. (2018a). Journal for STEM education research – promoting the development of interdisciplinary research in STEM education. Journal for STEM Education Research, 1 (1–2), 1–6. https://doi.org/10.1007/s41979-018-0009-z .

Li, Y. (2018b). Four years of development as a gathering place for international researchers and readers in STEM education. International Journal of STEM Education, 5 , 54. https://doi.org/10.1186/s40594-018-0153-0 .

Li, Y. (2019a). Five years of development in pursuing excellence in quality and global impact to become the first journal in STEM education covered in SSCI. International Journal of STEM Education, 6 , 42. https://doi.org/10.1186/s40594-019-0198-8 .

Li, Y. (2019b). STEM education research and development as a rapidly evolving and international field. 数学教育学报(Journal of Mathematics Education), 28 (3), 42–44.

Li, Y., Froyd, J. E., & Wang, K. (2019). Learning about research and readership development in STEM education: A systematic analysis of the journal’s publications from 2014 to 2018. International Journal of STEM Education, 6 , 19. https://doi.org/10.1186/s40594-019-0176-1 .

Li, Y., & Schoenfeld, A. H. (2019). Problematizing teaching and learning mathematics as ‘given’ in STEM education. International Journal of STEM Education, 6 , 44. https://doi.org/10.1186/s40594-019-0197-9 .

Li, Y., Wang, K., & Xiao, Y. (2019). Exploring the status and development trends of STEM education research: A review of research articles in selected journals published between 2000 and 2018. 数学教育学报(Journal of Mathematics Education), 28 (3), 45–52.

Lin, T.-J., Lin, T.-C., Potvin, P., & Tsai, C.-C. (2019). Research trends in science education from 2013 to 2017: A systematic content analysis of publications in selected journals. International Journal of Science Education, 41 (3), 367–387.

Margot, K. C., & Kettler, T. (2019). Teachers’ perception of STEM integration and education: A systematic literature review. International Journal of STEM Education, 6 , 2. https://doi.org/10.1186/s40594-018-0151-2 .

Minichiello, A., Hood, J. R., & Harkness, D. S. (2018). Bring user experience design to bear on STEM education: A narrative literature review. Journal for STEM Education Research, 1 (1–2), 7–33.

Minner, D. D., Levy, A. J., & Century, J. (2010). Inquiry-based science instruction – what is it and does it matter? Results from a research synthesis years 1984 to 2002. Journal of Research in Science Teaching, 47 (4), 474–496.

Mizell, S., & Brown, S. (2016). The current status of STEM education research 2013-2015. Journal of STEM Education: Innovations & Research, 17 (4), 52–56.

National Research Council. (2012). Discipline-based education research: Understanding and improving learning in undergraduate science and engineering . Washington DC: National Academies Press.

National Science Foundation (1998). Information technology: Its impact on undergraduate education in science, mathematics, engineering, and technology. (NSF 98–82), April 18–20, 1996. http://www.nsf.gov/cgi-bin/getpub?nsf9882 Accessed 16 Jan 2018.

Raju, P. K., & Sankar, C. S. (2003). Editorial. Journal of STEM Education: Innovations & Research, 4 (3&4), 2.

Ring-Whalen, E., Dare, E., Roehrig, G., Titu, P., & Crotty, E. (2018). From conception to curricula: The role of science, technology, engineering, and mathematics in integrated STEM units. International Journal of Education in Mathematics, Science and Technology, 6 (4), 343–362.

Schreffler, J., Vasquez III, E., Chini, J., & James, W. (2019). Universal design for learning in postsecondary STEM education for students with disabilities: A systematic literature review. International Journal of STEM Education, 6 , 8. https://doi.org/10.1186/s40594-019-0161-8 .

Schwab, D. B., Cole, L. W., Desai, K. M., Hemann, J., Hummels, K. R., & Maltese, A. V. (2018). A summer STEM outreach program run by graduate students: Successes, challenges, and recommendations for implementation. Journal of Research in STEM Education, 4 (2), 117–129.

Sochacka, N. W., Guyotte, K. W., & Walther, J. (2016). Learning together: A collaborative autoethnographic exploration of STEAM (STEM+ the Arts) education. Journal of Engineering Education, 105 (1), 15–42.

Sokolowski, A., Li, Y., & Willson, V. (2015). The effects of using exploratory computerized environments in grades 1 to 8 mathematics: A meta-analysis of research. International Journal of STEM Education, 2 , 8. https://doi.org/10.1186/s40594-015-0022-z .

Thibaut, L., Ceuppens, S., De Loof, H., De Meester, J., Goovaerts, L., Struyf, A., Pauw, J. B., Dehaene, W., Deprez, J., De Cock, M., Hellinckx, L., Knipprath, H., Langie, G., Struyven, K., Van de Velde, D., Van Petegem, P., & Depaepe, F. (2018). Integrated STEM education: A systematic review of instructional practices in secondary education. European Journal of STEM Education, 3 (1), 2.

Tsai, C. C., & Wen, L. M. C. (2005). Research and trends in science education from 1998 to 2002: A content analysis of publication in selected journals. International Journal of Science Education, 27 (1), 3–14.

United States Congress House Committee on Science. (1998). The state of science, math, engineering, and technology (SMET) education in America, parts I-IV, including the results of the Third International Mathematics and Science Study (TIMSS): hearings before the Committee on Science, U.S. House of Representatives, One Hundred Fifth Congress, first session, July 23, September 24, October 8 and 29, 1997. Washington: U.S. G.P.O.

Vasquez, J., Sneider, C., & Comer, M. (2013). STEM lesson essentials, grades 3–8: Integrating science, technology, engineering, and mathematics . Portsmouth, NH: Heinemann.

Wu, S. P. W., & Rau, M. A. (2019). How students learn content in science, technology, engineering, and mathematics (STEM) through drawing activities. Educational Psychology Review . https://doi.org/10.1007/s10648-019-09467-3 .

Xu, M., Williams, P. J., Gu, J., & Zhang, H. (2019). Hotspots and trends of technology education in the International Journal of Technology and Design Education: 2000-2018. International Journal of Technology and Design Education . https://doi.org/10.1007/s10798-019-09508-6 .

Download references

Not applicable

Author information

Authors and affiliations.

Texas A&M University, College Station, TX, 77843-4232, USA

Yeping Li & Yu Xiao

Nicholls State University, Thibodaux, LA, 70310, USA

Ohio State University, Columbus, OH, 43210, USA

Jeffrey E. Froyd

You can also search for this author in PubMed   Google Scholar

Contributions

YL conceptualized the study and drafted the manuscript. KW and YX contributed with data collection, coding, and analyses. JEF reviewed drafts and contributed to manuscript revisions. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Yeping Li .

Ethics declarations

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Cite this article.

Li, Y., Wang, K., Xiao, Y. et al. Research and trends in STEM education: a systematic review of journal publications. IJ STEM Ed 7 , 11 (2020). https://doi.org/10.1186/s40594-020-00207-6

Download citation

Received : 10 February 2020

Accepted : 12 February 2020

Published : 10 March 2020

DOI : https://doi.org/10.1186/s40594-020-00207-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Journal publication
• Literature review
• STEM education research

Academic Integrity and Artificial Intelligence in Higher Education (HE) Contexts: A Rapid Scoping Review

• Helen Pethrick University of Calgary
• Jason Wiens University of Calgary

Artificial Intelligence (AI) developments challenge higher education institutions’ teaching, learning, assessment, and research practices. To contribute timely and evidence-based recommendations for upholding academic integrity, we conducted a rapid scoping review focusing on what is known about academic integrity and AI in higher education. We followed the Updated Reviewer Manual for Scoping Reviews from the Joanna Briggs Institute (JBI) and the Preferred Reporting Items for Systematic reviews Meta-Analysis for Scoping Reviews (PRISMA-ScR) reporting standards. Five databases were searched, and the eligibility criteria included higher education stakeholders of any age and gender engaged with AI in the context of academic integrity from 2007 through November 2022 and available in English. The search retrieved 2223 records, of which 14 publications with mixed methods, qualitative, quantitative, randomized controlled trials, and text and opinion studies met the inclusion criteria. The results showed bounded and unbounded ethical implications of AI. Perspectives included: AI for cheating; AI as legitimate support; an equity, diversity, and inclusion lens into AI; and emerging recommendations to tackle AI implications in higher education. The evidence from the sources provides guidance that can inform educational stakeholders in decision-making processes for AI integration, in the analysis of misconduct cases involving AI, and in the exploration of AI as legitimate assistance. Likewise, this rapid scoping review signals key questions for future research, which we explore in our discussion.

Author Biographies

Beatriz antonieta moya, university of calgary.

Beatriz Moya is a Ph.D. candidate in the Werklund School of Education at the University of Calgary in the Educational Research program, specializing in Leadership. Her primary motivation as a student researcher is to contribute to the continuous transformation of higher education institutions’ cultures to pursue academic and research integrity and social justice. For this reason, Beatriz situates herself at the intersections of the Scholarship of Teaching and Learning (SoTL), Leadership, and Academic Integrity. In this space, Beatriz is currently involved in projects seeking to contribute to a ‘glocal’ scholarly dialogue concerning academic integrity as a teaching and learning imperative. For instance, Beatriz is exploring the situated meanings and experiential insights from academic integrity educational leaders, and is also a contributing author for a chapter on academic integrity policy in Latin America in the Handbook of Academic Integrity (2nd ed.). 

Sarah Elaine Eaton, University of Calgary

Dr. Sarah Elaine Eaton, PhD, is an Associate Professor of Education at the University of Calgary, Canada. She served as the inaugural Educational Leader in Residence, Academic Integrity at Taylor Institute for Teaching and Learning, University of Calgary. Dr. Eaton’s research focuses on academic ethics in higher education. Her work can be found in the British Educational Research Journal, the Journal of Academic Ethics, and the Journal of Educational Thought and Interchange, among other places. She is the Editor-in-Chief of the International Journal for Educational Integrity (Springer Nature) and co-founder and co-editor of Canadian Perspectives on Academic Integrity. In 2020 she received the National Research and Scholarship award from the Canadian Society for the Study of Higher Education (CSSHE) for her contributions to research on academic integrity in Canadian higher education. In 2022, she received the outstanding research award from the European Network for Academic Integrity (ENAI). Her books include Plagiarism in Higher Education: Tackling Tough Topics in Academic Integrity, Academic Integrity in Canada: An Enduring and Essential Challenge (Eaton & Christensen Hughes, eds.), Contract Cheating in Higher Education: Global Perspectives on Theory, Practice, and Policy (Eaton, Curtis, Stoesz, Clare, Rundle, & Seeland, eds.) and Ethics and Integrity in Teacher Education (Eaton & Khan, eds.). 

Helen Pethrick, University of Calgary

Helen Pethrick, MA, is a researcher and educator in Calgary, Alberta, Canada. Helen’s research encompasses academic integrity in higher education, post-secondary student mental health, and mentorship in academia. Helen has interdisciplinary expertise in systematic literature review methodology, qualitative research, project management, and knowledge exchange. With Dr. Sarah Elaine Eaton and Jamie J. Carmichael, Helen is co-Editor of the volume Fake Degrees and Fraudulent Credentials in Higher Education (Eaton, Carmichael, & Pethrick, forthcoming 2023). Currently, Helen’s role is Research Associate, Academic Integrity, which involves acting as Project Manager (administrative) for the Academic Integrity and Artificial Intelligence project at the University of Calgary. 

K. Alix Hayden, University of Calgary

Dr. Alix Hayden is a Librarian at the University of Calgary.

Robert Brennan

Holds a PhD in mechanical engineering from the University of Calgary. He is a Professor of Mechanical and Manufacturing Engineering at the University of Calgary, holds the NSERC Chair in Design Engineering, and has served as President of the Canadian Engineering Education Association (CEEA). His research interests range from engineering education to intelligent automation and control systems.  

Jason Wiens, University of Calgary

Dr. Wiens is a Professor (Teaching) in the Department of English at the University of Calgary. His areas of research interest include contemporary poetry, Canadian literature, archival studies, literary audio, and pedagogy.  He has published articles in numerous journals including LIT: Literature Interpretation Theory, Canadian Literature, Studies on Canadian Writing, and Canadian Poetry. He has recently guest edited a special issue of English Studies in Canada on "Pedagogies of the Archive." He is currently a co-investigator on the SSHRC Partnership project The SpokenWeb, and is a co-investigator on the Academic Integrity and Artificial Intelligence project at the University of Calgary.  

Brenda McDermott, University of Calgary

Dr. McDermott completed her PhD in communication studies at the University of Calgary.  Her research involves looking at ableism embedded in teaching and learning practices, particular assessment. She regularly provided training to faculty to help creating learning environments that reflect the diversity of learners.  

Alonso, A. N. (2022). Online translators in online language assessments. CALL-EJ, 23(3), 115-135. http://callej.org/journal/23-3/Alonso2022.pdf

Anselmo, L., Kendon, T. & Moya, B. (2023). A First Response to Assessment and ChatGPT in your Courses. Taylor Institute for Teaching and Learning. https://taylorinstitute.ucalgary.ca/first-response-assessment-and-chatgpt

Anson, C. M. (2022). AI-based text generation and the social construction of “fraudulent authorship”: A revisitation. Composition Studies, 50(1), 37-46.

Aromataris, E., & Munn, A. (2020). JBI manual for evidence synthesis. JBI. https://doi.org/10.46658/JBIMES-20-01

Australian Academic Integrity Network (AAIN). (2023). AAIN generative artificial intelligence guidelines. https://www.teqsa.gov.au/sites/default/files/2023-04/aain-generative-ai-guidelines.pdf

Barker, T., Stone J. C., Sears K., Klugar M., Tufanaru, C., & Leonardi-Bee, J. (2023). The revised JBI critical appraisal tool for the assessment of risk of bias for randomized controlled trials. JBI Evidence Synthesis, 21(3), 494-506. https://journals.lww.com/jbisrir/Fulltext/2023/03000/The_revised_JBI_critical_appraisal_tool_for_the.5.aspx

Bearman, M., & Luckin, R. (2020). Preparing university assessment for a world with AI: Tasks for human intelligence. In M. Bearman, P. Dawson, R. Ajjawi, J. Tai & D. Boud (Eds.), Re-imagining university assessment in a digital world (pp. 49-63). Springer International Publishing. https://doi.org/https://doi.org/10.1007/978-3-030-41956-1

Brake, J. (2022). Education in the world of ChatGPT. The Absent-Minded Professor https://joshbrake.substack.com/p/education-in-the-world-of-chatgpt?utm_source=direct&utm_campaign=post&utm_medium=web

Bretag, T. (2016). Defining academic integrity: International perspectives – Introduction. In T. Bretag (Ed.), Handbook of Academic Integrity (1st edition). Springer Singapore: 3-5. https://link.springer.com/content/pdf/10.1007/978-981-287-098-8_76.pdf

Brusini, A. (2023). ChatGPT: A brief introduction and considerations for academic integrity. The Innovative Instructor. https://ii.library.jhu.edu/2023/01/30/chatgpt-a-brief-introduction-and-considerations-for-academic-integrity/

Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Tat Lee, Y. Li, Y., Lundberg, S., Nori, H., Palangi, H., Ribeiro, M. T., & Zhang, Y. (2023). Sparks of artificial general intelligence: Early experiments with GPT-4. https://arxiv.org/pdf/2303.12712.pdf

Cassidy, C. (2023, January 10). Australian universities to return to ‘pen and paper’ exams after students caught using AI to write essays. The Guardian. https://www.theguardian.com/australia-news/2023/jan/10/universities-to-return-to-pen-and-paper-exams-after-students-caught-using-ai-to-write-essays

Chen, M. H., Huang, S.-T., Chang, J.S., & Liou, H.-C. (2015). Developing a corpus-based paraphrase tool to improve EFL learners' writing skills. Computer Assisted Language Learning, 28(1), 22-40. http://dx.doi.org/10.1080/09588221.2013.783873

Cochrane, T., & Ryan, T. (2023). ChatGPT and academic integrity: Options for adapting. Melbourne Centre for the Study of Higher Education. https://melbourne-cshe.unimelb.edu.au/__data/assets/pdf_file/0008/4533218/ChatGPT-and-Academic-Integrity.pdf

Dans, E. (2019, February 6). Meet Bertie, Heliograf and Cyborg, the new journalists on the block. Forbes. https://www.forbes.com/sites/enriquedans/2019/02/06/meet-bertie-heliograf-and-cyborg-the-new-journalists-on-the-block/?sh=669bf965138d

Dawson, P. (2020). E-Cheating, assessment security and artificial intelligence. In P. Dawson (Ed.), Defending assessment security in a digital world (pp. 83-97). Routledge. http://doi.org/10.4324/9780429324178

Delaney, M. (2023). ChatGPT, AI language bot, can pass business, law and medical exams. Washington Times. https://www.washingtontimes.com/news/2023/jan/29/chatgpt-ai-language-bot-can-pass-business-law-and-/

Delisio, L. A., & Butaky, C. A. (2019). UDL and assistive technology: Utilizing technology beyond mere accessibility. In W. W. Murawski & K. L. Scott (Eds.), What really works with Universal Design for Learning (pp. 157-172). Corwin.

Dignum, V. (2021). The role and challenges of education for responsible AI. London Review of Education, 19(1), 1-11. https://ezproxy.lib.ucalgary.ca/login?url=https://search.ebscohost.com/login.aspx?direct=true&db=a9h&AN=157082792&site=ehost-live

Dinneen, C. (2021). Students’ use of digital translation and paraphrasing tools in written assignments on direct entry English programs. English Australia Journal, 37(1). https://files.eric.ed.gov/fulltext/EJ1341751.pdf

Ducar, C., & Schocket, D. H. (2018). Machine translation and the L2 classroom: Pedagogical solutions for making peace with Google Translate. Foreign Language Annals 51(4), 779-795.

Eaton, S. E. (2022). Student Academic Integrity: A Handbook for Academic Staff and Teaching Assistants. Taylor Institute of Teaching and Learning. https://taylorinstitute.ucalgary.ca/resources/student-academic-integrity-handbook

Eaton, S. E. (2023). Sarah’s Thoughts: Artificial Intelligence and Academic Integrity. Learning, Teaching and Leadership: A blog for educators, researchers and other thinkers. https://drsaraheaton.wordpress.com/2022/12/09/sarahs-thoughts-artificial-intelligence-and-academic-integrity/

Eaton, S. E., & L. Anselmo (2023). Teaching and learning with artificial intelligence apps. Taylor Institute for Teaching and Learning. https://taylorinstitute.ucalgary.ca/teaching-with-AI-apps

Eaton, S. E., Pethrick, H., & Turner, K. L. (2023). Academic integrity and student mental well-being: A rapid review. Canadian Perspectives on Academic Integrity, 5(2). https://doi.org/10.11575/cpai.v5i2.73748

Education USA. (n.d.). The U.S. educational system - Glossary. https://educationusa.state.gov/experience-studying-usa/us-educational-system/glossary#T

European Union. (2022). Ethical guidelines on the use of artificial intelligence (AI) in teaching and learning for educators. https://op.europa.eu/en/publication-detail/-/publication/d81a0d54-5348-11ed-92ed-01aa75ed71a1/language-en

Flinders University (2023). What is artificial intelligence? Flinders University Library. https://library.flinders.edu.au/students/ai

Fyfe, P. (2022). How to cheat on your final paper: Assigning AI for student writing. AI & Society. https://link.springer.com/article/10.1007/s00146-022-01397-z

Foltýnek, T., Bjelobaba, S., Glendinning, I, Khan, Z.R., Santos, R. Pavletic, P., & Kravjar, J. (2023). ENAI Recommendations on the ethical use of Artificial Intelligence in Education. International Journal for Academic Integrity, 19, 12. https://doi.org/10.1007/s40979-023-00133-4

Gero, K. I., et al. (2022). Sparks: Inspiration for science writing using language models. DIS '22: Designing Interactive Systems Conference, Association for Computing Machinery, Inc. https://arxiv.org/abs/2110.07640

Government of Canada. (2022). Education in Canada: Post-secondary. https://www.canada.ca/en/immigration-refugees-citizenship/services/new-immigrants/new-life-canada/education/types-school/post-secondary.html

Groves, M., & Mundt, K. (2021). A ghostwriter in the machine? Attitudes of academic staff towards machine translation use in internationalised Higher Education. Journal of English for Academic Purposes, 50. https://doi.org/10.1016/j.jeap.2021.100957

Hartling, L., Guise, J. M., Hempel, S., Featherstone, R., Mitchell, M. D., Motu’Apuaka, M. L., Robinson, K. A., Schoelles, K., Totten, A., Whitlock, E., Wilt, T. J., Anderson, J., Berliner, E., Gozu, A., Kato, E., Paynter, R., & Umscheid, C. A. (2017). Fit for purpose: Perspectives on rapid reviews from end-user interviews. Systematic Reviews, 6(32). https://doi.org/10.1186/s13643-017-0425-7

Hemsley, B., Power, E., & Given, F. (January 18, 2023). Will AI tech like ChatGPT improve inclusion for people with communication disability. The Conversation. https://theconversation.com/will-ai-tech-like-chatgpt-improve-inclusion-for-people-with-communication-disability-196481

Hiatt, B. (2023, January 28). ChatGPT: Educators hold emergency meetings as AI disrupts schools and universities across Australia. The West Australian. https://thewest.com.au/technology/chatgpt-educators-scramble-as-ai-disrupts-schools-and-universities-across-australia-c-9565061

Hong, Q. N., Pluye, P., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M., Griffiths, F., Nicolau, B., O’Cathain, A., Rousseau, M., & Vedel, I. (2018). Mixed methods appraisal tool (MMAT). McGill University. http://mixedmethodsappraisaltoolpublic.pbworks.com/w/file/fetch/127916259/MMAT_2018_criteria-manual_2018-08-01_ENG.pdf

Hotson, B., & Bell, S. (2023). Academic writing and ChatGPT: Step back to step forward. Canadian Writing Centers Association. https://cwcaaccr.com/2023/04/09/chatgpt-step-back-to-step-forward/ .

ICAI (2021). The Fundamental Values of Academic Integrity. https://academicintegrity.org/resources/fundamental-values

Keith, T. (2023). Combating Academic Dishonesty, Part 6: ChatGPT, AI, and Academic Integrity. Academic Technology Solutions. https://academictech.uchicago.edu/2023/01/23/combating-academic-dishonesty-part-6-chatgpt-ai-and-academic-integrity/

Khan, Z. R. (2023). Artificial intelligence content generators in education for schools and universities: A good practice guide, European Network for Academic Integrity Working Group Centre for Academic Integrity in the UAE. University of Wollongong in Dubai. https://academicintegrity-uae.com/category/faculty- resources/

Kim, J. (2018, April 25). Are the professions (disciplines?) of educational developer and learning designer merging? Or not? Inside Higher Ed. https://www.insidehighered.com/digital-learning/blogs/technology-and-learning/are-professions-disciplines-educational-developer-and

Kublik, D., & Saboo, S. (2022). GPT-3: Building innovative NLP products using large language models. O’Reilly.

Lesage, J., Brennan, R., Eaton, S. E., Moya, B., McDermott, B., Wiens, J., & Herrero, K. (2023). Exploring natural language processing in mechanical engineering education: Implications for academic integrity. International Journal of Mechanical Engineering Education. https://journals.sagepub.com/doi/full/10.1177/03064190231166665

Lockwood, C., Munn, Z., & Porritt, K. Qualitative research synthesis: Methodological guidance for systematic reviewers utilizing meta-aggregation. International Journal of Evidence Based Healthcare, 13(3), 179–187. https://pubmed.ncbi.nlm.nih.gov/26262565/

Lunny, C., Antony, J., Ríos, P., Williams, C., Ramkissoon, N., Straus, S. E., & Tricco, A. C. (2021). Safety and effectiveness of dose-sparing strategies for intramuscular seasonal influenza vaccine: A rapid scoping review. BMJ Open, 11. https://doi.org/10.1136/bmjopen-2021-050596

McArthur A, Klugarova J, Yan H, & Florescu S. Chapter 4: Systematic reviews of text and opinion. In E. Aromataris & Z. Munn (Eds), JBI Manual for Evidence Synthesis. https://doi.org/10.46658/JBIMES-20-01

McNeill, L., & Chaudhuri, A. (2023). The opportunities of ChatGPT. University of British Columbia. https://academicintegrity.ubc.ca/2023/02/15/the-opportunities-of-chatgpt/

Merine, R., Purkayastha, S. (2022). Risks and benefits of AI-generated text summarization for expert level content in graduate health informatics. 2022 IEEE 10th International Conference on Healthcare Informatics (ICHI), Institute of Electrical and Electronics Engineers Inc. https://ieeexplore.ieee.org/document/9874678/

Mindzak, M. (2020, February 17). What happens when a machine can write as well as an academic? University Affairs. https://www.universityaffairs.ca/opinion/in-my-opinion/what-happens-when-a-machine-can-write-as-well-as-an-academic/

Monash University. (2023). Generative AI and assessment. https://www.monash.edu/learning-teaching/teachhq/Teaching-practices/artificial-intelligence/generative-ai-and-assessment

Moya, B. A., Eaton, S. E., Pethrick, H., Hayden, K. A., Brennan, R., Wiens, J., McDermott, B., & Lesage, J. (2023). Academic integrity and artificial intelligence in higher education contexts: A rapid scoping review protocol. Canadian Perspectives on Academic Integrity, 5(2). https://journalhosting.ucalgary.ca/index.php/ai/article/view/75990

Munn, Z., Peters, M. D. J., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018). Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Medical Research Methodology, 18(143). https://doi.org/10.1186/s12874-018-0611-x

Munn, Z., Pollock, D., Khalil, H., Alexander, L., Mclnerney, P., Godfrey, C. M., Peters, M., & Tricco, A. (2022). What are scoping reviews? Providing a formal definition of scoping reviews as a type of evidence synthesis. JBI Evidence Synthesis, 20(4), 950-952. https://pubmed.ncbi.nlm.nih.gov/35249995/

Murphy, S. (2023). ChatGPT passes exams from law and business schools. CNN Business. https://www.cnn.com/2023/01/26/tech/chatgpt-passes-exams/index.html

Nature (2023). Tools such as ChatGPT threaten transparent science. Nature (613). https://www.nature.com/articles/d41586-023-00191-1

OECD. (2002). Education at a glance: OECD indicators 2002. https://www.oecd-ilibrary.org/docserver/eag-2002-en.pdf?expires=1660686371&id=id&accname=guest&checksum=91C5FA8FBA74551E2A1C380F1079D47F

Ouyang, F., Zheng, F., & Jiao, P. (2022). Artificial intelligence in online higher education: A systematic review of empirical research from 2011 to 2020. Education and Information Technologies 27(6): 7893-7925. https://link.springer.com/article/10.1007/s10639-022-10925-9

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hrobjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., McGuiness, L. A., Stewart, L. A., Thomas, J., Tricco, A.C., Welch, V. A, Whiting, P., & Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372(71). https://doi.org/10.1136/bmj.n71

Perkins, M. (2023). Academic integrity considerations of AI Large Language Models in the post-pandemic era: ChatGPT and beyond. Journal of University Teaching and Learning Practice, 20(2). https://ro.uow.edu.au/cgi/viewcontent.cgi?article=3071&context=jutlp

Peters, M. D., Godfrey, C., McInerney, P., Munn, Z., Tricco, A., & Khalil, H. (2020). Chapter 11: Scoping reviews. In E. Aromataris & Z. Munn (Eds.), JBI manual for evidence synthesis. JBI. https://doi.org/10.46658/jbimes-20-12

Peters, M. D. J., Godfrey, C., McInerney, P., Khalil, H., Larsen, P., Marnie, C., Pollock, D., Tricco, A. C., & Munn, Z. (2022). Best practice guidance and reporting items for the development of scoping review protocols. JBI Evidence Synthesis, 20(4), 953-968. https://doi.org/10.11124/JBIES-21-00242

Popenici, S. A. D., & Kerr, S. (2017). Exploring the impact of artificial intelligence on teaching and learning in higher education. Research and Practice in Technology Enhanced Learning, 12(22). https://doi.org/10.1186/s41039-017-0062-8

Prentice, F. M., & Kinden, C. E. (2018). Paraphrasing tools, language translation tools and plagiarism: An exploratory study. International Journal for Educational Integrity, 14. https://edintegrity.biomedcentral.com/articles/10.1007/s40979-018-0036-7

Roe, J., & Perkins, M. (2022). What are automated paraphrasing tools and how do we address them? A review of a growing threat to academic integrity. International Journal for Educational Integrity, 18(1). https://edintegrity.biomedcentral.com/articles/10.1007/s40979-022-00109-w

Roe, J., Renandya, W., & Jacobs, G. (2023). A review of AI-powered writing tools and their implications for academic integrity in the language classroom. Journal of English and Applied Linguistics 2(1). https://animorepository.dlsu.edu.ph/jeal/vol2/iss1/3/

Rogerson, A. M., & McCarthy, G. (2017). Using internet based paraphrasing tools: Original work, patchwriting or facilitated plagiarism? International Journal for Educational Integrity, 13(2). https://edintegrity.biomedcentral.com/articles/10.1007/s40979-016-0013-y

Sabzalieva, E., & Valentini, A. (2023). ChatGPT and artificial intelligence in higher education: Quick start guide. UNESCO. https://www.iesalc.unesco.org/wp-content/uploads/2023/04/ChatGPT-and-Artificial-Intelligence-in-higher-education-Quick-Start-guide_EN_FINAL.pdf

Sharples, M. (2022). Automated essay writing: An AIED opinion. International Journal of Artificial Intelligence in Education, 32(4), 1119-1126. https://link.springer.com/article/10.1007/s40593-022-00300-7

Sloan, K. (2023, January 25). ChatGPT passes law school exams despite 'mediocre' performance. Reuters. https://www.reuters.com/legal/transactional/chatgpt-passes-law-school-exams-despite-mediocre-performance-2023-01-25/

Statistics Canada. (2018). National Occupational Classification (NOC) 2011. https://www23.statcan.gc.ca/imdb/p3VD.pl?Function=getVD&TVD=122372&CVD=122375&CPV=511&CST=01012011&CLV=3&MLV=4&D=1

Statistics Canada. (2022). Table 37-10-0076-01 number of full-time teaching staff at Canadian universities, by rank, sex, inactive. https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=3710007601

Sullivan, M., Kelly, A., & McLaughlan, P. (2023). ChatGPT in higher education: Considerations for academic integrity and student learning. Journal of Applied Learning & Teaching 6(1). https://journals.sfu.ca/jalt/index.php/jalt/article/view/731/559

Tauginiené, L., Gaižauskaité, I., Glendinning, I., Kravjar, J., Ojsteršek, M., Ribeiro, L., Odiņeca, T., Marino, F., Cosentino, M., Sivasubramaniam, S., Foltýnek, T. Glossary for Academic Integrity. https://www.academicintegrity.eu/wp/wp-content/uploads/2023/02/EN-Glossary_revised_final_24.02.23.pdf

Tertiary Education Quality and Standards Agency (TEQSA). (2023). Artificial intelligence. https://www.teqsa.gov.au/guides-resources/higher-education-good-practice-hub/artificial-intelligence

Tricco, A. C., Antony, J., Zarin, W., Strifler, L., Ghassemi, M., Ivory, J., Perrier, L., Hutton, B., Moher, D., & Straus, S. E. (2015). A scoping review of rapid review methods. BMC Medicine, 13(224). https://doi.org/10.1186/s12916-015-0465-6

UNESCO OECD EUROSTAT. (2001). 2001 Data Collection on Education Systems: Definitions, Explanations and Instructions.

UNESCO. (2021). The ethics of artificial intelligence. https://unesdoc.unesco.org/ark:/48223/pf0000381137

Wollscheid, S., & Tripney, J. (2021). Rapid reviews as an emerging approach to evidence synthesis in education. London Review of Education, 19(1). https://doi.org/10.14324/lre.19.1.32

Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education–where are the educators? International Journal of Educational Technology in Higher Education, 16(1), 1-27. https://doi.org/10.1186/s41239-019-0171-0

How to Cite

• Endnote/Zotero/Mendeley (RIS)

Most read articles by the same author(s)

• Beatriz Antonieta Moya, Sarah Elaine Eaton, Helen Pethrick, K. Alix Hayden, Robert Brennan, Jason Wiens, Brenda McDermott, Jonathan Lesage, Academic Integrity and Artificial Intelligence in Higher Education Contexts: A Rapid Scoping Review Protocol , Canadian Perspectives on Academic Integrity: Vol. 5 No. 2 (2023)
• Sarah Elaine Eaton, Helen Pethrick, Kristal Louise Turner, Academic Integrity and Student Mental Well-Being: A Rapid Review , Canadian Perspectives on Academic Integrity: Vol. 5 No. 2 (2023)
• Alix Hayden, Sarah Elaine Eaton, Katherine Crossman, Lee-Ann Penaluna, Bartlomiej A. Lenart, Text-matching software in post-secondary contexts: A systematic review protocol , Canadian Perspectives on Academic Integrity: Vol. 3 No. 1 (2020): Volume 3, Issue 1
• Helen Pethrick, Sarah Elaine Eaton, Kristal Louise Turner, Academic Integrity and Mental Well-being: Exploring an Unexplored Relationship , Canadian Perspectives on Academic Integrity: Vol. 4 No. 2 (2021): Volume 4, Issue 2
• Beatriz Moya, Sarah Elaine Eaton, Helen Pethrick, Robert Brennan, Jason Wiens, Brenda McDermott, Jonathan Lesage, A Rapid Scoping Review on Academic Integrity and Algorithmic Writing Technologies , Canadian Perspectives on Academic Integrity: Vol. 6 No. 1 (2023)

Developed By

• Français (Canada)

Information

• For Readers
• For Authors
• For Librarians

Schools are using research to try to improve children’s learning – but it’s not working

Senior Research Fellow in the Centre for Teachers and Teaching Research, UCL

Disclosure statement

Sally Riordan is currently working on two projects that receive funding from the Education Endowment Foundation.

University College London provides funding as a founding partner of The Conversation UK.

View all partners

Evidence is obviously a good thing. We take it for granted that evidence from research can help solve the post-lockdown crises in education – from how to keep teachers in the profession to how to improve behaviour in schools, get children back into school and protect the mental health of a generation.

But my research and that of others shows that incorporating strategies that have evidence backing them into teaching doesn’t always yield the results we want.

The Department for Education encourages school leadership teams to cite evidence from research studies when deciding how to spend school funding. Teachers are more frequently required to conduct their own research as part of their professional training than they were a decade ago. Independent consultancies have sprung up to support schools to bring evidence-based methods into their teaching.

This push for evidence to back up teaching methods has become particularly strong in the past ten years. The movement has been driven by the Education Endowment Foundation (EEF), a charity set up in 2011 with funding from the Conservative-Liberal Democrat coalition government to provide schools with information about which teaching methods and other approaches to education actually work.

The EEF funds randomised controlled trials – large-scale studies in which students are randomly assigned to an educational initiative or not and then comparisons are then made to see which students perform better. For instance, several of these studies have been carried out in which some children received one-on-one reading sessions with a trained classroom assistant, and their reading progress was compared to children who had not. The cost of one of these trials was around £500,000 over the course of a year.

Trials such as this in education were lobbied for by Ben Goldacre , a doctor and data scientist who wrote a report in 2013 on behalf of the Department for Education. Goldacre suggested that education should follow the lead of medicine in the use of evidence.

Using evidence

In 2023, however, researchers at the University of Warwick pointed out something that should have been obvious for some time but has been very much overlooked – that following the evidence is not resulting in the progress we might expect.

Reading is the most heavily supported area of the EEF’s research, accounting for more than 40% of projects . Most schools have implemented reading programmes with significant amounts of evidence behind them. But, despite this, reading abilities have not changed much in the UK for decades.

This flatlining of test scores is a global phenomenon . If reading programmes worked as the evidence says they do, reading abilities should be better.

And the evidence is coming back with unexpected results. A series of randomised controlled trials, including one looking at how to improve literacy through evidence , have suggested that schools that use methods based on research are not performing better than schools that do not.

In fact, research by a team at Sheffield Hallam University have demonstrated that on average, these kinds of education initiatives have very little to no impact .

My work has shown that when the findings of different research studies are brought together and synthesised, teachers may end up implementing these findings in contradictory ways. Research messages are frequently too vague to be effective because the skills and expertise of teaching are difficult to transfer.

It is also becoming apparent that the gains in education are usually very small, perhaps because learning is the sum total of trillions of interactions. It is possible that the research trials we really need in education would be so vast that they are currently too impractical to do.

It seems that evidence is much harder to tame and to apply sensibly in education than elsewhere. In my view, it was inevitable and necessary that educators had to follow medicine in our search for answers. But we now need to think harder about the peculiarities of how evidence works in education.

Right now, we don’t have enough evidence to be confident that evidence should always be our first port of call.

• Young people
• Department for Education

Project Officer, Student Volunteer Program

Audience Development Coordinator (fixed-term maternity cover)

Lecturer (Hindi-Urdu)

Director, Defence and Security

Opportunities with the new CIEHF

• A winning model: Bogotá’s charter schools boost students’ academic and social-emotional skills

Media Inquiries

• 615-322-6397 Email

Latest Stories

• Jessica Logan: a meta-scientist on a mission
• Update your personal devices to the secure VUIT Wi-Fi network

Apr 5, 2024, 2:04 PM

By Jenna Somers

Researchers at Vanderbilt University and William & Mary may have found a promising approach to education reform that could help provide high-quality education to students from low-income families. Results from a study on the quality of public-private partnership schools in Bogotá, Colombia—known as schools in administration—demonstrate that students enrolled in these schools have higher scores in cognitive and social-emotional skills than students of similar socio-economic and demographic backgrounds who did not enroll in these schools. Parents and guardians also reported higher satisfaction with the schools in administration and a lower likelihood of transferring to another school.

Much like charter schools in the United States, schools in administration are privately managed but publicly funded. Colombia seeks to leverage both sectors to more efficiently use public resources to provide a better-quality education to students from low-income backgrounds. To ensure the quality of these schools, the government holds them to the same accountability standards as public schools. Furthermore, non-profit organizations selected by the government to manage these schools must demonstrate prior experience managing high-quality private schools.

“The benefits in cognitive and social-emotional outcomes for students at these schools could potentially shift the approach to public education in Colombia. Based on the positive results for students and the satisfaction of parents, these schools not only deliver a quality education, but they promote community confidence,” said Felipe Barrera-Osorio , the study’s principal investigator and associate professor of public policy, education, and economics at Vanderbilt Peabody College of education and human development .

Students’ cognitive skills—those related to thinking, learning, and problem-solving—were measured by the Peabody Picture Vocabulary Test . Those with superior scores on the test demonstrated better social understanding, emotional health, and well-being. Parents of students in schools in administration seemed to notice the enhanced quality, as only 10 percent of them indicated a desire to change schools, compared to 37 percent of parents whose children were not enrolled in these schools.

Barrera-Osorio and co-PI Andrew Dustan , assistant professor of economics at William & Mary, note several key benefits of schools in administration that may support positive outcomes for students. These schools employ more full-time psychologists and provide more professional development training to teachers compared to traditional public schools. In particular, teacher trainings focus on classroom management and content, whereas teacher trainings in public schools focus more on coexistence in school and managing socio-emotional characteristics of students. That said, the researchers found that schools in administration and public schools collaborate through teacher networks to improve teaching practices.

While the findings from the study offer much promise for the future of providing cost-effective, quality education to students from low-income families in Colombia, the researchers say it is vital to further study the performance, benefits, and characteristics of schools in administration.

Importantly, this is the first study to compare students who applied to schools in administration and were assigned a spot with students who also applied and did not receive a spot. The Colombian government created a priority index of students based on socio-economic status and demographic background. The researchers formed pairs of students who each had the same score on the priority index. Through a lottery system, one student received a spot in a school in administration and the other student did not. Key to the validity of the findings is that the students’ profiles were identical, except whether they attended a school in administration.

This study was conducted in collaboration with Innovations for Poverty Action Colombia and the Bogotá Secretary of Education. It was supported by a one-year, $400,000 grant from the National Science Foundation.

Keep Reading

Peabody Scholars share guidance to inform school voucher policy

Education, bullying, mental health, school gun violence top list of parental concerns for their children: poll

Welsh’s study reveals the importance of parental trust in schools for reducing exclusionary discipline

Explore story topics.

• Education and Psychology
• Felipe Barrera-Osorio
• Ideas In Action
• Peabody College
• Peabody global engagement
• peabody-home

Making research accessible: The role of active participation in scientific inquiry in medical student education

Michael Friedlander, Fralin Biomedical Research Institute executive director and senior dean for research at the Virginia Tech Carilion School of Medicine, shares Virginia Tech’s integrated research model as a part of a panel on medical education.

Leigh Anne Kelley

• Share on Facebook
• Share on Twitter
• Copy address link to clipboard

Virginia Tech Carilion School of Medicine students spend 1,200 or more hours conducting research at the Fralin Biomedical Research Institute, Virginia Tech labs, Carilion Clinic, or with other partners. Some, like Kenneth Young, have extended their studies to earn an M.D.+Ph.D. degree. Photo by Clayton Metz for Virginia Tech.

Michael Friedlander, founding director of the Fralin Biomedical Research Institute at VTC, will stress the importance of doing research as part of medical training April 3 at an Association of American Medical Colleges conference in Arlington.

Friedlander works closely with Leslie LaConte , associate dean for research at VTC School of Medicine, who leads the research curriculum and works closely with all of the medical students on their research projects.

Friedlander will share perspectives from the experience of building of a culture at the Virginia Tech Carilion School of Medicine (VTCSOM), partly based on students actively participating in research that provides them additional opportunities to advance health and well-being for all, including people who may benefit from their findings in the future.

“Through this process, medical students are imbued with a deep appreciation of both the power and the limitations of data and how they are obtained, interpreted and applied,” Friedlander said.

“This provides them with exciting opportunities and a broadened perspective through their deep engagement with all aspects of the research enterprise, from identifying the knowledge gap and need to conceptualizing the question and approach to implementation of the methodology and interpretation and application of the findings,” Friedlander said. “Moreover, the students have the opportunity to amplify the impact of their work through presenting their findings at conferences, publishing it in peer-reviewed journals, and even commercializing it to bring their discoveries to patients.”

Friedlander has informed perspective, not only as Virginia Tech’s vice president for health sciences and technology, but as the senior dean for research at the Virginia Tech Carilion School of Medicine.

He also served on the  Scientific Foundations for Future Physicians  committee, a partnership between the Association of American Medical Colleges (AAMC) and the Howard Hughes Medical Institute, that identified the need for U.S. medical education to increase its emphasis on the emerging science of medicine and the role of the discovery process — principles that have been embraced and incorporated in the Virginia Tech Carilion School of Medicine since its inception under the leadership of previous Dean Cynda Johnson and current Dean Lee Learman.

The research institute and  Virginia Tech Carilion School of Medicine  were founded as part of a unique public-private partnership between Virginia Tech and Carilion Clinic and are in connecting, adjacent buildings on the Health Sciences and Technology campus in Roanoke.

Nearly 70 medical students selected mentors based in labs at the Fralin Biomedical Research Institute, while many others are carrying out their research at Carilion Clinic , on the Virginia Tech main campus in Blacksburg, or with mentors at the Children’s National Hospital .

“The research these students do contributes to their development as true scientists and physicians, sharpening their skills as clinicians. It is very gratifying to hear the reports of how deeply knowledgeable Virginia Tech Carilion medical students are about their research, both in integrating their clinical skills on wards as well as when they interview for residency positions after graduating,” Friedlander said. “The opportunity to share not only the VTCSOM experience but to contribute to the national dialogue as an invited panelist at the AAMC’s spring meeting is compelling and a chance to facilitate the consideration and incorporation of the VTCSOM model nationally in medical education.”

The panel discussion “How Learners can Distinguish Themselves Through Medical Research” is part of this joint conference that brings together the association's Council of Faculty and Academic Societies, the Group on Resident Affairs, and the Organization of Resident Representatives for three days of programming. 

Others who are part of the panel include Martha Alexander-Miller, professor and chair of the microbiology department at the Wake Forest University School of Medicine; Stella Hartano, a physician-scientist and allergy and immunology fellow at the National Institute of Allergy and Infectious Disease at the National Institutes of Health; and moderator Neil Osheroff, professor of biochemistry and medicine at the Vanderbilt University School of Medicine.

540-526-2002

• Faculty of Health Sciences
• Fralin Biomedical Research Institute at VTC
• Fralin Biomedical Research Institute at VTC - top news
• Friedlander News
• Graduate Education
• Graduate Research
• Health Sciences and Technology (Roanoke)
• Roanoke, Va.
• Virginia Tech Carilion School of Medicine

Related Content

College & Research Libraries News  ( C&RL News ) is the official newsmagazine and publication of record of the Association of College & Research Libraries,  providing articles on the latest trends and practices affecting academic and research libraries.

C&RL News  became an online-only publication beginning with the January 2022 issue.

C&RL News  Reader Survey

Give us your feedback in the 2024  C&RL News   reader survey ! The survey asks a series of questions today to gather your thoughts on the contents and presentation of the magazine and should only take approximately 5-7 minutes to complete. Thank you for taking the time to provide your feedback and suggestions for  C&RL News , we greatly appreciate and value your input.

Denise Brush is institutional repository and engineering/physical sciences librarian at the Rowan University Glassboro Campus Campbell Library, email: [email protected] .

ALA JobLIST

Advertising Information

• Preparing great speeches: A 10-step approach (211673 views)
• The American Civil War: A collection of free online primary sources (198022 views)
• 2018 top trends in academic libraries: A review of the trends and issues affecting academic libraries in higher education (77575 views)

The Way I See It

Denise Brush

Trust in Academic Libraries

How to Build Connections between New Co-workers

© 2024 Denise Brush

I am a tenured faculty librarian who has worked at the same academic library for 18 years. But prior to that I was a library assistant in a public library, an adjunct lecturer at two colleges, and an engineer for public and private employers. Because I have worked in these very different settings, I believe that I have a unique perspective on academic libraries.

In all my previous full-time positions, I worked with the same co-workers every day. We understood each other’s roles in the organization and knew a few things about each other’s personal lives. We trusted each other without giving it much thought. But when you don’t know your co-workers and you don’t understand why a co-worker did something, you begin to not trust them. Over time the lack of trust between individuals can become the norm in an organization. Frequent turnover of staff at all levels often leads to a work culture that lacks trust among employees. This situation can easily occur at libraries supporting rapidly growing colleges and universities unless action is taken to counteract it.

How Loss of Trust Occurs

A library where a large percentage of the employees are new must build trust. Employees who have worked together for a long time have shared understandings, but when new employees join the organization, they must develop relationships with their co-workers before there can be mutual trust. The best approach is talking to each other one-on-one to develop an understanding of each of their roles and how they contribute to the library, but this may be impractical due to different work schedules, different work locations, and lack of time.

Staff doing entry-level work in an academic library often have no idea what librarians do. They don’t understand why public services librarians are often unavailable to help walk-in patrons, because the classroom library instruction and work with faculty these librarians do is not visible to them. Similarly, librarians and staff who don’t work in technical services may not understand the complexity of managing 21st century library collections, electronic resources, and discovery systems. The divide between employees and management that exists in every organization becomes another source of distrust when large unions take over the task of negotiating with management, and direct dialogue is no longer allowed.

In small libraries, trust between co-workers develops quickly because you work regularly together. As a library grows, people begin to be siloed into different departments, and they don’t interact as much. When new employees join the library, they may not have much opportunity to interact with people in other departments and get to know them. When you are part of a large university, interacting with people directly is replaced by online forms, software platforms, and ticketing systems.

There are additional challenges in academic libraries where librarians have faculty status. Besides their primary roles, faculty librarians have research, scholarly, and service requirements to meet. It can be difficult for them to find the time to build relationships with staff members who don’t have similar obligations. When national searches are required for librarian positions, librarians may have confidential information about how their co-workers got their jobs that they can never reveal, which may make them hesitant to get to know co-workers. In this setting, career growth (whether lateral or upward) for librarians within the organization becomes nearly impossible.

Some Possible Solutions

Lack of trust can easily occur between library employees who don’t know each other. Opportunities to socialize across and within departments, such as potluck lunches or social events outside work, are important to developing trusting relationships and should be facilitated by library administration. Introverts may prefer to get to know their fellow employees through online collaboration platforms like Slack, but some form of personal connection between employees is critical. In organizations (like the places I worked previously) where co-workers know each other and are aware of the circumstances that their co-workers are experiencing outside of work, they can offer sympathy and a listening ear. These are workplaces where co-workers send flowers or a card when someone in their work family experiences a loss. As workplaces grow, that caring atmosphere can disappear unless proactive steps are taken.

New employee orientations are another important way to establish trust. They are crucial for making all new employees feel welcome. Employee orientation programs do take work and time to set up that an understaffed library may not have. But it is essential to the health of your library to find the time. New employee orientations provide several critical benefits to the new employee and to the existing staff. They help new employees understand the “big picture” of the library and where they fit in. They are an opportunity to set expectations and get an overview of administrative procedures. But orientations are also a key opportunity to explain to new non-librarian staff what librarians and library staff do. They may not realize yet that libraries are much more than places to check out books. Staff members who understand the work that is being done outside their immediate role will be more likely to trust that their co-workers are doing important, valued work.

Visibility is a touchy but important subject in academic libraries. Instruction and subject librarians whose work is outward facing need to make a special effort to make their schedules and their work visible to co-workers. Librarians who take the time to block off instruction sessions, consultations, meetings, and time off on their Outlook calendars make themselves more accessible to co-workers who want to connect with them. Those who are on the tenure/promotion track can also block out time on their calendars to work on a tenure application or a research project. Some librarians with faculty status may resent having to share their constantly changing schedules and go too far in the other direction, so that no one ever knows where they are. This type of behavior is likely to cause substantial loss of trust with co-workers.

Academic library administrators can increase trust among all their employees by providing occasional opportunities for socialization during the workday, creating new employee orientation programs that teach everyone about the role of employees in every department, and asking for schedule visibility at a reasonable level for all employees. These steps will create a happier workplace for everyone.

Article Views (Last 12 Months)

Contact ACRL for article usage statistics from 2010-April 2017.

Article Views (By Year/Month)

© 2024 Association of College and Research Libraries , a division of the American Library Association

Print ISSN: 0099-0086 | Online ISSN: 2150-6698

ALA Privacy Policy

ISSN: 2150-6698

share this!

April 3, 2024

This article has been reviewed according to Science X's editorial process and policies . Editors have highlighted the following attributes while ensuring the content's credibility:

fact-checked

trusted source

written by researcher(s)

China dominates new academic rankings based on open-access research

by Caroline Wagner, The Conversation

University leaders pay close attention to comparative rankings such as those offered by Times Higher Education , ShanghaiRanking Consultancy and others . Rankings influence student matriculation numbers, attract talented faculty and justify donations from wealthy donors . University leaders rail against them , and some schools "withdraw" from them , but rankings are influential.

A radical shift in the data underlying rankings is about to upend the rankings world—largely in favor of China's position.

For instance, in early 2024, the Leiden University Center for Science and Technology Studies CWTS group issued new university rankings that add open-data sources to the traditional curated list of elite journals that has been the standard. The results show a world turned upside down for university rankings.

Where once the list of universities with the highest scientific impact would have been Oxford, Stanford, Harvard and MIT, the new top 10 list of universities with high scientific impact includes eight universities from China. Only Harvard and the University of Toronto hold onto a top-10 spot.

What does this transformation mean for understanding scholarly excellence? I study the global research system and its contribution to social welfare. China's swift progress in science and technology , propelled by investments in research and university strength, has alarmed the United States and other nations . Concerns are mounting that the U.S. may be losing its competitive advantage to an assertive rival, with potential implications for national security, economic standing and global influence. These new rankings will likely raise even more alarm.

Broader range of more sources

The rankings programs draw heavily upon quantitative assessments called "indicators." A glance at the influential ShanghaiRanking criteria shows the inputs to its assessment include "papers indexed in major citation indices." The popular indices draw from a highly curated set of scholarly journals such as Cell , The Lancet and Chemical Reviews . The most reputed index collecting information on these and other journals is the Web of Science 's Science Citation Index, or SCI, a product of careful standardization and data enrichment by Clarivate .

SCI represents only a fraction of the work published worldwide , though. Among other critiques , many people decry the SCI's exclusivity and its perceived Western bias .

But careful curation makes it the gold standard of academic indexing and one that journals and authors aspire to join. Its value is in its replicability: It is possible to dip into it multiple times using different search strategies and produce comparable results.

Reliance on curated databases is about to end with the introduction of rankings based on open data like that collected by OpenAlex . OpenAlex claims to include over 100,000 journals—of highly varying quality and editorial practices—compared with SCI's 9,200. All data in OpenAlex has been released into the public domain with the laudable goal of making research freely available to all. The downside is that this wider net sweeps in predatory journals that exploit researchers and undermine the quality and integrity of scholarly communication.

Reflecting China's research productivity

The volume of scholarly articles represented in the open databases has a mighty influence on China's position in the open-source rankings. Chinese scholars produce a vast body of written work, some in English, some in Chinese; estimates of percentage shares for language range widely, but hover around 50-50. As China has invested in education and grown its science and engineering capacity, many more people turn out scholarly articles.

From a very small number in the 1980s, China had 2.2 million scientists and engineers by 2023, based on UNESCO data. China's scholarly output of scientific and engineering articles shows a very rapid rise since the 1990s, with growth outpacing all other nations. Quality has lagged quantity, but China is outproducing the United States in the total number of scientific publications in the Web of Science, by my count—a shift in leadership not seen since the U.S. overtook the U.K. in 1948.

Although the numbers are dated, when I counted China's scholarly publishing in 2010 , my colleague and I estimated that between 2000 and 2009, China published around 1 million scientific papers that were not captured by the Web of Science. That means they didn't "count" toward traditional rankings. These publications are counted in the new open databases. Many of the papers included in open-source or open-access journals will not be considered of high quality; nonetheless, they become part of the written record.

Open-access publishing services have grown rapidly and offer fast publication times, but there are questions about the quality of their journals. Open publishing services such as MDPI and Frontiers have an outsized number of Chinese contributors compared to those from other countries.

The open-access services often include content from potential paper mills, businesses manufacturing what look like scholarly manuscripts for sale. Despite concerns about the reputation and editorial practices of these publishers and editors, there's little oversight. These services are flooding the publishing world with vast numbers of lower-quality articles.

Chinese researchers and their sponsoring institutions put a huge premium on publishing in international journals, even those hosted by questionable publishers. Citation stacking practices —when authors cite the works of co-nationals to raise their citation profiles—skew counts to enhance China's performance.

China is attempting to address malign practices. To its credit, China's government recently announced the retraction of 17,000 articles with a Chinese author or co-author. Efforts are underway to enhance quality. Governmental payments to researchers for articles in ranked journals are being sunsetted .

Despite the quality questions, the numbers alone will push China up the rankings lists. This rapid shift will enhance China's position relative to the rest of the world. In itself, the rise does not reflect a change in quality, status or output, but it will continue to stoke the fires of those alarmed by the rise of China in world science , technology and innovation circles, and perhaps put rankings further into question.

Provided by The Conversation

Explore further

Feedback to editors

What do scientists hope to learn from total solar eclipse in US?

15 hours ago

Rare Javan rhino calf spotted in Indonesia

Scientists investigate information propagation in interacting bosonic systems

22 hours ago

DESI first-year data delivers unprecedented measurements of expanding universe

Apr 6, 2024

Saturday Citations: AI and the prisoner's dilemma; stellar cannibalism; evidence that EVs reduce atmospheric CO₂

Huge star explosion to appear in sky in once-in-a-lifetime event

Innovative sensing platform unlocks ultrahigh sensitivity in conventional sensors

Nonvolatile quantum memory: Discovery points path to flash-like memory for storing qubits

Can language models read the genome? This one decoded mRNA to make better vaccines

A simple, inexpensive way to make carbon atoms bind together

Relevant physicsforums posts, motivating high school physics students with popcorn physics.

Apr 3, 2024

How is Physics taught without Calculus?

Mar 29, 2024

Why are Physicists so informal with mathematics?

Mar 24, 2024

The changing physics curriculum in 1961

Suggestions for using math puzzles to stimulate my math students.

Mar 21, 2024

The New California Math Framework: Another Step Backwards?

Mar 14, 2024

More from STEM Educators and Teaching

Related Stories

Avalanche of published academic articles could erode trust in science

Nov 6, 2023

China now publishes more high-quality science than any other nation. Should the US be worried?

Jan 11, 2023

The death of open access mega-journals?

Mar 29, 2023

Disparity in open access practices in the earth sciences

Jun 18, 2021

China probes academic fraud by cancer researchers

Jun 15, 2017

Psychology and Wikipedia: Measuring journals' impact by Wikipedia citations

Sep 6, 2021

Recommended for you

Touchibo, a robot that fosters inclusion in education through touch

Apr 5, 2024

More than money, family and community bonds prep teens for college success: Study

Study on the psychology of blame points to promising strategies for reducing animosity within political divide

Characterizing social networks by the company they keep

Apr 2, 2024

Your emotional reaction to climate change may impact the policies you support, study finds

Mar 27, 2024

Value-added tax data could help countries prepare better for crises

Let us know if there is a problem with our content.

Use this form if you have come across a typo, inaccuracy or would like to send an edit request for the content on this page. For general inquiries, please use our contact form . For general feedback, use the public comments section below (please adhere to guidelines ).

Please select the most appropriate category to facilitate processing of your request

Thank you for taking time to provide your feedback to the editors.

Your feedback is important to us. However, we do not guarantee individual replies due to the high volume of messages.

E-mail the story

Your email address is used only to let the recipient know who sent the email. Neither your address nor the recipient's address will be used for any other purpose. The information you enter will appear in your e-mail message and is not retained by Phys.org in any form.

Newsletter sign up

Get weekly and/or daily updates delivered to your inbox. You can unsubscribe at any time and we'll never share your details to third parties.

More information Privacy policy

Donate and enjoy an ad-free experience

We keep our content available to everyone. Consider supporting Science X's mission by getting a premium account.

E-mail newsletter

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Elsevier - PMC COVID-19 Collection

A systematic review of research on online teaching and learning from 2009 to 2018

Associated data.

Systematic reviews were conducted in the nineties and early 2000's on online learning research. However, there is no review examining the broader aspect of research themes in online learning in the last decade. This systematic review addresses this gap by examining 619 research articles on online learning published in twelve journals in the last decade. These studies were examined for publication trends and patterns, research themes, research methods, and research settings and compared with the research themes from the previous decades. While there has been a slight decrease in the number of studies on online learning in 2015 and 2016, it has then continued to increase in 2017 and 2018. The majority of the studies were quantitative in nature and were examined in higher education. Online learning research was categorized into twelve themes and a framework across learner, course and instructor, and organizational levels was developed. Online learner characteristics and online engagement were examined in a high number of studies and were consistent with three of the prior systematic reviews. However, there is still a need for more research on organization level topics such as leadership, policy, and management and access, culture, equity, inclusion, and ethics and also on online instructor characteristics.

• • Twelve online learning research themes were identified in 2009–2018.
• • A framework with learner, course and instructor, and organizational levels was used.
• • Online learner characteristics and engagement were the mostly examined themes.
• • The majority of the studies used quantitative research methods and in higher education.
• • There is a need for more research on organization level topics.

1. Introduction

Online learning has been on the increase in the last two decades. In the United States, though higher education enrollment has declined, online learning enrollment in public institutions has continued to increase ( Allen & Seaman, 2017 ), and so has the research on online learning. There have been review studies conducted on specific areas on online learning such as innovations in online learning strategies ( Davis et al., 2018 ), empirical MOOC literature ( Liyanagunawardena et al., 2013 ; Veletsianos & Shepherdson, 2016 ; Zhu et al., 2018 ), quality in online education ( Esfijani, 2018 ), accessibility in online higher education ( Lee, 2017 ), synchronous online learning ( Martin et al., 2017 ), K-12 preparation for online teaching ( Moore-Adams et al., 2016 ), polychronicity in online learning ( Capdeferro et al., 2014 ), meaningful learning research in elearning and online learning environments ( Tsai, Shen, & Chiang, 2013 ), problem-based learning in elearning and online learning environments ( Tsai & Chiang, 2013 ), asynchronous online discussions ( Thomas, 2013 ), self-regulated learning in online learning environments ( Tsai, Shen, & Fan, 2013 ), game-based learning in online learning environments ( Tsai & Fan, 2013 ), and online course dropout ( Lee & Choi, 2011 ). While there have been review studies conducted on specific online learning topics, very few studies have been conducted on the broader aspect of online learning examining research themes.

2. Systematic Reviews of Distance Education and Online Learning Research

Distance education has evolved from offline to online settings with the access to internet and COVID-19 has made online learning the common delivery method across the world. Tallent-Runnels et al. (2006) reviewed research late 1990's to early 2000's, Berge and Mrozowski (2001) reviewed research 1990 to 1999, and Zawacki-Richter et al. (2009) reviewed research in 2000–2008 on distance education and online learning. Table 1 shows the research themes from previous systematic reviews on online learning research. There are some themes that re-occur in the various reviews, and there are also new themes that emerge. Though there have been reviews conducted in the nineties and early 2000's, there is no review examining the broader aspect of research themes in online learning in the last decade. Hence, the need for this systematic review which informs the research themes in online learning from 2009 to 2018. In the following sections, we review these systematic review studies in detail.

Comparison of online learning research themes from previous studies.

2.1. Distance education research themes, 1990 to 1999 ( Berge & Mrozowski, 2001 )

Berge and Mrozowski (2001) reviewed 890 research articles and dissertation abstracts on distance education from 1990 to 1999. The four distance education journals chosen by the authors to represent distance education included, American Journal of Distance Education, Distance Education, Open Learning, and the Journal of Distance Education. This review overlapped in the dates of the Tallent-Runnels et al. (2006) study. Berge and Mrozowski (2001) categorized the articles according to Sherry's (1996) ten themes of research issues in distance education: redefining roles of instructor and students, technologies used, issues of design, strategies to stimulate learning, learner characteristics and support, issues related to operating and policies and administration, access and equity, and costs and benefits.

In the Berge and Mrozowski (2001) study, more than 100 studies focused on each of the three themes: (1) design issues, (2) learner characteristics, and (3) strategies to increase interactivity and active learning. By design issues, the authors focused on instructional systems design and focused on topics such as content requirement, technical constraints, interactivity, and feedback. The next theme, strategies to increase interactivity and active learning, were closely related to design issues and focused on students’ modes of learning. Learner characteristics focused on accommodating various learning styles through customized instructional theory. Less than 50 studies focused on the three least examined themes: (1) cost-benefit tradeoffs, (2) equity and accessibility, and (3) learner support. Cost-benefit trade-offs focused on the implementation costs of distance education based on school characteristics. Equity and accessibility focused on the equity of access to distance education systems. Learner support included topics such as teacher to teacher support as well as teacher to student support.

2.2. Online learning research themes, 1993 to 2004 ( Tallent-Runnels et al., 2006 )

Tallent-Runnels et al. (2006) reviewed research on online instruction from 1993 to 2004. They reviewed 76 articles focused on online learning by searching five databases, ERIC, PsycINFO, ContentFirst, Education Abstracts, and WilsonSelect. Tallent-Runnels et al. (2006) categorized research into four themes, (1) course environment, (2) learners' outcomes, (3) learners’ characteristics, and (4) institutional and administrative factors. The first theme that the authors describe as course environment ( n  = 41, 53.9%) is an overarching theme that includes classroom culture, structural assistance, success factors, online interaction, and evaluation.

Tallent-Runnels et al. (2006) for their second theme found that studies focused on questions involving the process of teaching and learning and methods to explore cognitive and affective learner outcomes ( n  = 29, 38.2%). The authors stated that they found the research designs flawed and lacked rigor. However, the literature comparing traditional and online classrooms found both delivery systems to be adequate. Another research theme focused on learners’ characteristics ( n  = 12, 15.8%) and the synergy of learners, design of the online course, and system of delivery. Research findings revealed that online learners were mainly non-traditional, Caucasian, had different learning styles, and were highly motivated to learn. The final theme that they reported was institutional and administrative factors (n  = 13, 17.1%) on online learning. Their findings revealed that there was a lack of scholarly research in this area and most institutions did not have formal policies in place for course development as well as faculty and student support in training and evaluation. Their research confirmed that when universities offered online courses, it improved student enrollment numbers.

2.3. Distance education research themes 2000 to 2008 ( Zawacki-Richter et al., 2009 )

Zawacki-Richter et al. (2009) reviewed 695 articles on distance education from 2000 to 2008 using the Delphi method for consensus in identifying areas and classified the literature from five prominent journals. The five journals selected due to their wide scope in research in distance education included Open Learning, Distance Education, American Journal of Distance Education, the Journal of Distance Education, and the International Review of Research in Open and Distributed Learning. The reviewers examined the main focus of research and identified gaps in distance education research in this review.

Zawacki-Richter et al. (2009) classified the studies into macro, meso and micro levels focusing on 15 areas of research. The five areas of the macro-level addressed: (1) access, equity and ethics to deliver distance education for developing nations and the role of various technologies to narrow the digital divide, (2) teaching and learning drivers, markets, and professional development in the global context, (3) distance delivery systems and institutional partnerships and programs and impact of hybrid modes of delivery, (4) theoretical frameworks and models for instruction, knowledge building, and learner interactions in distance education practice, and (5) the types of preferred research methodologies. The meso-level focused on seven areas that involve: (1) management and organization for sustaining distance education programs, (2) examining financial aspects of developing and implementing online programs, (3) the challenges and benefits of new technologies for teaching and learning, (4) incentives to innovate, (5) professional development and support for faculty, (6) learner support services, and (7) issues involving quality standards and the impact on student enrollment and retention. The micro-level focused on three areas: (1) instructional design and pedagogical approaches, (2) culturally appropriate materials, interaction, communication, and collaboration among a community of learners, and (3) focus on characteristics of adult learners, socio-economic backgrounds, learning preferences, and dispositions.

The top three research themes in this review by Zawacki-Richter et al. (2009) were interaction and communities of learning ( n  = 122, 17.6%), instructional design ( n  = 121, 17.4%) and learner characteristics ( n  = 113, 16.3%). The lowest number of studies (less than 3%) were found in studies examining the following research themes, management and organization ( n  = 18), research methods in DE and knowledge transfer ( n  = 13), globalization of education and cross-cultural aspects ( n  = 13), innovation and change ( n  = 13), and costs and benefits ( n  = 12).

2.4. Online learning research themes

These three systematic reviews provide a broad understanding of distance education and online learning research themes from 1990 to 2008. However, there is an increase in the number of research studies on online learning in this decade and there is a need to identify recent research themes examined. Based on the previous systematic reviews ( Berge & Mrozowski, 2001 ; Hung, 2012 ; Tallent-Runnels et al., 2006 ; Zawacki-Richter et al., 2009 ), online learning research in this study is grouped into twelve different research themes which include Learner characteristics, Instructor characteristics, Course or program design and development, Course Facilitation, Engagement, Course Assessment, Course Technologies, Access, Culture, Equity, Inclusion, and Ethics, Leadership, Policy and Management, Instructor and Learner Support, and Learner Outcomes. Table 2 below describes each of the research themes and using these themes, a framework is derived in Fig. 1 .

Research themes in online learning.

Online learning research themes framework.

The collection of research themes is presented as a framework in Fig. 1 . The themes are organized by domain or level to underscore the nested relationship that exists. As evidenced by the assortment of themes, research can focus on any domain of delivery or associated context. The “Learner” domain captures characteristics and outcomes related to learners and their interaction within the courses. The “Course and Instructor” domain captures elements about the broader design of the course and facilitation by the instructor, and the “Organizational” domain acknowledges the contextual influences on the course. It is important to note as well that due to the nesting, research themes can cross domains. For example, the broader cultural context may be studied as it pertains to course design and development, and institutional support can include both learner support and instructor support. Likewise, engagement research can involve instructors as well as learners.

In this introduction section, we have reviewed three systematic reviews on online learning research ( Berge & Mrozowski, 2001 ; Tallent-Runnels et al., 2006 ; Zawacki-Richter et al., 2009 ). Based on these reviews and other research, we have derived twelve themes to develop an online learning research framework which is nested in three levels: learner, course and instructor, and organization.

2.5. Purpose of this research

In two out of the three previous reviews, design, learner characteristics and interaction were examined in the highest number of studies. On the other hand, cost-benefit tradeoffs, equity and accessibility, institutional and administrative factors, and globalization and cross-cultural aspects were examined in the least number of studies. One explanation for this may be that it is a function of nesting, noting that studies falling in the Organizational and Course levels may encompass several courses or many more participants within courses. However, while some research themes re-occur, there are also variations in some themes across time, suggesting the importance of research themes rise and fall over time. Thus, a critical examination of the trends in themes is helpful for understanding where research is needed most. Also, since there is no recent study examining online learning research themes in the last decade, this study strives to address that gap by focusing on recent research themes found in the literature, and also reviewing research methods and settings. Notably, one goal is to also compare findings from this decade to the previous review studies. Overall, the purpose of this study is to examine publication trends in online learning research taking place during the last ten years and compare it with the previous themes identified in other review studies. Due to the continued growth of online learning research into new contexts and among new researchers, we also examine the research methods and settings found in the studies of this review.

The following research questions are addressed in this study.

• 1. What percentage of the population of articles published in the journals reviewed from 2009 to 2018 were related to online learning and empirical?
• 2. What is the frequency of online learning research themes in the empirical online learning articles of journals reviewed from 2009 to 2018?
• 3. What is the frequency of research methods and settings that researchers employed in the empirical online learning articles of the journals reviewed from 2009 to 2018?

This five-step systematic review process described in the U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse Procedures and Standards Handbook, Version 4.0 ( 2017 ) was used in this systematic review: (a) developing the review protocol, (b) identifying relevant literature, (c) screening studies, (d) reviewing articles, and (e) reporting findings.

3.1. Data sources and search strategies

The Education Research Complete database was searched using the keywords below for published articles between the years 2009 and 2018 using both the Title and Keyword function for the following search terms.

“online learning" OR "online teaching" OR "online program" OR "online course" OR “online education”

3.2. Inclusion/exclusion criteria

The initial search of online learning research among journals in the database resulted in more than 3000 possible articles. Therefore, we limited our search to select journals that focus on publishing peer-reviewed online learning and educational research. Our aim was to capture the journals that published the most articles in online learning. However, we also wanted to incorporate the concept of rigor, so we used expert perception to identify 12 peer-reviewed journals that publish high-quality online learning research. Dissertations and conference proceedings were excluded. To be included in this systematic review, each study had to meet the screening criteria as described in Table 3 . A research study was excluded if it did not meet all of the criteria to be included.

Inclusion/Exclusion criteria.

3.3. Process flow selection of articles

Fig. 2 shows the process flow involved in the selection of articles. The search in the database Education Research Complete yielded an initial sample of 3332 articles. Targeting the 12 journals removed 2579 articles. After reviewing the abstracts, we removed 134 articles based on the inclusion/exclusion criteria. The final sample, consisting of 619 articles, was entered into the computer software MAXQDA ( VERBI Software, 2019 ) for coding.

Flowchart of online learning research selection.

3.4. Developing review protocol

A review protocol was designed as a codebook in MAXQDA ( VERBI Software, 2019 ) by the three researchers. The codebook was developed based on findings from the previous review studies and from the initial screening of the articles in this review. The codebook included 12 research themes listed earlier in Table 2 (Learner characteristics, Instructor characteristics, Course or program design and development, Course Facilitation, Engagement, Course Assessment, Course Technologies, Access, Culture, Equity, Inclusion, and Ethics, Leadership, Policy and Management, Instructor and Learner Support, and Learner Outcomes), four research settings (higher education, continuing education, K-12, corporate/military), and three research designs (quantitative, qualitative and mixed methods). Fig. 3 below is a screenshot of MAXQDA used for the coding process.

Codebook from MAXQDA.

3.5. Data coding

Research articles were coded by two researchers in MAXQDA. Two researchers independently coded 10% of the articles and then discussed and updated the coding framework. The second author who was a doctoral student coded the remaining studies. The researchers met bi-weekly to address coding questions that emerged. After the first phase of coding, we found that more than 100 studies fell into each of the categories of Learner Characteristics or Engagement, so we decided to pursue a second phase of coding and reexamine the two themes. Learner Characteristics were classified into the subthemes of Academic, Affective, Motivational, Self-regulation, Cognitive, and Demographic Characteristics. Engagement was classified into the subthemes of Collaborating, Communication, Community, Involvement, Interaction, Participation, and Presence.

3.6. Data analysis

Frequency tables were generated for each of the variables so that outliers could be examined and narrative data could be collapsed into categories. Once cleaned and collapsed into a reasonable number of categories, descriptive statistics were used to describe each of the coded elements. We first present the frequencies of publications related to online learning in the 12 journals. The total number of articles for each journal (collectively, the population) was hand-counted from journal websites, excluding editorials and book reviews. The publication trend of online learning research was also depicted from 2009 to 2018. Then, the descriptive information of the 12 themes, including the subthemes of Learner Characteristics and Engagement were provided. Finally, research themes by research settings and methodology were elaborated.

4.1. Publication trends on online learning

Publication patterns of the 619 articles reviewed from the 12 journals are presented in Table 4 . International Review of Research in Open and Distributed Learning had the highest number of publications in this review. Overall, about 8% of the articles appearing in these twelve journals consisted of online learning publications; however, several journals had concentrations of online learning articles totaling more than 20%.

Empirical online learning research articles by journal, 2009–2018.

Note . Journal's Total Article count excludes reviews and editorials.

The publication trend of online learning research is depicted in Fig. 4 . When disaggregated by year, the total frequency of publications shows an increasing trend. Online learning articles increased throughout the decade and hit a relative maximum in 2014. The greatest number of online learning articles ( n  = 86) occurred most recently, in 2018.

Online learning publication trends by year.

4.2. Online learning research themes that appeared in the selected articles

The publications were categorized into the twelve research themes identified in Fig. 1 . The frequency counts and percentages of the research themes are provided in Table 5 below. A majority of the research is categorized into the Learner domain. The fewest number of articles appears in the Organization domain.

Research themes in the online learning publications from 2009 to 2018.

The specific themes of Engagement ( n  = 179, 28.92%) and Learner Characteristics ( n  = 134, 21.65%) were most often examined in publications. These two themes were further coded to identify sub-themes, which are described in the next two sections. Publications focusing on Instructor Characteristics ( n  = 21, 3.39%) were least common in the dataset.

4.2.1. Research on engagement

The largest number of studies was on engagement in online learning, which in the online learning literature is referred to and examined through different terms. Hence, we explore this category in more detail. In this review, we categorized the articles into seven different sub-themes as examined through different lenses including presence, interaction, community, participation, collaboration, involvement, and communication. We use the term “involvement” as one of the terms since researchers sometimes broadly used the term engagement to describe their work without further description. Table 6 below provides the description, frequency, and percentages of the various studies related to engagement.

Research sub-themes on engagement.

In the sections below, we provide several examples of the different engagement sub-themes that were studied within the larger engagement theme.

Presence. This sub-theme was the most researched in engagement. With the development of the community of inquiry framework most of the studies in this subtheme examined social presence ( Akcaoglu & Lee, 2016 ; Phirangee & Malec, 2017 ; Wei et al., 2012 ), teaching presence ( Orcutt & Dringus, 2017 ; Preisman, 2014 ; Wisneski et al., 2015 ) and cognitive presence ( Archibald, 2010 ; Olesova et al., 2016 ).

Interaction . This was the second most studied theme under engagement. Researchers examined increasing interpersonal interactions ( Cung et al., 2018 ), learner-learner interactions ( Phirangee, 2016 ; Shackelford & Maxwell, 2012 ; Tawfik et al., 2018 ), peer-peer interaction ( Comer et al., 2014 ), learner-instructor interaction ( Kuo et al., 2014 ), learner-content interaction ( Zimmerman, 2012 ), interaction through peer mentoring ( Ruane & Koku, 2014 ), interaction and community building ( Thormann & Fidalgo, 2014 ), and interaction in discussions ( Ruane & Lee, 2016 ; Tibi, 2018 ).

Community. Researchers examined building community in online courses ( Berry, 2017 ), supporting a sense of community ( Jiang, 2017 ), building an online learning community of practice ( Cho, 2016 ), building an academic community ( Glazer & Wanstreet, 2011 ; Nye, 2015 ; Overbaugh & Nickel, 2011 ), and examining connectedness and rapport in an online community ( Bolliger & Inan, 2012 ; Murphy & Rodríguez-Manzanares, 2012 ; Slagter van Tryon & Bishop, 2012 ).

Participation. Researchers examined engagement through participation in a number of studies. Some of the topics include, participation patterns in online discussion ( Marbouti & Wise, 2016 ; Wise et al., 2012 ), participation in MOOCs ( Ahn et al., 2013 ; Saadatmand & Kumpulainen, 2014 ), features that influence students’ online participation ( Rye & Støkken, 2012 ) and active participation.

Collaboration. Researchers examined engagement through collaborative learning. Specific studies focused on cross-cultural collaboration ( Kumi-Yeboah, 2018 ; Yang et al., 2014 ), how virtual teams collaborate ( Verstegen et al., 2018 ), types of collaboration teams ( Wicks et al., 2015 ), tools for collaboration ( Boling et al., 2014 ), and support for collaboration ( Kopp et al., 2012 ).

Involvement. Researchers examined engaging learners through involvement in various learning activities ( Cundell & Sheepy, 2018 ), student engagement through various measures ( Dixson, 2015 ), how instructors included engagement to involve students in learning ( O'Shea et al., 2015 ), different strategies to engage the learner ( Amador & Mederer, 2013 ), and designed emotionally engaging online environments ( Koseoglu & Doering, 2011 ).

Communication. Researchers examined communication in online learning in studies using social network analysis ( Ergün & Usluel, 2016 ), using informal communication tools such as Facebook for class discussion ( Kent, 2013 ), and using various modes of communication ( Cunningham et al., 2010 ; Rowe, 2016 ). Studies have also focused on both asynchronous and synchronous aspects of communication ( Swaggerty & Broemmel, 2017 ; Yamagata-Lynch, 2014 ).

4.2.2. Research on learner characteristics

The second largest theme was learner characteristics. In this review, we explore this further to identify several aspects of learner characteristics. In this review, we categorized the learner characteristics into self-regulation characteristics, motivational characteristics, academic characteristics, affective characteristics, cognitive characteristics, and demographic characteristics. Table 7 provides the number of studies and percentages examining the various learner characteristics.

Research sub-themes on learner characteristics.

Online learning has elements that are different from the traditional face-to-face classroom and so the characteristics of the online learners are also different. Yukselturk and Top (2013) categorized online learner profile into ten aspects: gender, age, work status, self-efficacy, online readiness, self-regulation, participation in discussion list, participation in chat sessions, satisfaction, and achievement. Their categorization shows that there are differences in online learner characteristics in these aspects when compared to learners in other settings. Some of the other aspects such as participation and achievement as discussed by Yukselturk and Top (2013) are discussed in different research themes in this study. The sections below provide examples of the learner characteristics sub-themes that were studied.

Self-regulation. Several researchers have examined self-regulation in online learning. They found that successful online learners are academically motivated ( Artino & Stephens, 2009 ), have academic self-efficacy ( Cho & Shen, 2013 ), have grit and intention to succeed ( Wang & Baker, 2018 ), have time management and elaboration strategies ( Broadbent, 2017 ), set goals and revisit course content ( Kizilcec et al., 2017 ), and persist ( Glazer & Murphy, 2015 ). Researchers found a positive relationship between learner's self-regulation and interaction ( Delen et al., 2014 ) and self-regulation and communication and collaboration ( Barnard et al., 2009 ).

Motivation. Researchers focused on motivation of online learners including different motivation levels of online learners ( Li & Tsai, 2017 ), what motivated online learners ( Chaiprasurt & Esichaikul, 2013 ), differences in motivation of online learners ( Hartnett et al., 2011 ), and motivation when compared to face to face learners ( Paechter & Maier, 2010 ). Harnett et al. (2011) found that online learner motivation was complex, multifaceted, and sensitive to situational conditions.

Academic. Several researchers have focused on academic aspects for online learner characteristics. Readiness for online learning has been examined as an academic factor by several researchers ( Buzdar et al., 2016 ; Dray et al., 2011 ; Wladis & Samuels, 2016 ; Yu, 2018 ) specifically focusing on creating and validating measures to examine online learner readiness including examining students emotional intelligence as a measure of student readiness for online learning. Researchers have also examined other academic factors such as academic standing ( Bradford & Wyatt, 2010 ), course level factors ( Wladis et al., 2014 ) and academic skills in online courses ( Shea & Bidjerano, 2014 ).

Affective. Anderson and Bourke (2013) describe affective characteristics through which learners express feelings or emotions. Several research studies focused on the affective characteristics of online learners. Learner satisfaction for online learning has been examined by several researchers ( Cole et al., 2014 ; Dziuban et al., 2015 ; Kuo et al., 2013 ; Lee, 2014a ) along with examining student emotions towards online assessment ( Kim et al., 2014 ).

Cognitive. Researchers have also examined cognitive aspects of learner characteristics including meta-cognitive skills, cognitive variables, higher-order thinking, cognitive density, and critical thinking ( Chen & Wu, 2012 ; Lee, 2014b ). Lee (2014b) examined the relationship between cognitive presence density and higher-order thinking skills. Chen and Wu (2012) examined the relationship between cognitive and motivational variables in an online system for secondary physical education.

Demographic. Researchers have examined various demographic factors in online learning. Several researchers have examined gender differences in online learning ( Bayeck et al., 2018 ; Lowes et al., 2016 ; Yukselturk & Bulut, 2009 ), ethnicity, age ( Ke & Kwak, 2013 ), and minority status ( Yeboah & Smith, 2016 ) of online learners.

4.2.3. Less frequently studied research themes

While engagement and learner characteristics were studied the most, other themes were less often studied in the literature and are presented here, according to size, with general descriptions of the types of research examined for each.

Evaluation and Quality Assurance. There were 38 studies (6.14%) published in the theme of evaluation and quality assurance. Some of the studies in this theme focused on course quality standards, using quality matters to evaluate quality, using the CIPP model for evaluation, online learning system evaluation, and course and program evaluations.

Course Technologies. There were 35 studies (5.65%) published in the course technologies theme. Some of the studies examined specific technologies such as Edmodo, YouTube, Web 2.0 tools, wikis, Twitter, WebCT, Screencasts, and Web conferencing systems in the online learning context.

Course Facilitation. There were 34 studies (5.49%) published in the course facilitation theme. Some of the studies in this theme examined facilitation strategies and methods, experiences of online facilitators, and online teaching methods.

Institutional Support. There were 33 studies (5.33%) published in the institutional support theme which included support for both the instructor and learner. Some of the studies on instructor support focused on training new online instructors, mentoring programs for faculty, professional development resources for faculty, online adjunct faculty training, and institutional support for online instructors. Studies on learner support focused on learning resources for online students, cognitive and social support for online learners, and help systems for online learner support.

Learner Outcome. There were 32 studies (5.17%) published in the learner outcome theme. Some of the studies that were examined in this theme focused on online learner enrollment, completion, learner dropout, retention, and learner success.

Course Assessment. There were 30 studies (4.85%) published in the course assessment theme. Some of the studies in the course assessment theme examined online exams, peer assessment and peer feedback, proctoring in online exams, and alternative assessments such as eportfolio.

Access, Culture, Equity, Inclusion, and Ethics. There were 29 studies (4.68%) published in the access, culture, equity, inclusion, and ethics theme. Some of the studies in this theme examined online learning across cultures, multi-cultural effectiveness, multi-access, and cultural diversity in online learning.

Leadership, Policy, and Management. There were 27 studies (4.36%) published in the leadership, policy, and management theme. Some of the studies on leadership, policy, and management focused on online learning leaders, stakeholders, strategies for online learning leadership, resource requirements, university policies for online course policies, governance, course ownership, and faculty incentives for online teaching.

Course Design and Development. There were 27 studies (4.36%) published in the course design and development theme. Some of the studies examined in this theme focused on design elements, design issues, design process, design competencies, design considerations, and instructional design in online courses.

Instructor Characteristics. There were 21 studies (3.39%) published in the instructor characteristics theme. Some of the studies in this theme were on motivation and experiences of online instructors, ability to perform online teaching duties, roles of online instructors, and adjunct versus full-time online instructors.

4.3. Research settings and methodology used in the studies

The research methods used in the studies were classified into quantitative, qualitative, and mixed methods ( Harwell, 2012 , pp. 147–163). The research setting was categorized into higher education, continuing education, K-12, and corporate/military. As shown in Table A in the appendix, the vast majority of the publications used higher education as the research setting ( n  = 509, 67.6%). Table B in the appendix shows that approximately half of the studies adopted the quantitative method ( n  = 324, 43.03%), followed by the qualitative method ( n  = 200, 26.56%). Mixed methods account for the smallest portion ( n  = 95, 12.62%).

Table A shows that the patterns of the four research settings were approximately consistent across the 12 themes except for the theme of Leaner Outcome and Institutional Support. Continuing education had a higher relative frequency in Learner Outcome (0.28) and K-12 had a higher relative frequency in Institutional Support (0.33) compared to the frequencies they had in the total themes (0.09 and 0.08 respectively). Table B in the appendix shows that the distribution of the three methods were not consistent across the 12 themes. While quantitative studies and qualitative studies were roughly evenly distributed in Engagement, they had a large discrepancy in Learner Characteristics. There were 100 quantitative studies; however, only 18 qualitative studies published in the theme of Learner Characteristics.

In summary, around 8% of the articles published in the 12 journals focus on online learning. Online learning publications showed a tendency of increase on the whole in the past decade, albeit fluctuated, with the greatest number occurring in 2018. Among the 12 research themes related to online learning, the themes of Engagement and Learner Characteristics were studied the most and the theme of Instructor Characteristics was studied the least. Most studies were conducted in the higher education setting and approximately half of the studies used the quantitative method. Looking at the 12 themes by setting and method, we found that the patterns of the themes by setting or by method were not consistent across the 12 themes.

The quality of our findings was ensured by scientific and thorough searches and coding consistency. The selection of the 12 journals provides evidence of the representativeness and quality of primary studies. In the coding process, any difficulties and questions were resolved by consultations with the research team at bi-weekly meetings, which ensures the intra-rater and interrater reliability of coding. All these approaches guarantee the transparency and replicability of the process and the quality of our results.

5. Discussion

This review enabled us to identify the online learning research themes examined from 2009 to 2018. In the section below, we review the most studied research themes, engagement and learner characteristics along with implications, limitations, and directions for future research.

5.1. Most studied research themes

Three out of the four systematic reviews informing the design of the present study found that online learner characteristics and online engagement were examined in a high number of studies. In this review, about half of the studies reviewed (50.57%) focused on online learner characteristics or online engagement. This shows the continued importance of these two themes. In the Tallent-Runnels et al.’s (2006) study, the learner characteristics theme was identified as least studied for which they state that researchers are beginning to investigate learner characteristics in the early days of online learning.

One of the differences found in this review is that course design and development was examined in the least number of studies in this review compared to two prior systematic reviews ( Berge & Mrozowski, 2001 ; Zawacki-Richter et al., 2009 ). Zawacki-Richter et al. did not use a keyword search but reviewed all the articles in five different distance education journals. Berge and Mrozowski (2001) included a research theme called design issues to include all aspects of instructional systems design in distance education journals. In our study, in addition to course design and development, we also had focused themes on learner outcomes, course facilitation, course assessment and course evaluation. These are all instructional design focused topics and since we had multiple themes focusing on instructional design topics, the course design and development category might have resulted in fewer studies. There is still a need for more studies to focus on online course design and development.

5.2. Least frequently studied research themes

Three out of the four systematic reviews discussed in the opening of this study found management and organization factors to be least studied. In this review, Leadership, Policy, and Management was studied among 4.36% of the studies and Access, Culture, Equity, Inclusion, and Ethics was studied among 4.68% of the studies in the organizational level. The theme on Equity and accessibility was also found to be the least studied theme in the Berge and Mrozowski (2001) study. In addition, instructor characteristics was the least examined research theme among the twelve themes studied in this review. Only 3.39% of the studies were on instructor characteristics. While there were some studies examining instructor motivation and experiences, instructor ability to teach online, online instructor roles, and adjunct versus full-time online instructors, there is still a need to examine topics focused on instructors and online teaching. This theme was not included in the prior reviews as the focus was more on the learner and the course but not on the instructor. While it is helpful to see research evolving on instructor focused topics, there is still a need for more research on the online instructor.

5.3. Comparing research themes from current study to previous studies

The research themes from this review were compared with research themes from previous systematic reviews, which targeted prior decades. Table 8 shows the comparison.

Comparison of most and least studied online learning research themes from current to previous reviews.

L = Learner, C=Course O=Organization.

5.4. Need for more studies on organizational level themes of online learning

In this review there is a greater concentration of studies focused on Learner domain topics, and reduced attention to broader more encompassing research themes that fall into the Course and Organization domains. There is a need for organizational level topics such as Access, Culture, Equity, Inclusion and Ethics, and Leadership, Policy and Management to be researched on within the context of online learning. Examination of access, culture, equity, inclusion and ethics is very important to support diverse online learners, particularly with the rapid expansion of online learning across all educational levels. This was also least studied based on Berge and Mrozowski (2001) systematic review.

The topics on leadership, policy and management were least studied both in this review and also in the Tallent-Runnels et al. (2006) and Zawacki-Richter et al. (2009) study. Tallent-Runnels categorized institutional and administrative aspects into institutional policies, institutional support, and enrollment effects. While we included support as a separate category, in this study leadership, policy and management were combined. There is still a need for research on leadership of those who manage online learning, policies for online education, and managing online programs. In the Zawacki-Richter et al. (2009) study, only a few studies examined management and organization focused topics. They also found management and organization to be strongly correlated with costs and benefits. In our study, costs and benefits were collectively included as an aspect of management and organization and not as a theme by itself. These studies will provide research-based evidence for online education administrators.

6. Limitations

As with any systematic review, there are limitations to the scope of the review. The search is limited to twelve journals in the field that typically include research on online learning. These manuscripts were identified by searching the Education Research Complete database which focuses on education students, professionals, and policymakers. Other discipline-specific journals as well as dissertations and proceedings were not included due to the volume of articles. Also, the search was performed using five search terms “online learning" OR "online teaching" OR "online program" OR "online course" OR “online education” in title and keyword. If authors did not include these terms, their respective work may have been excluded from this review even if it focused on online learning. While these terms are commonly used in North America, it may not be commonly used in other parts of the world. Additional studies may exist outside this scope.

The search strategy also affected how we presented results and introduced limitations regarding generalization. We identified that only 8% of the articles published in these journals were related to online learning; however, given the use of search terms to identify articles within select journals it was not feasible to identify the total number of research-based articles in the population. Furthermore, our review focused on the topics and general methods of research and did not systematically consider the quality of the published research. Lastly, some journals may have preferences for publishing studies on a particular topic or that use a particular method (e.g., quantitative methods), which introduces possible selection and publication biases which may skew the interpretation of results due to over/under representation. Future studies are recommended to include more journals to minimize the selection bias and obtain a more representative sample.

Certain limitations can be attributed to the coding process. Overall, the coding process for this review worked well for most articles, as each tended to have an individual or dominant focus as described in the abstracts, though several did mention other categories which likely were simultaneously considered to a lesser degree. However, in some cases, a dominant theme was not as apparent and an effort to create mutually exclusive groups for clearer interpretation the coders were occasionally forced to choose between two categories. To facilitate this coding, the full-texts were used to identify a study focus through a consensus seeking discussion among all authors. Likewise, some studies focused on topics that we have associated with a particular domain, but the design of the study may have promoted an aggregated examination or integrated factors from multiple domains (e.g., engagement). Due to our reliance on author descriptions, the impact of construct validity is likely a concern that requires additional exploration. Our final grouping of codes may not have aligned with the original author's description in the abstract. Additionally, coding of broader constructs which disproportionately occur in the Learner domain, such as learner outcomes, learner characteristics, and engagement, likely introduced bias towards these codes when considering studies that involved multiple domains. Additional refinement to explore the intersection of domains within studies is needed.

7. Implications and future research

One of the strengths of this review is the research categories we have identified. We hope these categories will support future researchers and identify areas and levels of need for future research. Overall, there is some agreement on research themes on online learning research among previous reviews and this one, at the same time there are some contradicting findings. We hope the most-researched themes and least-researched themes provide authors a direction on the importance of research and areas of need to focus on.

The leading themes found in this review is online engagement research. However, presentation of this research was inconsistent, and often lacked specificity. This is not unique to online environments, but the nuances of defining engagement in an online environment are unique and therefore need further investigation and clarification. This review points to seven distinct classifications of online engagement. Further research on engagement should indicate which type of engagement is sought. This level of specificity is necessary to establish instruments for measuring engagement and ultimately testing frameworks for classifying engagement and promoting it in online environments. Also, it might be of importance to examine the relationship between these seven sub-themes of engagement.

Additionally, this review highlights growing attention to learner characteristics, which constitutes a shift in focus away from instructional characteristics and course design. Although this is consistent with the focus on engagement, the role of the instructor, and course design with respect to these outcomes remains important. Results of the learner characteristics and engagement research paired with course design will have important ramifications for the use of teaching and learning professionals who support instruction. Additionally, the review also points to a concentration of research in the area of higher education. With an immediate and growing emphasis on online learning in K-12 and corporate settings, there is a critical need for further investigation in these settings.

Lastly, because the present review did not focus on the overall effect of interventions, opportunities exist for dedicated meta-analyses. Particular attention to research on engagement and learner characteristics as well as how these vary by study design and outcomes would be logical additions to the research literature.

8. Conclusion

This systematic review builds upon three previous reviews which tackled the topic of online learning between 1990 and 2010 by extending the timeframe to consider the most recent set of published research. Covering the most recent decade, our review of 619 articles from 12 leading online learning journal points to a more concentrated focus on the learner domain including engagement and learner characteristics, with more limited attention to topics pertaining to the classroom or organizational level. The review highlights an opportunity for the field to clarify terminology concerning online learning research, particularly in the areas of learner outcomes where there is a tendency to classify research more generally (e.g., engagement). Using this sample of published literature, we provide a possible taxonomy for categorizing this research using subcategories. The field could benefit from a broader conversation about how these categories can shape a comprehensive framework for online learning research. Such efforts will enable the field to effectively prioritize research aims over time and synthesize effects.

Credit author statement

Florence Martin: Conceptualization; Writing - original draft, Writing - review & editing Preparation, Supervision, Project administration. Ting Sun: Methodology, Formal analysis, Writing - original draft, Writing - review & editing. Carl Westine: Methodology, Formal analysis, Writing - original draft, Writing - review & editing, Supervision

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

1 Includes articles that are cited in this manuscript and also included in the systematic review. The entire list of 619 articles used in the systematic review can be obtained by emailing the authors.*

Appendix B Supplementary data to this article can be found online at https://doi.org/10.1016/j.compedu.2020.104009 .

Appendix A. 

Research Themes by the Settings in the Online Learning Publications

Research Themes by the Methodology in the Online Learning Publications

Appendix B. Supplementary data

The following are the Supplementary data to this article:

References 1

• Ahn J., Butler B.S., Alam A., Webster S.A. Learner participation and engagement in open online courses: Insights from the Peer 2 Peer University. MERLOT Journal of Online Learning and Teaching. 2013; 9 (2):160–171. * [ Google Scholar ]
• Akcaoglu M., Lee E. Increasing social presence in online learning through small group discussions. International Review of Research in Open and Distance Learning. 2016; 17 (3) * [ Google Scholar ]
• Allen I.E., Seaman J. Babson survey research group; 2017. Digital compass learning: Distance education enrollment Report 2017. [ Google Scholar ]
• Amador J.A., Mederer H. Migrating successful student engagement strategies online: Opportunities and challenges using jigsaw groups and problem-based learning. Journal of Online Learning and Teaching. 2013; 9 (1):89. * [ Google Scholar ]
• Anderson L.W., Bourke S.F. Routledge; 2013. Assessing affective characteristics in the schools. [ Google Scholar ]
• Archibald D. Fostering the development of cognitive presence: Initial findings using the community of inquiry survey instrument. The Internet and Higher Education. 2010; 13 (1–2):73–74. * [ Google Scholar ]
• Artino A.R., Jr., Stephens J.M. Academic motivation and self-regulation: A comparative analysis of undergraduate and graduate students learning online. The Internet and Higher Education. 2009; 12 (3–4):146–151. [ Google Scholar ]
• Barnard L., Lan W.Y., To Y.M., Paton V.O., Lai S.L. Measuring self-regulation in online and blended learning environments. Internet and Higher Education. 2009; 12 (1):1–6. * [ Google Scholar ]
• Bayeck R.Y., Hristova A., Jablokow K.W., Bonafini F. Exploring the relevance of single‐gender group formation: What we learn from a massive open online course (MOOC) British Journal of Educational Technology. 2018; 49 (1):88–100. * [ Google Scholar ]
• Berge Z., Mrozowski S. Review of research in distance education, 1990 to 1999. American Journal of Distance Education. 2001; 15 (3):5–19. doi: 10.1080/08923640109527090. [ CrossRef ] [ Google Scholar ]
• Berry S. Building community in online doctoral classrooms: Instructor practices that support community. Online Learning. 2017; 21 (2):n2. * [ Google Scholar ]
• Boling E.C., Holan E., Horbatt B., Hough M., Jean-Louis J., Khurana C., Spiezio C. Using online tools for communication and collaboration: Understanding educators' experiences in an online course. The Internet and Higher Education. 2014; 23 :48–55. * [ Google Scholar ]
• Bolliger D.U., Inan F.A. Development and validation of the online student connectedness survey (OSCS) International Review of Research in Open and Distance Learning. 2012; 13 (3):41–65. * [ Google Scholar ]
• Bradford G., Wyatt S. Online learning and student satisfaction: Academic standing, ethnicity and their influence on facilitated learning, engagement, and information fluency. The Internet and Higher Education. 2010; 13 (3):108–114. * [ Google Scholar ]
• Broadbent J. Comparing online and blended learner's self-regulated learning strategies and academic performance. The Internet and Higher Education. 2017; 33 :24–32. [ Google Scholar ]
• Buzdar M., Ali A., Tariq R. Emotional intelligence as a determinant of readiness for online learning. International Review of Research in Open and Distance Learning. 2016; 17 (1) * [ Google Scholar ]
• Capdeferro N., Romero M., Barberà E. Polychronicity: Review of the literature and a new configuration for the study of this hidden dimension of online learning. Distance Education. 2014; 35 (3):294–310. [ Google Scholar ]
• Chaiprasurt C., Esichaikul V. Enhancing motivation in online courses with mobile communication tool support: A comparative study. International Review of Research in Open and Distance Learning. 2013; 14 (3):377–401. [ Google Scholar ]
• Chen C.H., Wu I.C. The interplay between cognitive and motivational variables in a supportive online learning system for secondary physical education. Computers & Education. 2012; 58 (1):542–550. * [ Google Scholar ]
• Cho H. Under co-construction: An online community of practice for bilingual pre-service teachers. Computers & Education. 2016; 92 :76–89. * [ Google Scholar ]
• Cho M.H., Shen D. Self-regulation in online learning. Distance Education. 2013; 34 (3):290–301. [ Google Scholar ]
• Cole M.T., Shelley D.J., Swartz L.B. Online instruction, e-learning, and student satisfaction: A three-year study. International Review of Research in Open and Distance Learning. 2014; 15 (6) * [ Google Scholar ]
• Comer D.K., Clark C.R., Canelas D.A. Writing to learn and learning to write across the disciplines: Peer-to-peer writing in introductory-level MOOCs. International Review of Research in Open and Distance Learning. 2014; 15 (5):26–82. * [ Google Scholar ]
• Cundell A., Sheepy E. Student perceptions of the most effective and engaging online learning activities in a blended graduate seminar. Online Learning. 2018; 22 (3):87–102. * [ Google Scholar ]
• Cung B., Xu D., Eichhorn S. Increasing interpersonal interactions in an online course: Does increased instructor email activity and voluntary meeting time in a physical classroom facilitate student learning? Online Learning. 2018; 22 (3):193–215. [ Google Scholar ]
• Cunningham U.M., Fägersten K.B., Holmsten E. Can you hear me, Hanoi?" Compensatory mechanisms employed in synchronous net-based English language learning. International Review of Research in Open and Distance Learning. 2010; 11 (1):161–177. [ Google Scholar ]
• Davis D., Chen G., Hauff C., Houben G.J. Activating learning at scale: A review of innovations in online learning strategies. Computers & Education. 2018; 125 :327–344. [ Google Scholar ]
• Delen E., Liew J., Willson V. Effects of interactivity and instructional scaffolding on learning: Self-regulation in online video-based environments. Computers & Education. 2014; 78 :312–320. [ Google Scholar ]
• Dixson M.D. Measuring student engagement in the online course: The Online Student Engagement scale (OSE) Online Learning. 2015; 19 (4):n4. * [ Google Scholar ]
• Dray B.J., Lowenthal P.R., Miszkiewicz M.J., Ruiz‐Primo M.A., Marczynski K. Developing an instrument to assess student readiness for online learning: A validation study. Distance Education. 2011; 32 (1):29–47. * [ Google Scholar ]
• Dziuban C., Moskal P., Thompson J., Kramer L., DeCantis G., Hermsdorfer A. Student satisfaction with online learning: Is it a psychological contract? Online Learning. 2015; 19 (2):n2. * [ Google Scholar ]
• Ergün E., Usluel Y.K. An analysis of density and degree-centrality according to the social networking structure formed in an online learning environment. Journal of Educational Technology & Society. 2016; 19 (4):34–46. * [ Google Scholar ]
• Esfijani A. Measuring quality in online education: A meta-synthesis. American Journal of Distance Education. 2018; 32 (1):57–73. [ Google Scholar ]
• Glazer H.R., Murphy J.A. Optimizing success: A model for persistence in online education. American Journal of Distance Education. 2015; 29 (2):135–144. [ Google Scholar ]
• Glazer H.R., Wanstreet C.E. Connection to the academic community: Perceptions of students in online education. Quarterly Review of Distance Education. 2011; 12 (1):55. * [ Google Scholar ]
• Hartnett M., George A.S., Dron J. Examining motivation in online distance learning environments: Complex, multifaceted and situation-dependent. International Review of Research in Open and Distance Learning. 2011; 12 (6):20–38. [ Google Scholar ]
• Harwell M.R. 2012. Research design in qualitative/quantitative/mixed methods. Section III. Opportunities and challenges in designing and conducting inquiry. [ Google Scholar ]
• Hung J.L. Trends of e‐learning research from 2000 to 2008: Use of text mining and bibliometrics. British Journal of Educational Technology. 2012; 43 (1):5–16. [ Google Scholar ]
• Jiang W. Interdependence of roles, role rotation, and sense of community in an online course. Distance Education. 2017; 38 (1):84–105. [ Google Scholar ]
• Ke F., Kwak D. Online learning across ethnicity and age: A study on learning interaction participation, perception, and learning satisfaction. Computers & Education. 2013; 61 :43–51. [ Google Scholar ]
• Kent M. Changing the conversation: Facebook as a venue for online class discussion in higher education. MERLOT Journal of Online Learning and Teaching. 2013; 9 (4):546–565. * [ Google Scholar ]
• Kim C., Park S.W., Cozart J. Affective and motivational factors of learning in online mathematics courses. British Journal of Educational Technology. 2014; 45 (1):171–185. [ Google Scholar ]
• Kizilcec R.F., Pérez-Sanagustín M., Maldonado J.J. Self-regulated learning strategies predict learner behavior and goal attainment in Massive Open Online Courses. Computers & Education. 2017; 104 :18–33. [ Google Scholar ]
• Kopp B., Matteucci M.C., Tomasetto C. E-tutorial support for collaborative online learning: An explorative study on experienced and inexperienced e-tutors. Computers & Education. 2012; 58 (1):12–20. [ Google Scholar ]
• Koseoglu S., Doering A. Understanding complex ecologies: An investigation of student experiences in adventure learning programs. Distance Education. 2011; 32 (3):339–355. * [ Google Scholar ]
• Kumi-Yeboah A. Designing a cross-cultural collaborative online learning framework for online instructors. Online Learning. 2018; 22 (4):181–201. * [ Google Scholar ]
• Kuo Y.C., Walker A.E., Belland B.R., Schroder K.E. A predictive study of student satisfaction in online education programs. International Review of Research in Open and Distance Learning. 2013; 14 (1):16–39. * [ Google Scholar ]
• Kuo Y.C., Walker A.E., Schroder K.E., Belland B.R. Interaction, Internet self-efficacy, and self-regulated learning as predictors of student satisfaction in online education courses. Internet and Higher Education. 2014; 20 :35–50. * [ Google Scholar ]
• Lee J. An exploratory study of effective online learning: Assessing satisfaction levels of graduate students of mathematics education associated with human and design factors of an online course. International Review of Research in Open and Distance Learning. 2014; 15 (1) [ Google Scholar ]
• Lee S.M. The relationships between higher order thinking skills, cognitive density, and social presence in online learning. The Internet and Higher Education. 2014; 21 :41–52. * [ Google Scholar ]
• Lee K. Rethinking the accessibility of online higher education: A historical review. The Internet and Higher Education. 2017; 33 :15–23. [ Google Scholar ]
• Lee Y., Choi J. A review of online course dropout research: Implications for practice and future research. Educational Technology Research & Development. 2011; 59 (5):593–618. [ Google Scholar ]
• Li L.Y., Tsai C.C. Accessing online learning material: Quantitative behavior patterns and their effects on motivation and learning performance. Computers & Education. 2017; 114 :286–297. [ Google Scholar ]
• Liyanagunawardena T., Adams A., Williams S. MOOCs: A systematic study of the published literature 2008-2012. International Review of Research in Open and Distance Learning. 2013; 14 (3):202–227. [ Google Scholar ]
• Lowes S., Lin P., Kinghorn B.R. Gender differences in online high school courses. Online Learning. 2016; 20 (4):100–117. [ Google Scholar ]
• Marbouti F., Wise A.F. Starburst: A new graphical interface to support purposeful attention to others' posts in online discussions. Educational Technology Research & Development. 2016; 64 (1):87–113. * [ Google Scholar ]
• Martin F., Ahlgrim-Delzell L., Budhrani K. Systematic review of two decades (1995 to 2014) of research on synchronous online learning. American Journal of Distance Education. 2017; 31 (1):3–19. [ Google Scholar ]
• Moore-Adams B.L., Jones W.M., Cohen J. Learning to teach online: A systematic review of the literature on K-12 teacher preparation for teaching online. Distance Education. 2016; 37 (3):333–348. [ Google Scholar ]
• Murphy E., Rodríguez-Manzanares M.A. Rapport in distance education. International Review of Research in Open and Distance Learning. 2012; 13 (1):167–190. * [ Google Scholar ]
• Nye A. Building an online academic learning community among undergraduate students. Distance Education. 2015; 36 (1):115–128. * [ Google Scholar ]
• Olesova L., Slavin M., Lim J. Exploring the effect of scripted roles on cognitive presence in asynchronous online discussions. Online Learning. 2016; 20 (4):34–53. * [ Google Scholar ]
• Orcutt J.M., Dringus L.P. Beyond being there: Practices that establish presence, engage students and influence intellectual curiosity in a structured online learning environment. Online Learning. 2017; 21 (3):15–35. * [ Google Scholar ]
• Overbaugh R.C., Nickel C.E. A comparison of student satisfaction and value of academic community between blended and online sections of a university-level educational foundations course. The Internet and Higher Education. 2011; 14 (3):164–174. * [ Google Scholar ]
• O'Shea S., Stone C., Delahunty J. “I ‘feel’like I am at university even though I am online.” Exploring how students narrate their engagement with higher education institutions in an online learning environment. Distance Education. 2015; 36 (1):41–58. * [ Google Scholar ]
• Paechter M., Maier B. Online or face-to-face? Students' experiences and preferences in e-learning. Internet and Higher Education. 2010; 13 (4):292–297. [ Google Scholar ]
• Phirangee K. Students' perceptions of learner-learner interactions that weaken a sense of community in an online learning environment. Online Learning. 2016; 20 (4):13–33. * [ Google Scholar ]
• Phirangee K., Malec A. Othering in online learning: An examination of social presence, identity, and sense of community. Distance Education. 2017; 38 (2):160–172. * [ Google Scholar ]
• Preisman K.A. Teaching presence in online education: From the instructor's point of view. Online Learning. 2014; 18 (3):n3. * [ Google Scholar ]
• Rowe M. Developing graduate attributes in an open online course. British Journal of Educational Technology. 2016; 47 (5):873–882. * [ Google Scholar ]
• Ruane R., Koku E.F. Social network analysis of undergraduate education student interaction in online peer mentoring settings. Journal of Online Learning and Teaching. 2014; 10 (4):577–589. * [ Google Scholar ]
• Ruane R., Lee V.J. Analysis of discussion board interaction in an online peer mentoring site. Online Learning. 2016; 20 (4):79–99. * [ Google Scholar ]
• Rye S.A., Støkken A.M. The implications of the local context in global virtual education. International Review of Research in Open and Distance Learning. 2012; 13 (1):191–206. * [ Google Scholar ]
• Saadatmand M., Kumpulainen K. Participants' perceptions of learning and networking in connectivist MOOCs. Journal of Online Learning and Teaching. 2014; 10 (1):16. * [ Google Scholar ]
• Shackelford J.L., Maxwell M. Sense of community in graduate online education: Contribution of learner to learner interaction. International Review of Research in Open and Distance Learning. 2012; 13 (4):228–249. * [ Google Scholar ]
• Shea P., Bidjerano T. Does online learning impede degree completion? A national study of community college students. Computers & Education. 2014; 75 :103–111. * [ Google Scholar ]
• Sherry L. Issues in distance learning. International Journal of Educational Telecommunications. 1996; 1 (4):337–365. [ Google Scholar ]
• Slagter van Tryon P.J., Bishop M.J. Evaluating social connectedness online: The design and development of the social perceptions in learning contexts instrument. Distance Education. 2012; 33 (3):347–364. * [ Google Scholar ]
• Swaggerty E.A., Broemmel A.D. Authenticity, relevance, and connectedness: Graduate students' learning preferences and experiences in an online reading education course. The Internet and Higher Education. 2017; 32 :80–86. * [ Google Scholar ]
• Tallent-Runnels M.K., Thomas J.A., Lan W.Y., Cooper S., Ahern T.C., Shaw S.M., Liu X. Teaching courses online: A review of the research. Review of Educational Research. 2006; 76 (1):93–135. doi: 10.3102/00346543076001093. [ CrossRef ] [ Google Scholar ]
• Tawfik A.A., Giabbanelli P.J., Hogan M., Msilu F., Gill A., York C.S. Effects of success v failure cases on learner-learner interaction. Computers & Education. 2018; 118 :120–132. [ Google Scholar ]
• Thomas J. Exploring the use of asynchronous online discussion in health care education: A literature review. Computers & Education. 2013; 69 :199–215. [ Google Scholar ]
• Thormann J., Fidalgo P. Guidelines for online course moderation and community building from a student's perspective. Journal of Online Learning and Teaching. 2014; 10 (3):374–388. * [ Google Scholar ]
• Tibi M.H. Computer science students' attitudes towards the use of structured and unstructured discussion forums in fully online courses. Online Learning. 2018; 22 (1):93–106. * [ Google Scholar ]
• Tsai C.W., Chiang Y.C. Research trends in problem‐based learning (pbl) research in e‐learning and online education environments: A review of publications in SSCI‐indexed journals from 2004 to 2012. British Journal of Educational Technology. 2013; 44 (6):E185–E190. [ Google Scholar ]
• Tsai C.W., Fan Y.T. Research trends in game‐based learning research in online learning environments: A review of studies published in SSCI‐indexed journals from 2003 to 2012. British Journal of Educational Technology. 2013; 44 (5):E115–E119. [ Google Scholar ]
• Tsai C.W., Shen P.D., Chiang Y.C. Research trends in meaningful learning research on e‐learning and online education environments: A review of studies published in SSCI‐indexed journals from 2003 to 2012. British Journal of Educational Technology. 2013; 44 (6):E179–E184. [ Google Scholar ]
• Tsai C.W., Shen P.D., Fan Y.T. Research trends in self‐regulated learning research in online learning environments: A review of studies published in selected journals from 2003 to 2012. British Journal of Educational Technology. 2013; 44 (5):E107–E110. [ Google Scholar ]
• U.S. Department of Education, Institute of Education Sciences . InstituteofEducationSciences; Washington,DC: 2017. What Works Clearinghouse procedures and standards handbook, version3.0. https://ies.ed.gov/ncee/wwc/Docs/referenceresources/wwc_procedures_v3_0_standards_handbook.pdf Retrievedfrom. [ Google Scholar ]
• Veletsianos G., Shepherdson P. A systematic analysis and synthesis of the empirical MOOC literature published in 2013–2015. International Review of Research in Open and Distance Learning. 2016; 17 (2) [ Google Scholar ]
• VERBI Software . 2019. MAXQDA 2020 online manual. Retrieved from maxqda. Com/help-max20/welcome [ Google Scholar ]
• Verstegen D., Dailey-Hebert A., Fonteijn H., Clarebout G., Spruijt A. How do virtual teams collaborate in online learning tasks in a MOOC? International Review of Research in Open and Distance Learning. 2018; 19 (4) * [ Google Scholar ]
• Wang Y., Baker R. Grit and intention: Why do learners complete MOOCs? International Review of Research in Open and Distance Learning. 2018; 19 (3) * [ Google Scholar ]
• Wei C.W., Chen N.S., Kinshuk A model for social presence in online classrooms. Educational Technology Research & Development. 2012; 60 (3):529–545. * [ Google Scholar ]
• Wicks D., Craft B.B., Lee D., Lumpe A., Henrikson R., Baliram N., Wicks K. An evaluation of low versus high collaboration in online learning. Online Learning. 2015; 19 (4):n4. * [ Google Scholar ]
• Wise A.F., Perera N., Hsiao Y.T., Speer J., Marbouti F. Microanalytic case studies of individual participation patterns in an asynchronous online discussion in an undergraduate blended course. The Internet and Higher Education. 2012; 15 (2):108–117. * [ Google Scholar ]
• Wisneski J.E., Ozogul G., Bichelmeyer B.A. Does teaching presence transfer between MBA teaching environments? A comparative investigation of instructional design practices associated with teaching presence. The Internet and Higher Education. 2015; 25 :18–27. * [ Google Scholar ]
• Wladis C., Hachey A.C., Conway K. An investigation of course-level factors as predictors of online STEM course outcomes. Computers & Education. 2014; 77 :145–150. * [ Google Scholar ]
• Wladis C., Samuels J. Do online readiness surveys do what they claim? Validity, reliability, and subsequent student enrollment decisions. Computers & Education. 2016; 98 :39–56. [ Google Scholar ]
• Yamagata-Lynch L.C. Blending online asynchronous and synchronous learning. International Review of Research in Open and Distance Learning. 2014; 15 (2) * [ Google Scholar ]
• Yang J., Kinshuk, Yu H., Chen S.J., Huang R. Strategies for smooth and effective cross-cultural online collaborative learning. Journal of Educational Technology & Society. 2014; 17 (3):208–221. * [ Google Scholar ]
• Yeboah A.K., Smith P. Relationships between minority students online learning experiences and academic performance. Online Learning. 2016; 20 (4):n4. * [ Google Scholar ]
• Yu T. Examining construct validity of the student online learning readiness (SOLR) instrument using confirmatory factor analysis. Online Learning. 2018; 22 (4):277–288. * [ Google Scholar ]
• Yukselturk E., Bulut S. Gender differences in self-regulated online learning environment. Educational Technology & Society. 2009; 12 (3):12–22. [ Google Scholar ]
• Yukselturk E., Top E. Exploring the link among entry characteristics, participation behaviors and course outcomes of online learners: An examination of learner profile using cluster analysis. British Journal of Educational Technology. 2013; 44 (5):716–728. [ Google Scholar ]
• Zawacki-Richter O., Backer E., Vogt S. Review of distance education research (2000 to 2008): Analysis of research areas, methods, and authorship patterns. International Review of Research in Open and Distance Learning. 2009; 10 (6):30. doi: 10.19173/irrodl.v10i6.741. [ CrossRef ] [ Google Scholar ]
• Zhu M., Sari A., Lee M.M. A systematic review of research methods and topics of the empirical MOOC literature (2014–2016) The Internet and Higher Education. 2018; 37 :31–39. [ Google Scholar ]
• Zimmerman T.D. Exploring learner to content interaction as a success factor in online courses. International Review of Research in Open and Distance Learning. 2012; 13 (4):152–165. [ Google Scholar ]

Read our research on: Gun Policy | International Conflict | Election 2024

Regions & Countries

3. problems students are facing at public k-12 schools.

We asked teachers about how students are doing at their school. Overall, many teachers hold negative views about students’ academic performance and behavior.

• 48% say the academic performance of most students at their school is fair or poor; a third say it’s good and only 17% say it’s excellent or very good.
• 49% say students’ behavior at their school is fair or poor; 35% say it’s good and 13% rate it as excellent or very good.

Teachers in elementary, middle and high schools give similar answers when asked about students’ academic performance. But when it comes to students’ behavior, elementary and middle school teachers are more likely than high school teachers to say it’s fair or poor (51% and 54%, respectively, vs. 43%).

Teachers from high-poverty schools are more likely than those in medium- and low-poverty schools to say the academic performance and behavior of most students at their school are fair or poor.

The differences between high- and low-poverty schools are particularly striking. Most teachers from high-poverty schools say the academic performance (73%) and behavior (64%) of most students at their school are fair or poor. Much smaller shares of teachers from low-poverty schools say the same (27% for academic performance and 37% for behavior).

In turn, teachers from low-poverty schools are far more likely than those from high-poverty schools to say the academic performance and behavior of most students at their school are excellent or very good.

Lasting impact of the COVID-19 pandemic

Among those who have been teaching for at least a year, about eight-in-ten teachers say the lasting impact of the pandemic on students’ behavior, academic performance and emotional well-being has been very or somewhat negative. This includes about a third or more saying that the lasting impact has been very negative in each area.

Shares ranging from 11% to 15% of teachers say the pandemic has had no lasting impact on these aspects of students’ lives, or that the impact has been neither positive nor negative. Only about 5% say that the pandemic has had a positive lasting impact on these things.

A smaller majority of teachers (55%) say the pandemic has had a negative impact on the way parents interact with teachers, with 18% saying its lasting impact has been very negative.

These results are mostly consistent across teachers of different grade levels and school poverty levels.

Major problems at school

When we asked teachers about a range of problems that may affect students who attend their school, the following issues top the list:

• Poverty (53% say this is a major problem at their school)
• Chronic absenteeism – that is, students missing a substantial number of school days (49%)
• Anxiety and depression (48%)

One-in-five say bullying is a major problem among students at their school. Smaller shares of teachers point to drug use (14%), school fights (12%), alcohol use (4%) and gangs (3%).

Differences by school level

Similar shares of teachers across grade levels say poverty is a major problem at their school, but other problems are more common in middle or high schools:

• 61% of high school teachers say chronic absenteeism is a major problem at their school, compared with 43% of elementary school teachers and 46% of middle school teachers.
• 69% of high school teachers and 57% of middle school teachers say anxiety and depression are a major problem, compared with 29% of elementary school teachers.
• 34% of middle school teachers say bullying is a major problem, compared with 13% of elementary school teachers and 21% of high school teachers.

Not surprisingly, drug use, school fights, alcohol use and gangs are more likely to be viewed as major problems by secondary school teachers than by those teaching in elementary schools.

Differences by poverty level

Teachers’ views on problems students face at their school also vary by school poverty level.

Majorities of teachers in high- and medium-poverty schools say chronic absenteeism is a major problem where they teach (66% and 58%, respectively). A much smaller share of teachers in low-poverty schools say this (34%).

Bullying, school fights and gangs are viewed as major problems by larger shares of teachers in high-poverty schools than in medium- and low-poverty schools.

When it comes to anxiety and depression, a slightly larger share of teachers in low-poverty schools (51%) than in high-poverty schools (44%) say these are a major problem among students where they teach.  

Discipline practices

About two-thirds of teachers (66%) say that the current discipline practices at their school are very or somewhat mild – including 27% who say they’re very mild. Only 2% say the discipline practices at their school are very or somewhat harsh, while 31% say they are neither harsh nor mild.

We also asked teachers about the amount of influence different groups have when it comes to determining discipline practices at their school.

• 67% say teachers themselves don’t have enough influence. Very few (2%) say teachers have too much influence, and 29% say their influence is about right.

• 31% of teachers say school administrators don’t have enough influence, 22% say they have too much, and 45% say their influence is about right.
• On balance, teachers are more likely to say parents, their state government and the local school board have too much influence rather than not enough influence in determining discipline practices at their school. Still, substantial shares say these groups have about the right amount of influence.

Teachers from low- and medium-poverty schools (46% each) are more likely than those in high-poverty schools (36%) to say parents have too much influence over discipline practices.

In turn, teachers from high-poverty schools (34%) are more likely than those from low- and medium-poverty schools (17% and 18%, respectively) to say that parents don’t have enough influence.

Social Trends Monthly Newsletter

Sign up to to receive a monthly digest of the Center's latest research on the attitudes and behaviors of Americans in key realms of daily life

Report Materials

Table of contents, ‘back to school’ means anytime from late july to after labor day, depending on where in the u.s. you live, among many u.s. children, reading for fun has become less common, federal data shows, most european students learn english in school, for u.s. teens today, summer means more schooling and less leisure time than in the past, about one-in-six u.s. teachers work second jobs – and not just in the summer, most popular.

About Pew Research Center Pew Research Center is a nonpartisan fact tank that informs the public about the issues, attitudes and trends shaping the world. It conducts public opinion polling, demographic research, media content analysis and other empirical social science research. Pew Research Center does not take policy positions. It is a subsidiary of The Pew Charitable Trusts .