literature review analysis data

Conducting a Literature Review

• Getting Started
• Developing a Question
• Searching the Literature
• Identifying Peer-Reviewed Resources
• Managing Results

Analyzing the Literature

• Writing the Review

Need Help? Ask Your librarian!

Evidence synthesis and critical appraisal are two distinct but interrelated processes in the field of evidence-based practice and research. Here's a breakdown of the differences between them:

Critical Appraisal:

• Definition : Critical appraisal involves systematically evaluating the quality, relevance, and validity of research studies or evidence sources. It aims to assess the strengths and weaknesses of individual studies to determine their trustworthiness and applicability to a particular research question or clinical scenario.
• Focus : Critical appraisal focuses on examining the methodology, design, data analysis, and results of research studies. It involves assessing factors such as study design, sample size, bias, confounding variables, statistical methods, and generalizability.
• Purpose : The purpose of critical appraisal is to identify high-quality evidence that can inform decision-making in healthcare practice, policy, or research. It helps researchers and practitioners assess the credibility and reliability of evidence sources and make informed judgments about their use in practice.

Evidence Synthesis:

• Definition : Evidence synthesis involves systematically collecting, analyzing, and integrating evidence from multiple sources to generate new knowledge, insights, or conclusions about a particular topic or research question. It aims to aggregate and synthesize findings from individual studies to produce a comprehensive summary of the available evidence.
• Focus : Evidence synthesis encompasses a variety of methods, including systematic reviews, meta-analyses, scoping reviews, and narrative reviews. It focuses on synthesizing data, findings, and conclusions from multiple studies to provide a comprehensive overview of the evidence base on a particular topic.
• Purpose : The purpose of evidence synthesis is to provide stakeholders with a robust and comprehensive summary of the existing evidence on a particular topic or research question. It helps identify patterns, trends, inconsistencies, and gaps in the literature, informing decision-making, guiding policy development, and identifying future research priorities.

In summary, critical appraisal involves assessing the quality and validity of individual research studies, while evidence synthesis involves aggregating and synthesizing findings from multiple studies to generate new knowledge or insights about a particular topic. While they are distinct processes, they are often conducted sequentially, with critical appraisal informing the selection and inclusion of studies in evidence synthesis. Together, critical appraisal and evidence synthesis play essential roles in evidence-based practice and research, 

Synthesizing the Articles

Literature reviews synthesize large amounts of information and present it in a coherent, organized fashion. In a literature review you will be combining material from several texts to create a new text – your literature review.

You will use common points among the sources you have gathered to help you synthesize the material. This will help ensure that your literature review is organized by subtopic, not by source. This means various authors' names can appear and reappear throughout the literature review, and each paragraph will mention several different authors.

When you shift from writing summaries of the content of a source to synthesizing content from sources, there is a number things you must keep in mind:

• Look for specific connections and or links between your sources and how those relate to your thesis or question.
• When writing and organizing your literature review be aware that your readers need to understand how and why the information from the different sources overlap.
• Organize your literature review by the themes you find within your sources or themes you have identified. 

You can use a synthesis chart to help keep your sources and main ideas organized. Here are some examples:

• Virginia Commonwealth University Literature Matrix
• Johns Hopkins University Literature Review Matrix
• Writing A Literature Review and Using a Synthesis Matrix Tutorial from NC State University

California State University, Northridge. (2017). Literature Review How-To: Synthesizing Sources. Retrieved from https://libguides.csun.edu/literature-review/synthesis.

Things to Think About

Before you begin to analyze and synthesize the articles you have selected, read quickly through each article to get a sense of what they are about. One way to do this is to read the abstract and the conclusion for each article.

It is also helpful at this stage to begin sorting your articles by type of source; this will help you with the next step in the process. Many papers (but not all) fall into one of two categories:

• Primary source: a report by the original researchers of a study.
• Secondary source: a description or summary of research by somebody other than the original author(s), like a review article.

These are a selection of questions to consider while reading each article selected for your literature review. 

Primary Sources:

• Author and Year
• Purpose of Study
• Type of Study
• Data Collection Method
• Major Findings
• Recommendations
• Key thoughts/comments (eg. strengths and weaknesses)

Secondary Sources (ie. reviews)

• Author and year
• Review questions/purpose
• Key definitions
• Review boundaries
• Appraisal criteria
• Synthesis of studies
• Summary/conclusions

Cronin, P., Ryan, F., & Coughlan, M. (2008). Undertaking a literature review: A step-by-step approach. British Journal of Nursing, 17 (1), 38-43. Retrieved from: https://bit.ly/2wLeCge .

When Am I Done?

You are done with your literature review synthesis when :

• You are not finding any new ideas,
• When you encounter the same authors repeatedly, and/or
• When you feel that you have a strong understanding of the topic
• << Previous: Managing Results
• Next: Writing the Review >>
• Last Updated: Apr 16, 2024 3:40 PM
• URL: https://libguides.libraries.wsu.edu/litreview

Research Methods

• Getting Started
• Literature Review Research
• Research Design
• Research Design By Discipline
• SAGE Research Methods
• Teaching with SAGE Research Methods

Literature Review

• What is a Literature Review?
• What is NOT a Literature Review?
• Purposes of a Literature Review
• Types of Literature Reviews
• Literature Reviews vs. Systematic Reviews
• Systematic vs. Meta-Analysis

Literature Review  is a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works.

Also, we can define a literature review as the collected body of scholarly works related to a topic:

• Summarizes and analyzes previous research relevant to a topic
• Includes scholarly books and articles published in academic journals
• Can be an specific scholarly paper or a section in a research paper

The objective of a Literature Review is to find previous published scholarly works relevant to an specific topic

• Help gather ideas or information
• Keep up to date in current trends and findings
• Help develop new questions

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Helps focus your own research questions or problems
• Discovers relationships between research studies/ideas.
• Suggests unexplored ideas or populations
• Identifies major themes, concepts, and researchers on a topic.
• Tests assumptions; may help counter preconceived ideas and remove unconscious bias.
• Identifies critical gaps, points of disagreement, or potentially flawed methodology or theoretical approaches.
• Indicates potential directions for future research.

All content in this section is from Literature Review Research from Old Dominion University 

Keep in mind the following, a literature review is NOT:

Not an essay 

Not an annotated bibliography  in which you summarize each article that you have reviewed.  A literature review goes beyond basic summarizing to focus on the critical analysis of the reviewed works and their relationship to your research question.

Not a research paper   where you select resources to support one side of an issue versus another.  A lit review should explain and consider all sides of an argument in order to avoid bias, and areas of agreement and disagreement should be highlighted.

A literature review serves several purposes. For example, it

• provides thorough knowledge of previous studies; introduces seminal works.
• helps focus one’s own research topic.
• identifies a conceptual framework for one’s own research questions or problems; indicates potential directions for future research.
• suggests previously unused or underused methodologies, designs, quantitative and qualitative strategies.
• identifies gaps in previous studies; identifies flawed methodologies and/or theoretical approaches; avoids replication of mistakes.
• helps the researcher avoid repetition of earlier research.
• suggests unexplored populations.
• determines whether past studies agree or disagree; identifies controversy in the literature.
• tests assumptions; may help counter preconceived ideas and remove unconscious bias.

As Kennedy (2007) notes*, it is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the original studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally that become part of the lore of field. In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews.

Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are several approaches to how they can be done, depending upon the type of analysis underpinning your study. Listed below are definitions of types of literature reviews:

Argumentative Review      This form examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to to make summary claims of the sort found in systematic reviews.

Integrative Review      Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication.

Historical Review      Few things rest in isolation from historical precedent. Historical reviews are focused on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review      A review does not always focus on what someone said [content], but how they said it [method of analysis]. This approach provides a framework of understanding at different levels (i.e. those of theory, substantive fields, research approaches and data collection and analysis techniques), enables researchers to draw on a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection and data analysis, and helps highlight many ethical issues which we should be aware of and consider as we go through our study.

Systematic Review      This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyse data from the studies that are included in the review. Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?"

Theoretical Review      The purpose of this form is to concretely examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review help establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

* Kennedy, Mary M. "Defining a Literature."  Educational Researcher  36 (April 2007): 139-147.

All content in this section is from The Literature Review created by Dr. Robert Larabee USC

Robinson, P. and Lowe, J. (2015),  Literature reviews vs systematic reviews.  Australian and New Zealand Journal of Public Health, 39: 103-103. doi: 10.1111/1753-6405.12393

What's in the name? The difference between a Systematic Review and a Literature Review, and why it matters . By Lynn Kysh from University of Southern California

Systematic review or meta-analysis?

A  systematic review  answers a defined research question by collecting and summarizing all empirical evidence that fits pre-specified eligibility criteria.

A  meta-analysis  is the use of statistical methods to summarize the results of these studies.

Systematic reviews, just like other research articles, can be of varying quality. They are a significant piece of work (the Centre for Reviews and Dissemination at York estimates that a team will take 9-24 months), and to be useful to other researchers and practitioners they should have:

• clearly stated objectives with pre-defined eligibility criteria for studies
• explicit, reproducible methodology
• a systematic search that attempts to identify all studies
• assessment of the validity of the findings of the included studies (e.g. risk of bias)
• systematic presentation, and synthesis, of the characteristics and findings of the included studies

Not all systematic reviews contain meta-analysis. 

Meta-analysis is the use of statistical methods to summarize the results of independent studies. By combining information from all relevant studies, meta-analysis can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review.  More information on meta-analyses can be found in  Cochrane Handbook, Chapter 9 .

A meta-analysis goes beyond critique and integration and conducts secondary statistical analysis on the outcomes of similar studies.  It is a systematic review that uses quantitative methods to synthesize and summarize the results.

An advantage of a meta-analysis is the ability to be completely objective in evaluating research findings.  Not all topics, however, have sufficient research evidence to allow a meta-analysis to be conducted.  In that case, an integrative review is an appropriate strategy. 

Some of the content in this section is from Systematic reviews and meta-analyses: step by step guide created by Kate McAllister.

• << Previous: Getting Started
• Next: Research Design >>
• Last Updated: Aug 21, 2023 4:07 PM
• URL: https://guides.lib.udel.edu/researchmethods

Harvey Cushing/John Hay Whitney Medical Library

• Collections
• Research Help

YSN Doctoral Programs: Steps in Conducting a Literature Review

• Biomedical Databases
• Global (Public Health) Databases
• Soc. Sci., History, and Law Databases
• Grey Literature
• Trials Registers
• Data and Statistics
• Public Policy
• Google Tips
• Recommended Books
• Steps in Conducting a Literature Review

What is a literature review?

A literature review is an integrated analysis -- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question.  That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.

A literature review may be a stand alone work or the introduction to a larger research paper, depending on the assignment.  Rely heavily on the guidelines your instructor has given you.

Why is it important?

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Discovers relationships between research studies/ideas.
• Identifies major themes, concepts, and researchers on a topic.
• Identifies critical gaps and points of disagreement.
• Discusses further research questions that logically come out of the previous studies.

APA7 Style resources

APA Style Blog - for those harder to find answers

1. Choose a topic. Define your research question.

Your literature review should be guided by your central research question.  The literature represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.

• Make sure your research question is not too broad or too narrow.  Is it manageable?
• Begin writing down terms that are related to your question. These will be useful for searches later.
• If you have the opportunity, discuss your topic with your professor and your class mates.

2. Decide on the scope of your review

How many studies do you need to look at? How comprehensive should it be? How many years should it cover? 

• This may depend on your assignment.  How many sources does the assignment require?

3. Select the databases you will use to conduct your searches.

Make a list of the databases you will search. 

Where to find databases:

• use the tabs on this guide
• Find other databases in the Nursing Information Resources web page
• More on the Medical Library web page
• ... and more on the Yale University Library web page

4. Conduct your searches to find the evidence. Keep track of your searches.

• Use the key words in your question, as well as synonyms for those words, as terms in your search. Use the database tutorials for help.
• Save the searches in the databases. This saves time when you want to redo, or modify, the searches. It is also helpful to use as a guide is the searches are not finding any useful results.
• Review the abstracts of research studies carefully. This will save you time.
• Use the bibliographies and references of research studies you find to locate others.
• Check with your professor, or a subject expert in the field, if you are missing any key works in the field.
• Ask your librarian for help at any time.
• Use a citation manager, such as EndNote as the repository for your citations. See the EndNote tutorials for help.

Review the literature

Some questions to help you analyze the research:

• What was the research question of the study you are reviewing? What were the authors trying to discover?
• Was the research funded by a source that could influence the findings?
• What were the research methodologies? Analyze its literature review, the samples and variables used, the results, and the conclusions.
• Does the research seem to be complete? Could it have been conducted more soundly? What further questions does it raise?
• If there are conflicting studies, why do you think that is?
• How are the authors viewed in the field? Has this study been cited? If so, how has it been analyzed?

Tips: 

• Review the abstracts carefully.  
• Keep careful notes so that you may track your thought processes during the research process.
• Create a matrix of the studies for easy analysis, and synthesis, across all of the studies.
• << Previous: Recommended Books
• Last Updated: Jan 4, 2024 10:52 AM
• URL: https://guides.library.yale.edu/YSNDoctoral

• University of Texas Libraries

Literature Reviews

Analyze results.

• What is a literature review?
• Steps in the Literature Review Process
• Define your research question
• Determine inclusion and exclusion criteria
• Choose databases and search
• Review Results
• Synthesize Results
• Librarian Support

Analysis should lead to insight. This is how you will contribute to the field. 

• Analysis requires that you have an approach or a point of view to evaluate the material you found.
• Are there gaps in the literature?
• Where has significant research taken place, and who has done it?
• Is there consensus or debate on this topic?
• Which methodological approaches work best?

Analysis is the part of the literature review process where you justify why your research is needed, how others have not addressed it, and/or how your research advances the field.

Tips for Writing a Literature Review

Though this video is titled "Tips for Writing a Literature Review," the ideas expressed relate to being focused on the research topic and building a strong case, which is also part of the analysis phase.

• Last Updated: Oct 26, 2022 2:49 PM
• URL: https://guides.lib.utexas.edu/literaturereviews

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Dissertation
• What is a Literature Review? | Guide, Template, & Examples

What is a Literature Review? | Guide, Template, & Examples

Published on 22 February 2022 by Shona McCombes . Revised on 7 June 2022.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research.

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarise sources – it analyses, synthesises, and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Be assured that you'll submit flawless writing. Upload your document to correct all your mistakes.

Table of contents

Why write a literature review, examples of literature reviews, step 1: search for relevant literature, step 2: evaluate and select sources, step 3: identify themes, debates and gaps, step 4: outline your literature review’s structure, step 5: write your literature review, frequently asked questions about literature reviews, introduction.

• Quick Run-through
• Step 1 & 2

When you write a dissertation or thesis, you will have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and scholarly context
• Develop a theoretical framework and methodology for your research
• Position yourself in relation to other researchers and theorists
• Show how your dissertation addresses a gap or contributes to a debate

You might also have to write a literature review as a stand-alone assignment. In this case, the purpose is to evaluate the current state of research and demonstrate your knowledge of scholarly debates around a topic.

The content will look slightly different in each case, but the process of conducting a literature review follows the same steps. We’ve written a step-by-step guide that you can follow below.

The only proofreading tool specialized in correcting academic writing

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Correct my document today

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research objectives and questions .

If you are writing a literature review as a stand-alone assignment, you will have to choose a focus and develop a central question to direct your search. Unlike a dissertation research question, this question has to be answerable without collecting original data. You should be able to answer it based only on a review of existing publications.

Make a list of keywords

Start by creating a list of keywords related to your research topic. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list if you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can use boolean operators to help narrow down your search:

Read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

To identify the most important publications on your topic, take note of recurring citations. If the same authors, books or articles keep appearing in your reading, make sure to seek them out.

You probably won’t be able to read absolutely everything that has been written on the topic – you’ll have to evaluate which sources are most relevant to your questions.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models and methods? Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• How does the publication contribute to your understanding of the topic? What are its key insights and arguments?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible, and make sure you read any landmark studies and major theories in your field of research.

You can find out how many times an article has been cited on Google Scholar – a high citation count means the article has been influential in the field, and should certainly be included in your literature review.

The scope of your review will depend on your topic and discipline: in the sciences you usually only review recent literature, but in the humanities you might take a long historical perspective (for example, to trace how a concept has changed in meaning over time).

Remember that you can use our template to summarise and evaluate sources you’re thinking about using!

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It’s important to keep track of your sources with references to avoid plagiarism . It can be helpful to make an annotated bibliography, where you compile full reference information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

You can use our free APA Reference Generator for quick, correct, consistent citations.

Prevent plagiarism, run a free check.

To begin organising your literature review’s argument and structure, you need to understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly-visual platforms like Instagram and Snapchat – this is a gap that you could address in your own research.

There are various approaches to organising the body of a literature review. You should have a rough idea of your strategy before you start writing.

Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarising sources in order.

Try to analyse patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organise your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text, your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

If you are writing the literature review as part of your dissertation or thesis, reiterate your central problem or research question and give a brief summary of the scholarly context. You can emphasise the timeliness of the topic (“many recent studies have focused on the problem of x”) or highlight a gap in the literature (“while there has been much research on x, few researchers have taken y into consideration”).

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, make sure to follow these tips:

• Summarise and synthesise: give an overview of the main points of each source and combine them into a coherent whole.
• Analyse and interpret: don’t just paraphrase other researchers – add your own interpretations, discussing the significance of findings in relation to the literature as a whole.
• Critically evaluate: mention the strengths and weaknesses of your sources.
• Write in well-structured paragraphs: use transitions and topic sentences to draw connections, comparisons and contrasts.

In the conclusion, you should summarise the key findings you have taken from the literature and emphasise their significance.

If the literature review is part of your dissertation or thesis, reiterate how your research addresses gaps and contributes new knowledge, or discuss how you have drawn on existing theories and methods to build a framework for your research. This can lead directly into your methodology section.

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a dissertation , thesis, research paper , or proposal .

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarise yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your  dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

McCombes, S. (2022, June 07). What is a Literature Review? | Guide, Template, & Examples. Scribbr. Retrieved 13 May 2024, from https://www.scribbr.co.uk/thesis-dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, how to write a dissertation proposal | a step-by-step guide, what is a theoretical framework | a step-by-step guide, what is a research methodology | steps & tips.

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

• Open access
• Published: 17 August 2023

Data visualisation in scoping reviews and evidence maps on health topics: a cross-sectional analysis

• Emily South   ORCID: orcid.org/0000-0003-2187-4762 1 &
• Mark Rodgers 1  

Systematic Reviews volume  12 , Article number:  142 ( 2023 ) Cite this article

3636 Accesses

13 Altmetric

Metrics details

Scoping reviews and evidence maps are forms of evidence synthesis that aim to map the available literature on a topic and are well-suited to visual presentation of results. A range of data visualisation methods and interactive data visualisation tools exist that may make scoping reviews more useful to knowledge users. The aim of this study was to explore the use of data visualisation in a sample of recent scoping reviews and evidence maps on health topics, with a particular focus on interactive data visualisation.

Ovid MEDLINE ALL was searched for recent scoping reviews and evidence maps (June 2020-May 2021), and a sample of 300 papers that met basic selection criteria was taken. Data were extracted on the aim of each review and the use of data visualisation, including types of data visualisation used, variables presented and the use of interactivity. Descriptive data analysis was undertaken of the 238 reviews that aimed to map evidence.

Of the 238 scoping reviews or evidence maps in our analysis, around one-third (37.8%) included some form of data visualisation. Thirty-five different types of data visualisation were used across this sample, although most data visualisations identified were simple bar charts (standard, stacked or multi-set), pie charts or cross-tabulations (60.8%). Most data visualisations presented a single variable (64.4%) or two variables (26.1%). Almost a third of the reviews that used data visualisation did not use any colour (28.9%). Only two reviews presented interactive data visualisation, and few reported the software used to create visualisations.

Conclusions

Data visualisation is currently underused by scoping review authors. In particular, there is potential for much greater use of more innovative forms of data visualisation and interactive data visualisation. Where more innovative data visualisation is used, scoping reviews have made use of a wide range of different methods. Increased use of these more engaging visualisations may make scoping reviews more useful for a range of stakeholders.

Peer Review reports

Scoping reviews are “a type of evidence synthesis that aims to systematically identify and map the breadth of evidence available on a particular topic, field, concept, or issue” ([ 1 ], p. 950). While they include some of the same steps as a systematic review, such as systematic searches and the use of predetermined eligibility criteria, scoping reviews often address broader research questions and do not typically involve the quality appraisal of studies or synthesis of data [ 2 ]. Reasons for conducting a scoping review include the following: to map types of evidence available, to explore research design and conduct, to clarify concepts or definitions and to map characteristics or factors related to a concept [ 3 ]. Scoping reviews can also be undertaken to inform a future systematic review (e.g. to assure authors there will be adequate studies) or to identify knowledge gaps [ 3 ]. Other evidence synthesis approaches with similar aims have been described as evidence maps, mapping reviews or systematic maps [ 4 ]. While this terminology is used inconsistently, evidence maps can be used to identify evidence gaps and present them in a user-friendly (and often visual) way [ 5 ].

Scoping reviews are often targeted to an audience of healthcare professionals or policy-makers [ 6 ], suggesting that it is important to present results in a user-friendly and informative way. Until recently, there was little guidance on how to present the findings of scoping reviews. In recent literature, there has been some discussion of the importance of clearly presenting data for the intended audience of a scoping review, with creative and innovative use of visual methods if appropriate [ 7 , 8 , 9 ]. Lockwood et al. suggest that innovative visual presentation should be considered over dense sections of text or long tables in many cases [ 8 ]. Khalil et al. suggest that inspiration could be drawn from the field of data visualisation [ 7 ]. JBI guidance on scoping reviews recommends that reviewers carefully consider the best format for presenting data at the protocol development stage and provides a number of examples of possible methods [ 10 ].

Interactive resources are another option for presentation in scoping reviews [ 9 ]. Researchers without the relevant programming skills can now use several online platforms (such as Tableau [ 11 ] and Flourish [ 12 ]) to create interactive data visualisations. The benefits of using interactive visualisation in research include the ability to easily present more than two variables [ 13 ] and increased engagement of users [ 14 ]. Unlike static graphs, interactive visualisations can allow users to view hierarchical data at different levels, exploring both the “big picture” and looking in more detail ([ 15 ], p. 291). Interactive visualizations are often targeted at practitioners and decision-makers [ 13 ], and there is some evidence from qualitative research that they are valued by policy-makers [ 16 , 17 , 18 ].

Given their focus on mapping evidence, we believe that scoping reviews are particularly well-suited to visually presenting data and the use of interactive data visualisation tools. However, it is unknown how many recent scoping reviews visually map data or which types of data visualisation are used. The aim of this study was to explore the use of data visualisation methods in a large sample of recent scoping reviews and evidence maps on health topics. In particular, we were interested in the extent to which these forms of synthesis use any form of interactive data visualisation.

This study was a cross-sectional analysis of studies labelled as scoping reviews or evidence maps (or synonyms of these terms) in the title or abstract.

The search strategy was developed with help from an information specialist. Ovid MEDLINE® ALL was searched in June 2021 for studies added to the database in the previous 12 months. The search was limited to English language studies only.

The search strategy was as follows:

Ovid MEDLINE(R) ALL

(scoping review or evidence map or systematic map or mapping review or scoping study or scoping project or scoping exercise or literature mapping or evidence mapping or systematic mapping or literature scoping or evidence gap map).ab,ti.

limit 1 to english language

(202006* or 202007* or 202008* or 202009* or 202010* or 202011* or 202012* or 202101* or 202102* or 202103* or 202104* or 202105*).dt.

The search returned 3686 records. Records were de-duplicated in EndNote 20 software, leaving 3627 unique records.

A sample of these reviews was taken by screening the search results against basic selection criteria (Table 1 ). These criteria were piloted and refined after discussion between the two researchers. A single researcher (E.S.) screened the records in EPPI-Reviewer Web software using the machine-learning priority screening function. Where a second opinion was needed, decisions were checked by a second researcher (M.R.).

Our initial plan for sampling, informed by pilot searching, was to screen and data extract records in batches of 50 included reviews at a time. We planned to stop screening when a batch of 50 reviews had been extracted that included no new types of data visualisation or after screening time had reached 2 days. However, once data extraction was underway, we found the sample to be richer in terms of data visualisation than anticipated. After the inclusion of 300 reviews, we took the decision to end screening in order to ensure the study was manageable.

Data extraction

A data extraction form was developed in EPPI-Reviewer Web, piloted on 50 reviews and refined. Data were extracted by one researcher (E. S. or M. R.), with a second researcher (M. R. or E. S.) providing a second opinion when needed. The data items extracted were as follows: type of review (term used by authors), aim of review (mapping evidence vs. answering specific question vs. borderline), number of visualisations (if any), types of data visualisation used, variables/domains presented by each visualisation type, interactivity, use of colour and any software requirements.

When categorising review aims, we considered “mapping evidence” to incorporate all of the six purposes for conducting a scoping review proposed by Munn et al. [ 3 ]. Reviews were categorised as “answering a specific question” if they aimed to synthesise study findings to answer a particular question, for example on effectiveness of an intervention. We were inclusive with our definition of “mapping evidence” and included reviews with mixed aims in this category. However, some reviews were difficult to categorise (for example where aims were unclear or the stated aims did not match the actual focus of the paper) and were considered to be “borderline”. It became clear that a proportion of identified records that described themselves as “scoping” or “mapping” reviews were in fact pseudo-systematic reviews that failed to undertake key systematic review processes. Such reviews attempted to integrate the findings of included studies rather than map the evidence, and so reviews categorised as “answering a specific question” were excluded from the main analysis. Data visualisation methods for meta-analyses have been explored previously [ 19 ]. Figure  1 shows the flow of records from search results to final analysis sample.

Flow diagram of the sampling process

Data visualisation was defined as any graph or diagram that presented results data, including tables with a visual mapping element, such as cross-tabulations and heat maps. However, tables which displayed data at a study level (e.g. tables summarising key characteristics of each included study) were not included, even if they used symbols, shading or colour. Flow diagrams showing the study selection process were also excluded. Data visualisations in appendices or supplementary information were included, as well as any in publicly available dissemination products (e.g. visualisations hosted online) if mentioned in papers.

The typology used to categorise data visualisation methods was based on an existing online catalogue [ 20 ]. Specific types of data visualisation were categorised in five broad categories: graphs, diagrams, tables, maps/geographical and other. If a data visualisation appeared in our sample that did not feature in the original catalogue, we checked a second online catalogue [ 21 ] for an appropriate term, followed by wider Internet searches. These additional visualisation methods were added to the appropriate section of the typology. The final typology can be found in Additional file 1 .

We conducted descriptive data analysis in Microsoft Excel 2019 and present frequencies and percentages. Where appropriate, data are presented using graphs or other data visualisations created using Flourish. We also link to interactive versions of some of these visualisations.

Almost all of the 300 reviews in the total sample were labelled by review authors as “scoping reviews” ( n  = 293, 97.7%). There were also four “mapping reviews”, one “scoping study”, one “evidence mapping” and one that was described as a “scoping review and evidence map”. Included reviews were all published in 2020 or 2021, with the exception of one review published in 2018. Just over one-third of these reviews ( n  = 105, 35.0%) included some form of data visualisation. However, we excluded 62 reviews that did not focus on mapping evidence from the following analysis (see “ Methods ” section). Of the 238 remaining reviews (that either clearly aimed to map evidence or were judged to be “borderline”), 90 reviews (37.8%) included at least one data visualisation. The references for these reviews can be found in Additional file 2 .

Number of visualisations

Thirty-six (40.0%) of these 90 reviews included just one example of data visualisation (Fig.  2 ). Less than a third ( n  = 28, 31.1%) included three or more visualisations. The greatest number of data visualisations in one review was 17 (all bar or pie charts). In total, 222 individual data visualisations were identified across the sample of 238 reviews.

Number of data visualisations per review

Categories of data visualisation

Graphs were the most frequently used category of data visualisation in the sample. Over half of the reviews with data visualisation included at least one graph ( n  = 59, 65.6%). The least frequently used category was maps, with 15.6% ( n  = 14) of these reviews including a map.

Of the total number of 222 individual data visualisations, 102 were graphs (45.9%), 34 were tables (15.3%), 23 were diagrams (10.4%), 15 were maps (6.8%) and 48 were classified as “other” in the typology (21.6%).

Types of data visualisation

All of the types of data visualisation identified in our sample are reported in Table 2 . In total, 35 different types were used across the sample of reviews.

The most frequently used data visualisation type was a bar chart. Of 222 total data visualisations, 78 (35.1%) were a variation on a bar chart (either standard bar chart, stacked bar chart or multi-set bar chart). There were also 33 pie charts (14.9% of data visualisations) and 24 cross-tabulations (10.8% of data visualisations). In total, these five types of data visualisation accounted for 60.8% ( n  = 135) of all data visualisations. Figure  3 shows the frequency of each data visualisation category and type; an interactive online version of this treemap is also available ( https://public.flourish.studio/visualisation/9396133/ ). Figure  4 shows how users can further explore the data using the interactive treemap.

Data visualisation categories and types. An interactive version of this treemap is available online: https://public.flourish.studio/visualisation/9396133/ . Through the interactive version, users can further explore the data (see Fig.  4 ). The unit of this treemap is the individual data visualisation, so multiple data visualisations within the same scoping review are represented in this map. Created with flourish.studio ( https://flourish.studio )

Screenshots showing how users of the interactive treemap can explore the data further. Users can explore each level of the hierarchical treemap ( A Visualisation category >  B Visualisation subcategory >  C Variables presented in visualisation >  D Individual references reporting this category/subcategory/variable permutation). Created with flourish.studio ( https://flourish.studio )

Data presented

Around two-thirds of data visualisations in the sample presented a single variable ( n  = 143, 64.4%). The most frequently presented single variables were themes ( n  = 22, 9.9% of data visualisations), population ( n  = 21, 9.5%), country or region ( n  = 21, 9.5%) and year ( n  = 20, 9.0%). There were 58 visualisations (26.1%) that presented two different variables. The remaining 21 data visualisations (9.5%) presented three or more variables. Figure  5 shows the variables presented by each different type of data visualisation (an interactive version of this figure is available online).

Variables presented by each data visualisation type. Darker cells indicate a larger number of reviews. An interactive version of this heat map is available online: https://public.flourish.studio/visualisation/10632665/ . Users can hover over each cell to see the number of data visualisations for that combination of data visualisation type and variable. The unit of this heat map is the individual data visualisation, so multiple data visualisations within a single scoping review are represented in this map. Created with flourish.studio ( https://flourish.studio )

Most reviews presented at least one data visualisation in colour ( n  = 64, 71.1%). However, almost a third ( n  = 26, 28.9%) used only black and white or greyscale.

Interactivity

Only two of the reviews included data visualisations with any level of interactivity. One scoping review on music and serious mental illness [ 22 ] linked to an interactive bubble chart hosted online on Tableau. Functionality included the ability to filter the studies displayed by various attributes.

The other review was an example of evidence mapping from the environmental health field [ 23 ]. All four of the data visualisations included in the paper were available in an interactive format hosted either by the review management software or on Tableau. The interactive versions linked to the relevant references so users could directly explore the evidence base. This was the only review that provided this feature.

Software requirements

Nine reviews clearly reported the software used to create data visualisations. Three reviews used Tableau (one of them also used review management software as discussed above) [ 22 , 23 , 24 ]. Two reviews generated maps using ArcGIS [ 25 ] or ArcMap [ 26 ]. One review used Leximancer for a lexical analysis [ 27 ]. One review undertook a bibliometric analysis using VOSviewer [ 28 ], and another explored citation patterns using CitNetExplorer [ 29 ]. Other reviews used Excel [ 30 ] or R [ 26 ].

To our knowledge, this is the first systematic and in-depth exploration of the use of data visualisation techniques in scoping reviews. Our findings suggest that the majority of scoping reviews do not use any data visualisation at all, and, in particular, more innovative examples of data visualisation are rare. Around 60% of data visualisations in our sample were simple bar charts, pie charts or cross-tabulations. There appears to be very limited use of interactive online visualisation, despite the potential this has for communicating results to a range of stakeholders. While it is not always appropriate to use data visualisation (or a simple bar chart may be the most user-friendly way of presenting the data), these findings suggest that data visualisation is being underused in scoping reviews. In a large minority of reviews, visualisations were not published in colour, potentially limiting how user-friendly and attractive papers are to decision-makers and other stakeholders. Also, very few reviews clearly reported the software used to create data visualisations. However, 35 different types of data visualisation were used across the sample, highlighting the wide range of methods that are potentially available to scoping review authors.

Our results build on the limited research that has previously been undertaken in this area. Two previous publications also found limited use of graphs in scoping reviews. Results were “mapped graphically” in 29% of scoping reviews in any field in one 2014 publication [ 31 ] and 17% of healthcare scoping reviews in a 2016 article [ 6 ]. Our results suggest that the use of data visualisation has increased somewhat since these reviews were conducted. Scoping review methods have also evolved in the last 10 years; formal guidance on scoping review conduct was published in 2014 [ 32 ], and an extension of the PRISMA checklist for scoping reviews was published in 2018 [ 33 ]. It is possible that an overall increase in use of data visualisation reflects increased quality of published scoping reviews. There is also some literature supporting our findings on the wide range of data visualisation methods that are used in evidence synthesis. An investigation of methods to identify, prioritise or display health research gaps (25/139 included studies were scoping reviews; 6/139 were evidence maps) identified 14 different methods used to display gaps or priorities, with half being “more advanced” (e.g. treemaps, radial bar plots) ([ 34 ], p. 107). A review of data visualisation methods used in papers reporting meta-analyses found over 200 different ways of displaying data [ 19 ].

Only two reviews in our sample used interactive data visualisation, and one of these was an example of systematic evidence mapping from the environmental health field rather than a scoping review (in environmental health, systematic evidence mapping explicitly involves producing a searchable database [ 35 ]). A scoping review of papers on the use of interactive data visualisation in population health or health services research found a range of examples but still limited use overall [ 13 ]. For example, the authors noted the currently underdeveloped potential for using interactive visualisation in research on health inequalities. It is possible that the use of interactive data visualisation in academic papers is restricted by academic publishing requirements; for example, it is currently difficult to incorporate an interactive figure into a journal article without linking to an external host or platform. However, we believe that there is a lot of potential to add value to future scoping reviews by using interactive data visualisation software. Few reviews in our sample presented three or more variables in a single visualisation, something which can easily be achieved using interactive data visualisation tools. We have previously used EPPI-Mapper [ 36 ] to present results of a scoping review of systematic reviews on behaviour change in disadvantaged groups, with links to the maps provided in the paper [ 37 ]. These interactive maps allowed policy-makers to explore the evidence on different behaviours and disadvantaged groups and access full publications of the included studies directly from the map.

We acknowledge there are barriers to use for some of the data visualisation software available. EPPI-Mapper and some of the software used by reviews in our sample incur a cost. Some software requires a certain level of knowledge and skill in its use. However numerous online free data visualisation tools and resources exist. We have used Flourish to present data for this review, a basic version of which is currently freely available and easy to use. Previous health research has been found to have used a range of different interactive data visualisation software, much of which does not required advanced knowledge or skills to use [ 13 ].

There are likely to be other barriers to the use of data visualisation in scoping reviews. Journal guidelines and policies may present barriers for using innovative data visualisation. For example, some journals charge a fee for publication of figures in colour. As previously mentioned, there are limited options for incorporating interactive data visualisation into journal articles. Authors may also be unaware of the data visualisation methods and tools that are available. Producing data visualisations can be time-consuming, particularly if authors lack experience and skills in this. It is possible that many authors prioritise speed of publication over spending time producing innovative data visualisations, particularly in a context where there is pressure to achieve publications.

Limitations

A limitation of this study was that we did not assess how appropriate the use of data visualisation was in our sample as this would have been highly subjective. Simple descriptive or tabular presentation of results may be the most appropriate approach for some scoping review objectives [ 7 , 8 , 10 ], and the scoping review literature cautions against “over-using” different visual presentation methods [ 7 , 8 ]. It cannot be assumed that all of the reviews that did not include data visualisation should have done so. Likewise, we do not know how many reviews used methods of data visualisation that were not well suited to their data.

We initially relied on authors’ own use of the term “scoping review” (or equivalent) to sample reviews but identified a relatively large number of papers labelled as scoping reviews that did not meet the basic definition, despite the availability of guidance and reporting guidelines [ 10 , 33 ]. It has previously been noted that scoping reviews may be undertaken inappropriately because they are seen as “easier” to conduct than a systematic review ([ 3 ], p.6), and that reviews are often labelled as “scoping reviews” while not appearing to follow any established framework or guidance [ 2 ]. We therefore took the decision to remove these reviews from our main analysis. However, decisions on how to classify review aims were subjective, and we did include some reviews that were of borderline relevance.

A further limitation is that this was a sample of published reviews, rather than a comprehensive systematic scoping review as have previously been undertaken [ 6 , 31 ]. The number of scoping reviews that are published has increased rapidly, and this would now be difficult to undertake. As this was a sample, not all relevant scoping reviews or evidence maps that would have met our criteria were included. We used machine learning to screen our search results for pragmatic reasons (to reduce screening time), but we do not see any reason that our sample would not be broadly reflective of the wider literature.

Data visualisation, and in particular more innovative examples of it, is currently underused in published scoping reviews on health topics. The examples that we have found highlight the wide range of methods that scoping review authors could draw upon to present their data in an engaging way. In particular, we believe that interactive data visualisation has significant potential for mapping the available literature on a topic. Appropriate use of data visualisation may increase the usefulness, and thus uptake, of scoping reviews as a way of identifying existing evidence or research gaps by decision-makers, researchers and commissioners of research. We recommend that scoping review authors explore the extensive free resources and online tools available for data visualisation. However, we also think that it would be useful for publishers to explore allowing easier integration of interactive tools into academic publishing, given the fact that papers are now predominantly accessed online. Future research may be helpful to explore which methods are particularly useful to scoping review users.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Organisation formerly known as Joanna Briggs Institute

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

Munn Z, Pollock D, Khalil H, Alexander L, McLnerney P, Godfrey CM, Peters M, Tricco AC. What are scoping reviews? Providing a formal definition of scoping reviews as a type of evidence synthesis. JBI Evid Synth. 2022;20:950–952.

Peters MDJ, Marnie C, Colquhoun H, Garritty CM, Hempel S, Horsley T, Langlois EV, Lillie E, O’Brien KK, Tunçalp Ӧ, et al. Scoping reviews: reinforcing and advancing the methodology and application. Syst Rev. 2021;10:263.

Article   PubMed   PubMed Central   Google Scholar  

Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18:143.

Sutton A, Clowes M, Preston L, Booth A. Meeting the review family: exploring review types and associated information retrieval requirements. Health Info Libr J. 2019;36:202–22.

Article   PubMed   Google Scholar  

Miake-Lye IM, Hempel S, Shanman R, Shekelle PG. What is an evidence map? A systematic review of published evidence maps and their definitions, methods, and products. Syst Rev. 2016;5:28.

Tricco AC, Lillie E, Zarin W, O’Brien K, Colquhoun H, Kastner M, Levac D, Ng C, Sharpe JP, Wilson K, et al. A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol. 2016;16:15.

Khalil H, Peters MDJ, Tricco AC, Pollock D, Alexander L, McInerney P, Godfrey CM, Munn Z. Conducting high quality scoping reviews-challenges and solutions. J Clin Epidemiol. 2021;130:156–60.

Lockwood C, dos Santos KB, Pap R. Practical guidance for knowledge synthesis: scoping review methods. Asian Nurs Res. 2019;13:287–94.

Article   Google Scholar  

Pollock D, Peters MDJ, Khalil H, McInerney P, Alexander L, Tricco AC, Evans C, de Moraes ÉB, Godfrey CM, Pieper D, et al. Recommendations for the extraction, analysis, and presentation of results in scoping reviews. JBI Evidence Synthesis. 2022;10:11124.

Google Scholar  

Peters MDJ GC, McInerney P, Munn Z, Tricco AC, Khalil, H. Chapter 11: Scoping reviews (2020 version). In: Aromataris E MZ, editor. JBI Manual for Evidence Synthesis. JBI; 2020. Available from https://synthesismanual.jbi.global . Accessed 1 Feb 2023.

Tableau Public. https://www.tableau.com/en-gb/products/public . Accessed 24 January 2023.

flourish.studio. https://flourish.studio/ . Accessed 24 January 2023.

Chishtie J, Bielska IA, Barrera A, Marchand J-S, Imran M, Tirmizi SFA, Turcotte LA, Munce S, Shepherd J, Senthinathan A, et al. Interactive visualization applications in population health and health services research: systematic scoping review. J Med Internet Res. 2022;24: e27534.

Isett KR, Hicks DM. Providing public servants what they need: revealing the “unseen” through data visualization. Public Adm Rev. 2018;78:479–85.

Carroll LN, Au AP, Detwiler LT, Fu T-c, Painter IS, Abernethy NF. Visualization and analytics tools for infectious disease epidemiology: a systematic review. J Biomed Inform. 2014;51:287–298.

Lundkvist A, El-Khatib Z, Kalra N, Pantoja T, Leach-Kemon K, Gapp C, Kuchenmüller T. Policy-makers’ views on translating burden of disease estimates in health policies: bridging the gap through data visualization. Arch Public Health. 2021;79:17.

Zakkar M, Sedig K. Interactive visualization of public health indicators to support policymaking: an exploratory study. Online J Public Health Inform. 2017;9:e190–e190.

Park S, Bekemeier B, Flaxman AD. Understanding data use and preference of data visualization for public health professionals: a qualitative study. Public Health Nurs. 2021;38:531–41.

Kossmeier M, Tran US, Voracek M. Charting the landscape of graphical displays for meta-analysis and systematic reviews: a comprehensive review, taxonomy, and feature analysis. BMC Med Res Methodol. 2020;20:26.

Ribecca, S. The Data Visualisation Catalogue. https://datavizcatalogue.com/index.html . Accessed 23 November 2021.

Ferdio. Data Viz Project. https://datavizproject.com/ . Accessed 23 November 2021.

Golden TL, Springs S, Kimmel HJ, Gupta S, Tiedemann A, Sandu CC, Magsamen S. The use of music in the treatment and management of serious mental illness: a global scoping review of the literature. Front Psychol. 2021;12: 649840.

Keshava C, Davis JA, Stanek J, Thayer KA, Galizia A, Keshava N, Gift J, Vulimiri SV, Woodall G, Gigot C, et al. Application of systematic evidence mapping to assess the impact of new research when updating health reference values: a case example using acrolein. Environ Int. 2020;143: 105956.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Jayakumar P, Lin E, Galea V, Mathew AJ, Panda N, Vetter I, Haynes AB. Digital phenotyping and patient-generated health data for outcome measurement in surgical care: a scoping review. J Pers Med. 2020;10:282.

Qu LG, Perera M, Lawrentschuk N, Umbas R, Klotz L. Scoping review: hotspots for COVID-19 urological research: what is being published and from where? World J Urol. 2021;39:3151–60.

Article   CAS   PubMed   Google Scholar  

Rossa-Roccor V, Acheson ES, Andrade-Rivas F, Coombe M, Ogura S, Super L, Hong A. Scoping review and bibliometric analysis of the term “planetary health” in the peer-reviewed literature. Front Public Health. 2020;8:343.

Hewitt L, Dahlen HG, Hartz DL, Dadich A. Leadership and management in midwifery-led continuity of care models: a thematic and lexical analysis of a scoping review. Midwifery. 2021;98: 102986.

Xia H, Tan S, Huang S, Gan P, Zhong C, Lu M, Peng Y, Zhou X, Tang X. Scoping review and bibliometric analysis of the most influential publications in achalasia research from 1995 to 2020. Biomed Res Int. 2021;2021:8836395.

Vigliotti V, Taggart T, Walker M, Kusmastuti S, Ransome Y. Religion, faith, and spirituality influences on HIV prevention activities: a scoping review. PLoS ONE. 2020;15: e0234720.

van Heemskerken P, Broekhuizen H, Gajewski J, Brugha R, Bijlmakers L. Barriers to surgery performed by non-physician clinicians in sub-Saharan Africa-a scoping review. Hum Resour Health. 2020;18:51.

Pham MT, Rajić A, Greig JD, Sargeant JM, Papadopoulos A, McEwen SA. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods. 2014;5:371–85.

Peters MDJ, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, McInerney P, Godfrey CM, Khalil H. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth. 2020;18:2119–26.

Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, Moher D, Peters MDJ, Horsley T, Weeks L, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169:467–73.

Nyanchoka L, Tudur-Smith C, Thu VN, Iversen V, Tricco AC, Porcher R. A scoping review describes methods used to identify, prioritize and display gaps in health research. J Clin Epidemiol. 2019;109:99–110.

Wolffe TAM, Whaley P, Halsall C, Rooney AA, Walker VR. Systematic evidence maps as a novel tool to support evidence-based decision-making in chemicals policy and risk management. Environ Int. 2019;130:104871.

Digital Solution Foundry and EPPI-Centre. EPPI-Mapper, Version 2.0.1. EPPI-Centre, UCL Social Research Institute, University College London. 2020. https://eppi.ioe.ac.uk/cms/Default.aspx?tabid=3790 .

South E, Rodgers M, Wright K, Whitehead M, Sowden A. Reducing lifestyle risk behaviours in disadvantaged groups in high-income countries: a scoping review of systematic reviews. Prev Med. 2022;154: 106916.

Download references

Acknowledgements

We would like to thank Melissa Harden, Senior Information Specialist, Centre for Reviews and Dissemination, for advice on developing the search strategy.

This work received no external funding.

Author information

Authors and affiliations.

Centre for Reviews and Dissemination, University of York, York, YO10 5DD, UK

Emily South & Mark Rodgers

You can also search for this author in PubMed   Google Scholar

Contributions

Both authors conceptualised and designed the study and contributed to screening, data extraction and the interpretation of results. ES undertook the literature searches, analysed data, produced the data visualisations and drafted the manuscript. MR contributed to revising the manuscript, and both authors read and approved the final version.

Corresponding author

Correspondence to Emily South .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

Typology of data visualisation methods.

Additional file 2.

References of scoping reviews included in main dataset.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

South, E., Rodgers, M. Data visualisation in scoping reviews and evidence maps on health topics: a cross-sectional analysis. Syst Rev 12 , 142 (2023). https://doi.org/10.1186/s13643-023-02309-y

Download citation

Received : 21 February 2023

Accepted : 07 August 2023

Published : 17 August 2023

DOI : https://doi.org/10.1186/s13643-023-02309-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Scoping review
• Evidence map
• Data visualisation

Systematic Reviews

ISSN: 2046-4053

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Dimensions of teachers’ data literacy: A systematic review of literature from 1990 to 2021

• Open access
• Published: 06 May 2024

Cite this article

You have full access to this open access article

• Jihyun Lee   ORCID: orcid.org/0000-0001-5896-0686 1 ,
• Dennis Alonzo 1 ,
• Kim Beswick 1 ,
• Jan Michael Vincent Abril 1 ,
• Adrian W. Chew 1 &
• Cherry Zin Oo 2  

361 Accesses

2 Altmetric

Explore all metrics

The current study presents a systematic review of teachers’ data literacy, arising from a synthesis of 83 empirical studies published between 1990 to 2021. Our review identified 95 distinct indicators across five dimensions: (a) knowledge about data, (b) skills in using data, (c) dispositions towards data use, (d) data application for various purposes, and (e) data-related behaviors. Our findings indicate that teachers' data literacy goes beyond addressing the needs of supporting student learning and includes elements such as teacher reflection, collaboration, communication, and participation in professional development. Considering these findings, future policies should acknowledge the significance of teacher dispositions and behaviors in relation to data, recognizing that they are as important as knowledge and skills acquisition. Additionally, prioritizing the provision of system-level support to foster teacher collaboration within in-school professional development programs may prove useful in enhancing teachers’ data literacy.

Similar content being viewed by others

How Teachers Use Data: Description and Differences Across PreK Through Third Grade

Teachers’ data literacy for learning analytics: a central predictor for digital data use in upper secondary schools, teacher capacity for and beliefs about data-driven decision making: a literature review of international research.

Avoid common mistakes on your manuscript.

1 Introduction

In recent years, there has been a growing recognition of the importance of teachers’ data literacy for educational policy, research, and practice. This trend was ignited in 2009 when Arne Duncan, the former Secretary of Education of the United States, advocated evidence-driven practices in schools to enhance student performance (Mandinach & Gummer, 2016 ). Since then, there has been an increasing expectation for teachers to engage in data-informed practices to guide teaching and decision-making in schools. Following this trend, educational researchers have also increasingly directed their attention towards offering conceptual and theoretical foundations for teachers’ data literacy.

Various organizations and researchers have provided the definitions of teachers’ data literacy. For example, drawing on the opinions of diverse stakeholder groups, Data Quality Campaign ( 2014 ) defined teachers’ data literacy as teachers’ capabilities to “continuously, effectively, and ethically access, interpret, act on, and communicate multiple types of data from state, local, classroom, and other sources to improve outcomes for students in a manner appropriate to educators' professional roles and responsibilities” (p. 1). Kippers et al. ( 2018 ) defined teachers’ data literacy as “educators’ ability to set a purpose, collect, analyze, and interpret data and take instructional action” (p. 21). Similarly, teachers’ data literacy has been defined as “one’s ability, or the broad knowledge and skills, needed to engage in data use or implement a data use inquiry process (Abrams et al., 2021 , p. 100,868).

The data literacy for teachers (DLFT) framework proposed by Mandinach and Gummer defined teachers’ data literacy as “… the ability to transform information into actionable instructional knowledge and practices by collecting, analyzing, and interpreting all types of data to help determine instructional steps” (Gummer & Mandinach, 2015 , p. 2). In recent years, much of the research efforts to provide a theoretical framework on teachers’ data literacy has been led by Mandinach and Gummer (Gummer & Mandinach, 2015 ; Mandinach & Gummer, 2012 , 2013a , 2016 ; Mandinach et al., 2015 ). As far as we can ascertain, their work presents the most comprehensive framework of teachers’ data literacy in the current literature. The primary sources of Mandinach and Gummer’s DLFT framework were their previous works, Mandinach and Gummer ( 2012 ) and Mandinach et al. ( 2015 ). Their DLFT framework was developed as the results of the analysis of the teacher licensure documents across the US states (Mandinach et al., 2015 ) and the text analysis of the perspectives and definitions provided by 55 researchers and professional development providers during a braining storming at the conference held in 2012 (cf. Mandinach & Gummer, 2012 ). There are five components in the framework: (a) identifying problems and framing questions, (b) using data, (c) transforming data into information, (d) transforming information into decisions, and (e) evaluating outcomes. Their framework aimed to identify “the specific knowledge, skills, and dispositions teachers need to use data effectively and responsibly” (Mandinach & Gummer, 2016 , p. 366). However, a potential sixth dimension, “dispositions, habits of mind, or factors that influence data use” (Mandinach & Gummer, 2016 , p. 372) was mentioned but not included in the framework.

2 The present study

In the present study, we conducted a systematic review of the empirical studies on teachers’ data literacy and data use published in academic journals between 1990 and 2021. Our primary purpose was to enhance the conceptual clarity of teachers’ data literacy by providing its updated definition, indicators, and dimensions.

We argue that there are several reasons to justify the need for this systematic review. Firstly, we update, complement, and compare our review outcomes and the DLFT framework in Mandinach and Gummer ( 2016 ). A systematic review of research studies on teachers’ data use was conducted by Mandinach and Gummer ( 2013b ), but the study selection was limited to years between 2001 and 2009. Therefore, one of the aims of the present study is to compare our systematic review outcomes against the dimensions and specific indicators identified in the DLFT framework (Mandinach & Gummer, 2016 ). The present literature search spans a period from 1990 to 2021. We have set 1990 as the lower-boundary year because “during the 1990s, a new hypothesis – that the quality of teaching would provide a high-leverage policy target – began to gain currency” (Darling-Hammond et al., 2003 , p. 5).

Secondly, it appears that much work on teachers’ data literacy, including that of Mandinach and Gummer, has tended to focus on teachers’ data use in relation to teaching (e.g., Beck et al., 2020 ; Datnow et al., 2012 ) and instructional improvement (e.g., Datnow et al., 2021 ; Kerr et al., 2006 ; Wachen et al., 2018 ) or in relation to student academic performance (e.g., Poortman & Schildkamp, 2016 ; Staman et al., 2017 ). However, we argue that classroom teachers’ tasks and responsibilities go beyond teaching itself and include many other tasks such as advising/counselling, organising excursions, and administrative work (e.g., Albiladi et al., 2020 ; Kallemeyn, 2014 ). Our review, therefore, examines how teachers’ data use practices may be manifested across a range of teacher responsibilities beyond teaching and teaching-related tasks.

Thirdly, there has been a relative lack of attention to teachers’ personal dispositions in data literacy research. Dispositions refer to a person's inherent tendencies, attitudes, approaches, and inclinations towards ways of thinking, behaving, and believing (Lee & Stankov, 2018 ; Mischel & Shoda, 1995 ). According to Katz ( 1993 ), a disposition can be defined as “a tendency to exhibit frequently, consciously, and voluntarily a pattern of behavior that is directed to a broad goal” (p. 2). In the context of education, disposition refers to the attitudes, beliefs, and values that influence a teacher’s actions, decision-making, and interactions with various stakeholders including students, colleagues, and school leaders (Darling-Hammond et al., 2003 ). While teachers’ dispositions were mentioned in Mandinach and Gummer ( 2016 ), dispositions were not included in their DLFT framework. Teacher educators have long emphasized that accomplished teachers need to possess extensive knowledge, skills, and a range of dispositions to support the learning of all students in the classroom, engage in on-going professional development, and continuously strive to enhance their own learning throughout their careers (Darling-Hammond et al., 2003 ; Sykes, 1999 ). Therefore, we aim to identify a range of teachers’ dispositions in relation to data literacy and data use in the school contexts.

Fourthly, we argue that teachers’ data literacy may be more important in the current context of the rapidly evolving data and digital landscape influenced by the technical advancements in artificial intelligence. Teachers may encounter significant challenges in comprehending and addressing a wide array of issues, both anticipated and unforeseen, as well as observed and unobserved situations, stemming from various artificial intelligence tools and automated machines. In this sense, comprehending the nature, types, and functions of data is crucial for teachers. Without such understanding, the educational community and teaching workforce may soon find themselves in an increasingly worrisome situation when it comes to evaluating data and information.

Finally, we argue that there is a need to update conceptual clarity regarding teachers’ data literacy in the current literature. Several systematic review studies have focused on features in professional development interventions (PDIs) aimed at improving teachers’ data use in schools (e.g., Ansyari et al., 2020 ; 2022 ; Espin et al., 2021 ), emphasizing the need to understand data literacy as a continuum spanning from pre-service to in-service teachers and from novice to veteran educators (Beck & Nunnaley, 2021 ). Other systematic review studies have given substantial attention to data-based decision-making (DBDM) in the schools (e.g., Espin et al., 2021 ; Filderman et al., 2018 ; Gesel et al., 2021 ; Hoogland et al., 2016 ). For example, Hoogland et al. ( 2016 ) investigated the prerequisites for data-based decision-making (DBDM) in the classroom, highlighting nine themes that influence DBDM, such as collaboration, leadership, culture, time, and resources. These systematic reviews are highly relevant to the current review, as the PDIs, understanding the continuum, or data-based decision-making would require a clear and updated understanding of what teachers’ data literacy should be. We hope that the current study’s definition, indicators, and dimensions of teachers’ data literacy may be useful in conjunction with other systematic review studies on teachers’ data use and factors influencing teachers’ data use.

3.1 Data sources and selection of the studies

Our strategies for literature search were based on the guidelines of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), a framework for reporting and synthesising literature review (Moher et al., 2009 ). In accordance with PRISMA suggestions, we followed the four steps in locating and reviewing the relevant studies. First, we conducted initial searches to identify relevant studies, using three databases: Scopus, ProQuest, and Web of Science. Keywords in our search were teacher, school, data, data use, data literacy, evidence-based, and decision-making (see Table  1 for the detailed search strategy syntax). This initial search, using the combination of the identified keywords, yielded 2,414 journal articles (see Fig.  1 ). After removing duplicates, 1,976 articles remained.

Study selection flow using PRISMA guidelines

Secondly, we set and applied the inclusion criteria to screen the studies. The inclusion criteria were: (a) topics relating to the key words, (b) school context of primary or secondary school settings (i.e., excluding studies focusing on university, vocational education, and adult learning), (c) the full text written in English (excluding studies if the full text is presented in another language or if only the abstract was presented in English), (d) peer-reviewed empirical studies (across quantitative, qualitative, and mixed-methods) published in academic journals (excluding book chapters, conference papers, thesis) to ensure the inclusion of the published work that has undergone peer-review process, and finally, (e) published studies from 1990 onwards. The titles and abstracts of the studies were reviewed to assess their eligibility based on the inclusion criteria. As a result of applying these criteria, 117 articles were selected for the next step, full-text review.

Thirdly, we evaluated the eligibility of the full-text versions of the published studies. This full-text review resulted in a further exclusion of 34 studies as they were found to not meet all the inclusion criteria. We also examined whether the studies included data literacy or data-driven decision-making. Following these assessments, we identified 83 articles that met all the inclusion criteria.

Finally, we reviewed, coded, and analyzed the final set of the selected studies. The analysis approaches are described below.

3.2 Approach to analysis

We employed a thematic synthesis methodology, following the framework outlined by Thomas & Harden ( 2008 ). The coding and analysis process consisted of three main stages: (a) conducting a line-by-line reading and coding of the text, (b) identifying specific descriptive codes, and (c) generating analytical themes by grouping conceptually inter-related descriptive codes. The final analytic process was, therefore, categorizing and naming related descriptive codes to produce analytical themes. During the development of the analytic themes, we utilized an inductive approach, organizing conceptually interconnected codes into broader themes.

The first author developed the descriptive and analytical themes, which were then reviewed by another two authors. To ensure coding rigor and consistency, three authors independently coded the same two articles, and then compared the coding to address any inconsistencies and reach a consensus. This process was repeated in four iterations. Once the three authors who were involved in the initial coding reached the consensus, the remaining authors double-checked the final outputs of the thematic analysis (i.e., codes, and themes). We have labelled descriptive codes as ‘indicators’ of teachers' data literacy, while the broader groups of descriptive codes, referred to as analytic themes, represent ‘dimensions’ of teachers’ data literacy.

4.1 Characteristics of the reviewed studies

The main purpose of the present study was to examine the conceptualization of teachers’ data literacy from 83 peer-reviewed empirical studies. Table 2 presents the studies included in our systematic review, along with the summary of the study characteristics such as country, school-level, study focus (i.e., main constructs), study purposes/objectives, research method, data collection tools, and sample size. Figure  2 presents the number of the reviewed studies by publication year. We found that since 2015, there has been an increase in the number of published empirical studies on teachers' data literacy.

Number of the reviewed studies by publication year

Out of 83 studies, 50 were conducted in the United States. Thirteen studies were from Netherlands, four from Belgium, three from Australia, two from each of Canada and the United Kingdom, and one study for each of the following ten countries: China, Denmark, Germany, Indonesia, Ireland, Kenya, Korea, Norway, South Africa, and Sweden. Therefore, more than half of the studies (i.e., 58 studies, 70%) were conducted in the English-speaking countries. In terms of school-settings, studies were mostly conducted in primary school settings or in combination with high school: 36 studies in primary school settings, 16 in secondary school settings, and 30 studies were in both primary and secondary school settings. The most common design was qualitative ( n  = 35 studies), followed by mixed methods ( n  = 30) and quantitative ( n  = 18). Multiple sources of data collection (e.g., interview and survey) were used in 22 studies. The most commonly used data collection tool was interview ( n  = 55), which was followed by surveys ( n  = 37) and observation ( n  = 25). A smaller set of studies used focus group discussion ( n  = 18) and document analysis ( n  = 19). A few studies used students’ standardised assessment data ( n  = 4), field notes ( n  = 4), and teacher performance on data literacy test ( n  = 4).

We also reviewed the study topics and found that there are seven foci among the reviewed studies: (a) factors influencing teachers’ data use ( n  = 29), (b) specific practices in teachers’ data use ( n  = 27), (c) teachers’ data use to enhance teaching practices ( n  = 25), (d) teachers’ data use for various purposes ( n  = 24), (e) approaches to improve teachers’ data literacy ( n  = 22), (f) approaches to improve teachers’ assessment literacy ( n  = 19), and (g) teachers’ data use to improve student learning outcomes ( n  = 19).

4.2 Dimensions and indicators of teaches’ data literacy

Our thematic analysis identified 95 descriptive codes (see Table  3 ). Careful review of the identified descriptive codes suggested that they can be viewed as indicators of teachers’ knowledge, attitudes, behaviors, and dispositions in data use. These indicators were further organized into inter-related concepts, which formed analytic themes; we refer to these as ‘dimensions’ (see Table  3 ). There were five broad dimensions that emerged from the indicators: knowledge about data (Dimension 1), skills in using data (Dimension 2), dispositions towards data use (Dimension 3), data application for various purposes (Dimension 4), and data-related behaviors (Dimension 5).

It is necessary to point out that Dimension 1 pertains to understanding the nature of data itself, focusing on knowledge about data. On the other hand, Dimension 2 revolves around data-related skills in the actual use of data , encompassing a spectrum of sequences including data generation, processing, and production. These two dimensions, i.e., knowledge and skills, are highly interconnected and complement each other. Proficiency in data-use skills (Dimension 2) may not be developed without a solid understanding of how data can be utilised, for instance, in teaching practices or school improvement in data use (Dimension 1). Conversely, teachers' understanding of how data can enhance teaching practices (Dimension 1) can guide them in determining specific approaches to analysing particular datasets (Dimension 2). While we acknowledge the complementary nature of knowledge and skills, it is important to note that certain aspects of knowledge and skills may not completely overlap. For instance, a teacher who understands the process of creating state-level assessment data may not necessarily possess the technical expertise required to analyze state-level data, taking into account measurement errors. Therefore, we maintain knowledge and skills as two distinct dimensions to highlight both as the core components of teachers’ data literacy.

Within each of the five broad dimensions, we also uncovered sub-themes to illuminate the constituent elements of those dimensions. Under Dimension 1, four sub-themes emerged: “knowledge about data”, knowledge about data for “teaching practices”, understanding “data culture in the school”, and understanding the use of “external assessment”. Dimension 2 featured sub-themes highlighting the sequential stages of data utilization: “data generation & collection”, “data analysis”, “data interpretation”, “data integration”, “evaluation”, and “reporting”. Within Dimension 3, we identified dispositions towards data use, encompassing sub-themes such as confidence, values/beliefs, trust/respect, and anxiety. Dimension 4 revealed various purposes of data applications, categorized into three sub-themes: “teaching,” “student learning,” and “school improvement.” Lastly, Dimension 5 delineated teachers’ behaviors related to data into two sub-themes: “communication & discussion” and “participation & engagement.”

In the following passages we provide detailed descriptions of the indicators and their associated dimensions. Figure  3 presents a visual a summary of these indicators and dimensions.

A summary of the dimensions and indicators of teachers’ data literacy

4.2.1 Dimension 1. Knowledge about data

The first dimension of teachers’ data literacy pertains to teachers’ knowledge about data . Many studies recognized the importance of data-related knowledge to be utilized in the schools (e.g., Jacobs et al., 2009 ; Omoso et al., 2019 ; Schildkamp et al., 2017 ). Our review revealed four major ways that teachers' data-related knowledge can be manifested. Firstly, teachers’ knowledge about data involves their understanding of the necessary steps in data analysis procedures (Ebbeler et al., 2016 ; Snodgrass Rangel et al., 2016 ; Vanlommel et al., 2021 ; Wardrip & Herman, 2018 ) and understanding of different data types to be used for varying purposes (Abdusyakur & Poortman, 2019 ; Beck et al., 2020 ; Howley et al., 2013 ; Reeves et al., 2016 ).

Secondly, teachers’ knowledge about data involves their capability to relate the insights gleaned from data to inform their teaching practices (Abrams et al., 2016 ; Jimerson et al., 2016 ). Specifically, data-literate teachers leverage student assessment data to evaluate learning progress (Abrams et al., 2016 ; Jimerson, 2014 ; Jimerson & Wayman, 2015 ; Jimerson et al., 2016 ; Snodgrass Rangel et al., 2016 ), to tailor classroom instruction based on data insights (Mokhtari et al., 2009 ; Poortman & Schildkamp, 2016 ; Staman et al., 2017 ; van der Scheer & Visscher, 2018 ), and to ensure alignment between instructional approaches and appropriate assessment methods (Howley et al., 2013 ; Marsh & Farrell, 2015 ; van der Scheer & Visscher, 2018 ).

Thirdly, teachers’ data literacy extends to understanding of the school culture surrounding data utilization (e.g., Andersen, 2020 ; Schildkamp, 2019 ; Wachen et al., 2018 ). This encompasses recognizing the conditions that may facilitate or hinder teachers’ data use (Abdusyakur & Poortman, 2019 ; Anderson et al., 2010 ; Keuning et al., 2017 ) and navigating various challenges associated with using assessment data in the school (Datnow et al., 2012 ; Ford, 2018 ; Kanjee & Moloi, 2014 ; Thomas & Huffman, 2011 ).

Lastly, teachers’ knowledge about data includes understanding of externally administered assessment data and data system, such as state-level assessment policies related to data use (Copp, 2017 ; Hardy, 2019 ; Reed, 2015 ) and understanding the broader state-level contexts that impact data utilization within the school (Datnow et al., 2013 ; Dunn et al., 2013a ; Ford, 2018 ; Omoso et al., 2019 ; Powell et al., 2021 ). Teachers may need to have thorough knowledge of educational government policies to ensure alignment between state-level curriculum initiatives and school-level assessment policies (Anderson et al., 2010 ; Copp, 2017 ; Gelderblom et al., 2016 ; Hardy, 2015 ).

In summary, existing literature highlights that data-literate teachers would have a comprehensive understanding of a diverse range of data sources and purposes, regularly reviewing and evaluating student outcomes from various channels. Consequently, if teachers face excessive pressure to meet accountability measures and improve standardized testing results, it could potentially hinder their overall development and growth in a broad spectrum of data-related knowledge.

4.2.2 Dimension 2. Skills in using data

Skills in using data is the second key dimension in teachers’ data literacy. There were a wide range of specific data-skills mentioned in the literature, spanning from data generation and collection (Farley-Ripple et al., 2019 ; Jimerson & Wayman, 2015 ) to data analysis (Farley-Ripple et al., 2019 ; Jimerson & Wayman, 2015 ; Marsh et al., 2010 ), data interpretation and integration (Jimerson & Wayman, 2015 ; Marsh et al., 2010 ), evaluation (Andersen, 2020 ; Dunn et al., 2013b ; Thomas & Huffman, 2011 ), and report writing (Farley-Ripple et al., 2019 ; Jimerson & Wayman, 2015 ). These indicators (see Table  3 ) emphasize that teachers’ data literacy requires proficiency across the entire sequence, across different stages of data generation, processing, and production.

Teachers’ skills in data use also involve selecting specific data types appropriate for different purposes (Anderson et al., 2010 ; Jimerson et al., 2016 ; Kanjee & Moloi, 2014 ), analysing multiple sources of data on student learning outcomes (Datnow et al., 2012 ; Vanlommel et al., 2021 ; von der Embse et al., 2021 ), and integrating multiple data sources to arrive at a holistic assessment of student progress (Brunner et al., 2005 ; Farley-Ripple et al., 2019 ; Ford, 2018 ; Jacobs et al., 2009 ; Mausethagen et al., 2018 ). For example, teachers may need to apply different data analytic approaches when evaluating student outcomes based on school-based versus externally administered standardized assessments (Copp, 2017 ; Curry et al., 2016 ; Omoso et al., 2019 ; Wardrip & Herman, 2018 ; Zeuch et al., 2017 ). Data-literate teachers may also plan data analysis for targeted purposes, such as analyzing students’ social-emotional outcomes (Abrams et al., 2021 ; Jimerson et al., 2021 ; von der Embse et al., 2021 ; Wardrip & Herman, 2018 ), identifying individual students’ learning needs, making recommendations for curriculum revisions, or evaluating pedagogical approaches (Dunn et al., 2013a ; Snodgrass Rangel et al., 2016 ; Wolff et al., 2019 ; Young, 2006 ).

In summary, this “skills” dimension highlights the importance of teachers possessing a diverse array of competencies to leverage data effectively. The literature reviewed identified various aspects of teachers’ data use, spanning the spectrum from data collection and generation to analysis, interpretation, integration across multiple sources, evaluation, and reporting.

4.2.3 Dimension 3. Dispositions towards data use

While somewhat overlooked in data literacy literature, teachers’ disposition is a crucial component of their data literacy. Our review identified four major types of such dispositions in the context of teachers’ data literacy (see Table  3 ). Firstly, studies have underscored that teachers’ confidence in using data may be necessary when making data-driven school-level decisions, for example, to design intervention programs (Andersen, 2020 ; Keuning et al., 2017 ; Staman et al., 2017 ; Thompson, 2012 ), or to develop strategic plans for school improvement (Dunn et al., 2013b ; Poortman & Schildkamp, 2016 ). Researchers also claimed that teachers may need to feel confident in many steps of data processes, across accessing, analyzing, interpreting, evaluating, and discussing data within the school environment (Abrams et al., 2021 ; Dunn et al., 2013a ; von der Embse et al., 2021 ).

The second disposition pertains to teachers valuing and believing in the importance of data use in schools. Data-literate teachers would recognize the usefulness of data in informing school improvement and enhancing student performance (Howley et al., 2013 ; Poortman & Schildkamp, 2016 ; Prenger & Schildkamp, 2018 ). They would also place value on collaboration among colleagues and actively seek institutional support for effective data use (Kallemeyn, 2014 ; Marsh & Farrell, 2015 ; Nicholson et al., 2017 ; Poortman & Schildkamp, 2016 ). Furthermore, they would appreciate the pivotal role of school leaders in supporting and promoting teachers’ data use within the school (Albiladi et al., 2020 ; Curry et al., 2016 ; Joo, 2020 ; Young, 2006 ).

A third type of teacher disposition that our review identified is trust in and respect towards colleagues and school leaders . Teachers often work collaboratively in the school environment when they learn about and utilise school-level data. In this sense, teacher collaboration and sustaining trusting relationships are fundamental in fostering a school culture that appreciates data-driven decision-making, as well as for encouraging teachers to further develop their own data knowledge and skills (Abrams et al., 2021 ; Andersen, 2020 ; Keuning et al., 2017 ). Mutual trust and respect among teachers can allow them to have open and honest conversations about their experiences and share any concerns arising from data use in the school context (Andersen, 2020 ; Datnow et al., 2013 ; Ford, 2018 ; Wachen et al., 2018 ).

Lastly, data anxiety may play a role when teachers use or are expected to use data in the school (Abrams et al., 2021 ; Dunn et al., 2013b ; Reeves et al., 2016 ). Teachers may experience data anxiety when they are expected to effectively analyze student assessment outcomes (Dunn et al., 2013b ; Powell et al., 2021 ), when they are introduced to new data management systems in the school, when they feel pressured to quickly grasp the school’s data management system (Andersen, 2020 ; Dunn et al., 2013a ), or when they are tasked with developing specific strategies to assess and enhance student learning outcomes (Dunn et al., 2013a , b ; Jimerson et al., 2019 ). These types of teacher responsibilities demand proficient data skills and knowledge, which not all teachers may possess, and thus, anxiety may hinder their ability to further develop their data literacy.

In summary, teacher dispositions towards data use can impact their effective utilization of data or impede the capacity to further develop their own data literacy. Our review also illuminated that it is not just individual teachers’ confidence or anxiety towards data use, but also the social dynamics within the school environment, including teacher collaboration, trust and respect, and relationships with the school management team, that can influence teachers’ data literacy. Therefore, fostering a collaborative climate within the school community and creating more opportunities for data use may strengthen a data-driven culture within the school.

4.2.4 Dimension 4. Data applications for various purposes

Our review suggests that teachers' data literacy can be manifested in their use of data for multiple purposes, primarily in three areas: (a) to enhance teaching practices (e.g., Datnow et al., 2012 , 2021 ; Farrell, 2015 ; Gelderblom et al., 2016 ; Wachen et al., 2018 ), (b) to support student learning (e.g., Joo, 2020 ; Lockton et al., 2020 ; Staman et al., 2017 ; Vanlommel et al., 2021 ; van der Scheer & Visscher,  2018 ), and (c) to make plans and strategies for school improvement (e.g., Abdusyakur & Poortman, 2019 ; Jimerson et al., 2021 ; Kallemeyn, 2014 ).

With respect to teaching enhancement purposes, teachers use data to inform their lesson plans (Ford, 2018 ; Gelderblom et al., 2016 ; Snodgrass Rangel et al., 2016 ; Reeves et al., 2016 ), set lesson objectives (Kallemeyn, 2014 ; Snodgrass Rangel et al., 2016 ; Reeves et al., 2016 ), develop differentiated instructions (Beck et al., 2020 ; Datnow et al., 2012 ; Farley-Ripple et al., 2019 ), and provide feedback to students (Gelderblom et al., 2016 ; Andersen, 2020 ; Jimerson et al., 2019 ; Marsh & Farrell, 2015 ). Furthermore, teachers use data to reflect on their own teaching practices (Datnow et al., 2021 ; Ford, 2018 ; Jimerson et al., 2019 ; Snodgrass Rangel et al., 2016 ) and evaluate the impact of using data on teaching and learning outcomes (Gelderblom et al., 2016 ; Marsh & Farrell, 2015 ).

In relation to supporting student learning, teachers use data to recognize individual students’ learning needs (Curry et al., 2016 ; Gelderblom et al., 2016 ), guide students to learning new or challenging concepts (Abrams et al., 2021 ; Keuning et al., 2017 ; Marsh et al., 2010 ; Reeves et al., 2016 ), set learning goals (Abdusyakur & Poortman, 2019 ; Curry et al., 2016 ), and monitor learning progress (Curry et al., 2016 ; Gelderblom et al., 2016 ; Marsh et al., 2010 ).

In terms of guiding school improvement strategies, teachers use data to develop school-based intervention programs (Abdusyakur & Poortman, 2019 ; Jimerson et al., 2021 ; Kallemeyn, 2014 ; Thompson, 2012 ), make decisions about school directions (Huffman & Kalnin, 2003 ; Prenger & Schildkamp, 2018 ; Schildkamp, 2019 ), and evaluate school performance for meeting the accountability requirements (Hardy, 2015 ; Jacobs et al., 2009 ; Jimerson & Wayman, 2015 ; Marsh et al., 2010 ; Omoso et al., 2019 ; Snodgrass Rangel et al., 2019 ).

In summary, the literature indicates that data-literate teachers use data for multiple purposes and consider it essential in fulfilling their various roles and responsibilities within the school. Teachers’ data use for supporting student learning tends to focus primarily on helping students achieve better learning outcomes; in contrast, teachers’ data use for teaching enhancement includes a broader range of data processes and practices.

4.2.5 Dimension 5. Data-related behavior

The fifth and final dimension we identified pertains to teachers' data-related behaviors within and outside the school context. Within this dimension, there appear to be two distinctive sets of teacher behaviors: (a) teachers’ data use to enhance communication and discussion with various stakeholders such as colleagues (Datnow et al., 2013 ; Van Gasse et al., 2017 ), school leaders (Jimerson, 2014 ; Marsh & Farrell, 2015 ; Nicholson et al., 2017 ), and parents (Jimerson & Wayman, 2015 ; Jimerson et al., 2019 ); and (b) teachers’ participation in and engagement with learning about data use (Schildkamp et al., 2019 ; Wardrip & Herman, 2018 ) and data culture in schools (Datnow et al., 2021 ; Keuning et al., 2016 ). These behaviors were found to be integral aspects of teachers' data literacy. Teacher engagement with data is manifested in multiple ways, such as involvement in team-based approaches to data utilization (Michaud, 2016 ; Schildkamp et al., 2017 ; Wardrip & Herman, 2018 ; Young, 2006 ), active participation in creating a school culture of data use (Abrams et al., 2021 ; Albiladi et al., 2020 ), evaluation of the organizational culture and conditions pertaining to data use (Andersen, 2020 ; Datnow et al., 2021 ; Lockton et al., 2020 ), and participation in professional development opportunities focused on data literacy (Ebbeler et al., 2016 ; O’Brien et al., 2022 ; Schildkamp et al., 2017 ).

In summary, this dimension highlights that teachers’ data literacy includes various forms of their active engagement and behavior to enhance the effective use and understanding of data. Our findings also indicate that teacher communication and discussions regarding data primarily focus on student assessment data with various stakeholder groups including colleagues, school leaders, and parents.

5 Discussion

The present study reviews 83 empirical studies on teachers' data literacy published in peer-reviewed journals from 1990 to 2021, and we identified 95 specific indicators categorized across five dimensions: (a) knowledge about data , (b) skills in using data , (c) dispositions towards data use , (d) data applications for various purposes , and (e) data-related behaviors in the school . Our review of the identified indicators of this study has led to the following definition of teachers’ data literacy:

A set of knowledge, skills, and dispositions that empower teachers to utilize data for various purposes, including generating, collecting, analyzing, interpreting, integrating, evaluating, reporting, and communicating, aimed at enhancing teaching, supporting student learning, engaging in school improvement, and fostering self-reflection. Teachers’ data literacy also involves the appreciation for working together with colleagues and school leaders to (a) assess organizational conditions for data use, (b) foster a supportive school culture, and (c) engage in ongoing learning to optimize the effective utilization of data.

Our analysis also revealed several noteworthy findings that are presented in the following sections.

5.1 Teachers’ data literacy and assessment literacy

There have been concerns expressed by scholars about conceptual fuzziness in teachers’ data literacy and assessment literacy (cf. Brookhart, 2011 ; Ebbeler et al., 2016 ; Mandinach, 2014 ; Mandinach & Gummer, 2016 ). Indeed, student assessment data are the most salient form of data in the school (Mandinach & Schildkamp, 2021 ). The research trend of recognising the importance of teachers’ data literacy is often based on the premise that teachers’ data literacy would enhance teaching and ultimately improve student outcomes (cf. Ebbeler et al., 2016 ; Mandinach & Gummer, 2016 ; Poortman & Schildkamp, 2016 ; Thompson, 2012 ; Van Gasse et al., 2018 ; Zhao et al., 2016 ). Furthermore, the systemic pressure on schools to meet accountability requirements has also impacted their endeavors to utilize, assess, and demonstrate school performance based on student assessment data in recent years (Abdusyakur & Poortman, 2019 ; Farrell, 2015 ; Schildkamp et al., 2017 ; Weiss, 2012 ). In these contexts, it is not surprising that educational practitioners would think about student assessment data when they are expected to improve their data skills.

In this light, we have tallied the teacher data literacy indicators that directly relate to student assessment or about students’ learning outcomes . In Table  3 , the symbol “⁑” is used for the indicators related to student assessment, and “ξ” is used for the indicators related to students’ learning outcomes. We found that there were only 19 out of 95 indicators that directly related to student assessment (e.g., knowledge about different purposes of assessment, understanding the alignment between instruction and assessment, understanding state-level assessment policies on data use). Similarly, there were only 13 out of 95 indicators that directly related to students’ learning outcomes (e.g., identifying evidence of student learning outcomes, understanding student learning outcomes using multiple sources).

Our review demonstrates that teachers regularly interact with a diverse array of data and undertake various tasks closely associated with its utilization. Therefore, teachers' data literacy encompasses more than just its use in student assessment and learning outcomes; it extends to understanding students’ social-emotional learning and higher-order thinking skills, assessing school conditions for data use, reflecting on teaching practices, and communicating with colleagues. Consequently, limiting the perspective of teachers’ data literacy solely to assessment literacy may impede their full utilization and appreciation of data applications essential to their multifaceted work in supporting and enhancing student and school outcomes.

5.2 Teachers’ data literacy and data-related dispositions

We found that one of the key aspects of teachers’ data literacy is teachers’ dispositions towards data use. As noted by Mandinach and Gummer ( 2012 , 2016 ), this aspect of teacher characteristics has not received as much research attention as data knowledge or data skills. It is perhaps due to ‘literacy’ being traditionally linked to knowledge and skills (Shavelson et al., 2005 ; also see Mandinach & Gummer, 2012 ) or due to the research trend of unpacking teachers’ needs and pedagogical approaches in specific subject/learning domains (Sykes,  1999 ; see Mandinach & Gummer, 2016 ). However, our review suggests that teacher dispositions towards data use are required in virtually all aspects of data use and data analyses processes. We also found that the most important data-related teacher disposition was confidence . The data literacy literature recognized the importance of teacher confidence, with respect to accessing, collecting, analysing, integrating, evaluating, discussing, and making decisions, suggesting that for teachers to be data literate, confidence may be required in every step of data use. There has been extensive research that has demonstrated a strong link between confidence and learning motivation, indicating that individuals tend to gravitate towards domains in which they feel comfortable and confident (e.g., Lee & Durksen, 2018 ; Lee & Stankov, 2018 ; Stankov & Lee, 2008 ). Our review findings contribute to this existing body of research, emphasizing the importance of confidence in teachers’ data utilization. This underscores the necessity for policies and professional development initiatives aimed at enhancing teachers’ data use to also prioritize strategies for building teachers’ confidence in this area.

Our findings also indicate that teachers’ data literacy is associated with their trust in colleagues and school leaders, as well as their respect for the leadership team's role in leading data use and school improvement (Andersen, 2020 ; Ford, 2018 ; Wachen et al., 2018 ). This suggests that for teachers to be effective data users, they need to feel empowered to voice concerns and express frustrations with colleagues (Andersen, 2020 ; Ford, 2018 ; Wachen et al., 2018 ), seek help when necessary (Wardrip & Herman, 2018 ; Young, 2006 ), and collaboratively develop strategies for effective collaboration within the school (Datnow et al., 2013 ; Huffman & Kalnin, 2003 ; Michaud, 2016 ; Van Gasse et al., 2021 ).

Many teacher tasks are deeply intertwined with human relationships (Lee, 2021 ) and often completed through collaborative efforts (Li et al., 2022 ). Therefore, school leaders and policymakers may recognize that fostering teachers’ data literacy may necessitate cultivating open, honest, and trusting school environments conducive to collaboration. Notably, the social aspect of data literacy was not prominently evident in dimensions related to teachers' knowledge and skills, which suggests that teachers may enhance their knowledge and skills independently from others in the school environment. However, fostering teacher dispositions, such as active engagement in effective data use within the school, appears to be influenced by collaborative relationships with colleagues, as well as the supportive roles of school leaders.

5.3 Teachers’ data literacy and data-related behaviors

Our review showed that teachers’ data literacy goes beyond the knowledge, skills, and dispositions that are required to effectively use data; it also involves a range of behaviors that enhance their ways of using and learning about data. Within this dimension, we noted two sub-categories, communication/discussion and participation/engagement. Therefore, one core aspect of teacher behaviors related to data was found to be communicating with various stakeholders such as colleagues, parents, and school leaders to discuss instructional approaches (e.g., Datnow et al., 2013 ; Militello et al., 2013 ; van der Scheer & Visscher, 2018 ) and assessment results (e.g., Curry et al., 2016 ; Howley et al., 2013 ). The other aspect—participation and engagement—underscores the importance of teacher involvement in team-based learning regarding data use (e.g., Andersen, 2020 ; Young, 2006 ), active engagement in establishing conducive school conditions and fostering a culture of data use within the school community (e.g., Datnow et al., 2021 ; Keuning et al., 2016 ), and proactive participation in professional development to enhance knowledge and skills (e.g., Ebbeler et al., 2016 ; van der Scheer & Visscher, 2018 ). Existing studies on data literacy have not given substantial attention to the importance of teachers' behaviors related to data. However, we argue that teachers’ behaviors related to data deserve recognition as a distinct category within the concept of teachers’ data literacy.

Dimension 4 (about teachers’ disposition) and Dimension 5 (about teachers’ behaviors) would be correlated. For example, teachers who are confident in data use may be more inclined to lead the discussions with other colleagues about data use in the school, and they may pursue additional learning opportunities to become an effective leader in school data use. Trust and respect within the school communities mentioned above would also influence how teachers behave in order to collectively enhance data literacy within the school. Studies (e.g., Ebbeler et al., 2016 ; van der Scheer & Visscher, 2018 ) have highlighted teacher participation in professional development, but there has been a relative lack of research attention to examine the collaborative nature of teacher engagement and learning within the professional settings. With the rapid evolution of educational tools and applications driven by learning analytics and artificial intelligence, the influx of data generated in this new era poses a significant challenge for teachers and school leaders. Accordingly, teacher collaboration in learning and addressing data-related challenges in schools will increasingly become a paramount concern, more so than ever before. In this regard, future policies concerning data use may prioritize the expansion of teacher collaboration and mutual learning as essential components of in-school professional development activities.

5.4 Reflections on Mandinach and Gummer’s ( 2016 ) DLFT framework

We have compared the indicators and dimensions arising from the present study and those in Mandinach and Gummer's ( 2016 ) “data literacy for teachers” (DLFT) framework. For this purpose, the conceptually similar indicators of Mandinach and Gummer ( 2016 ) are included in Table  3 alongside the corresponding indicators identified in this study. As can be seen in Table  3 , some indicators were identified in both studies, but there were also notable differences between the two sets of indicators.

Firstly, it appears that there were more fine-grained indicators across the five dimensions arising from the present study, compared to those included in Mandinach and Gummer’s ( 2016 ) DLFT framework. For instance, our study identified the importance of teacher knowledge about externally administered assessments and associated policies to guide teacher use of data, which were not a part of Mandinach and Gummer’s ( 2016 ) DLFT framework. Overall, 95 indicators of the present study, compared to 59 indicators within Mandinach and Gummer’s ( 2016 ) DLFT framework, indicates the level of details incorporated in our framework.

Secondly, perhaps the most important discrepancy is articulated in our Dimension 3 “Dispositions towards Data Use”. We have identified 25 specific indicators under this dimension, which were clustered into confidence, values/belief, trust/respect, and anxiety. These four constructs were identified as the most prominently featured psychological dispositions when teachers deal with data in the school. In Mandinach and Gummer ( 2016 ), “Dispositions, habits of mind, or factors that include data use” is mentioned, but they “chose not to include them in the conceptual framework… [due to the nature of] these positions as general to effective teaching, rather than specific to data use. They are likely to influence data literacy but are seen as more general habits of mind of good teaching” (p. 372). As such, their framework did not include dispositions as integral part of teachers’ data literacy. We argue that teacher dispositions are an essential component of teachers’ data literacy. Perhaps this discrepancy may have arisen from the views that the teacher dispositions identified in Mandinach and Gummer ( 2016 ) are general teacher qualities – such as “belief that all students can learn”, “belief in data/think critically” and “belief that improvement in education requires a continuous inquiry cycle” (p. 372). On the other hand, teachers’ dispositions in our framework were all specific to data use – such as “confidence in integrating data from multiple sources”, “confidence in discussing data with colleagues”, “trust in principals’ leadership in data use”, “trust in open and honest discussions about data use with colleagues”, and “anxiety in using data to make decision”.

On a related point, and thirdly, our framework has two separate dimensions, one focusing on individuals’ psychological dispositions under “Dimension 3: Dispositions towards Data Use”, and the other centered on behaviors “Dimension 5: Data-Related Behavior”. Most of the indicators under the behavioral dimensions were found to be social interactions, communication, discussion, participation, and engagement, as mentioned above. In Mandinach and Gummer ( 2016 ), psychological dispositions (such as belief) and behavioral tendencies (such as ethical use of data, collaboration, and communication skills) were grouped into one dimension of “Dispositions, habits of mind, or factors that include data use”. Considering these, it appears that there was less emphasis on the dispositions and behavioral tendencies in Mandinach and Gummer ( 2016 ).

On the other hand, Mandinach and Gummer ( 2016 ) offered a fine-grained description of skill-related indicators within their DLFT framework. For example, our indicator of “selecting data appropriate for different purposes” was described with more granularity in the DLFT framework: “understand what data are appropriate”, “use qualitative and quantitative data”, “prioritize data”, and “understand specificity of data to question/problem”. Likewise, our indicator of “describing data patterns” was further divided into “understand data properties”, “access patterns and trends”, “drill down into data” and “examine data” in the DLFT framework. Additionally, two indicators within the Mandinach and Gummer’ ( 2016 ) framework—“understand student privacy” and “ethics use of data, including the protection of privacy and confidentiality of data”—did not fit into any of the indicators or dimensions of the present study. This is because we were unable to locate empirical studies that directly examined ethical data management and data use among teachers. Therefore, data ethics issues, which we believe to be an important aspect of teachers’ data literacy, were omitted from our framework.

Finally, we also note the differences between the broad dimensions proposed by Mandinach and Gummer's ( 2016 ) DLFT framework and our framework. The DLFT framework consisted of: (a) identifying problems and framing questions, (b) using data, (c) transforming data into information, (d) transforming information into decisions, and (e) evaluating outcomes. These five dimensions are primarily about data skills, which was just one dimension of our framework. Furthermore, their indicator descriptions suggest heavy emphasis on data use to inform teaching and learning. In contrast, our dimensions and indicators illustrate the broader purposes and contexts of teachers' data use, highlighting the significance of fostering teacher dispositions and data-related behaviors through effective leadership and a collaborative school environment. In particular, the detailed descriptors for each of the indicators under Dimensions 3, 4, and 5 of the present study are the strengths of our framework, as they illustrate a wide range of varied and specific purposes and data-related dispositions and behaviors related to teachers’ data literacy; these descriptions are relatively sparse in Mandinach and Gummer ( 2016 ).

5.5 Limitations of the present study and future research directions

We acknowledge several limitations of the present study. First, our review focused on empirical studies published in journal articles, and omitted government documents, books, and book chapters and publications by professional organizations. Second, we did not differentiate the studies based on in-service teachers vs. pre-service teachers. Future studies may look into potential differences between these two groups and suggest policy directions and strategies for teacher preparation. Third, teachers may possess discipline-unique capabilities and inclinations, and thus it may be worthwhile to identify teacher characteristics across different disciplines (e.g., Science vs. English) and examine the influences of discipline contexts on teachers’ data use and data literacy. Fourth, exploring teachers’ data literacy required for students at different levels of schooling (e.g., early childhood, primary, and secondary) and for students with diverse needs (e.g., learning difficulties, dyslexia) may provide further insights into the specific expectations within the daily tasks and responsibilities of teachers. Fifth, most of the reviewed studies were conducted in Western or English-speaking countries, and thus our findings may have limited relevance to teacher data literacy in different world regions. Future studies may investigate cross-country characteristics in teachers’ data literacy. Sixth, our research also reveals that current studies of teachers’ data literacy have not explored the possible connections between technological advancements, particularly in AI-based systems, and teachers’ data literacy. This suggests a need to investigate the link between teachers’ data literacy and their proficiency in understanding emerging technologies such as AI-based systems. It is anticipated that discussions on data ethics will emerge as a crucial aspect of teachers’ data literacy in the era of artificial intelligence (AI). Finally, our review did not include, and thus future reviews may examine, system-level contextual factors (e.g., digital technology infrastructure, schools’ socio-economic standing) and their influences on teacher practices in data use.

6 Conclusion

Our review of 83 empirical studies published between 1990 and 2021 produced 95 specific indicators of teachers’ data literacy. The indicators were further categorised into five dimensions: (a) knowledge about data , (b) skills in using data , (c) dispositions towards data use , (d) data applications for various purposes , and (e) data-related behaviors . Our findings suggest that teachers' data literacy encompasses more than just knowledge and skills; it also includes a wide range of dispositions and behaviors. Additionally, teacher data literacy extends beyond assessing student learning outcomes and meeting accountability requirements and includes teachers’ reflection and engagement in professional development.

Data availability

Data sharing not applicable to this article as no datasets were generated or analysed during the current study.

*indicates the studies under the systematic review of this investigation

*Abdusyakur, I., & Poortman, C. L. (2019). Study on data use in Indonesian primary schools. Journal of Professional Capital and Community, 4 (3), 198-215. https://doi.org/10.1108/JPCC-11-2018-0029

*Abrams, L., Varier, D., & Jackson, L. (2016). Unpacking instructional alignment: The influence of teachers’ use of assessment data on instruction. Perspectives in Education, 34 (4), 15–28. https://doi.org/10.18820/2519593X/pie.v34i4.2

*Abrams, L. M., Varier, D., & Mehdi, T. (2021). The intersection of school context and teachers’ data use practice: Implications for an integrated approach to capacity building. Studies in Educational Evaluation , 69 , 100868. https://doi.org/10.1016/j.stueduc.2020.100868

*Albiladi, W. S., Lasater, K., & Bengtson, E. (2020). Data use among principals and teachers: divergent paths or common ground? Implications for the leadership preparation programs . Journal of School Administration Research and Development, 5 (2), 63–76.

*Andersen, I. G. (2020). What went wrong? Examining teachers’ data use and instructional decision making through a bottom-up data intervention in Denmark. International Journal of Educational Research, 102 , 101585. https://doi.org/10.1016/j.ijer.2020.101585

*Anderson, S., Leithwood, K., & Strauss, T. (2010). Leading data use in schools: Organizational conditions and practices at the school and district levels. Leadership and Policy in Schools , 9 (3), 292–327. https://doi.org/10.1080/15700761003731492

Ansyari, M. F., Groot, W., & De Witte, K. (2020). Tracking the process of data use professional development interventions for instructional improvement: A systematic literature review. Educational Research Review, 31 , 100362.

Article   Google Scholar  

Ansyari, M. F., Groot, W., & De Witte, K. (2022). A systematic review and meta-analysis of data use professional development interventions. Journal of Professional Capital and Community, 7 (3), 256–289.

*Beck, J. S., Morgan, J. J., Brown, N., Whitesides, H., & Riddle, D. R. (2020). “Asking, Learning, Seeking Out”: An exploration of data literacy for teaching. The Educational Forum , 84 (2), 150–165. https://doi.org/10.1080/00131725.2020.1674438

Beck, J. S., & Nunnaley, D. (2021). A continuum of data literacy for teaching. Studies in Educational Evaluation, 69 , 100871.

*Bianco, S. D. (2010). Improving student outcomes: data-driven instruction and fidelity of implementation in a response to intervention (RTI) model.  Teaching Exceptional Children Plus ,  6 (5).

Brookhart, S. M. (2011). Educational assessment knowledge and skills for teachers. Educational Measurement: Issues and Practice, 30 (1), 3–12.

*Brunner, C., Fasca, C., Heinze, J., Honey, M., Light, D., Mardinach, E., & Wexler, D. (2005). Linking data and learning: The grow network study. Journal of Education for Students Placed at Risk (JESPAR) , 10 (3), 241–267. https://doi.org/10.1207/s15327671espr1003_2

*Chen, L.-L. (2019). Enhancing teaching with effective data mining protocols. Journal of Educational Technology Systems , 47 (4), 500–512. https://doi.org/10.1177/0047239519835846

*Copp, D. T. (2016). The impact of teacher attitudes and beliefs about large-scale assessment on the use of provincial data for instructional change. Education Policy Analysis Archives , 24 , 109. https://doi.org/10.14507/epaa.24.2522

*Copp, D. T. (2017). Policy incentives in Canadian large-scale assessment: How policy levers influence teacher decisions about instructional change. Education Policy Analysis Archives , 25 , 115. https://doi.org/10.14507/epaa.25.3299

*Curry, K. A., Mwavita, M., Holter, A., & Harris, E. (2016). Getting assessment right at the classroom level: using formative assessment for decision making. Educational Assessment, Evaluation and Accountability, 28 (1), 89–104. https://doi.org/10.1007/s11092-015-9226-5

Darling-Hammond, L., Hightower, A. M., Husbands, J., LaFors, J. R., Young, V. M. & Christopher, C. (2003) Building instructional quality: ‘inside-out’ and ‘outside-in’ perspectives on San Diego’s school reform (Seattle, WA, Center for the Study of Teaching and Policy, University of Washington).

Data Quality Campaign. (2014). Teacher data literacy: It's about time.

*Datnow, A., Park, V., & Kennedy-Lewis, B. (2012). High school teachers' use of data to inform instruction. Journal of Education for Students Placed at Risk (JESPAR), 17 (4), 247–265. https://doi.org/10.1080/10824669.2012.718944

*Datnow, A., Park, V., & Kennedy‐Lewis, B. (2013). Affordances and constraints in the context of teacher collaboration for the purpose of data use. Journal of Educational Administration , 51 (3), 341–362. https://doi.org/10.1108/09578231311311500

*Datnow, A., Lockton, M., & Weddle, H. (2021). Capacity building to bridge data use and instructional improvement through evidence on student thinking. Studies in Educational Evaluation , 69 , 100869. https://doi.org/10.1016/j.stueduc.2020.100869

*Dunn, K. E., Airola, D. T., Lo, W.-J., & Garrison, M. (2013a). Becoming data driven: The influence of teachers’ sense of efficacy on concerns related to data-driven decision making. The Journal of Experimental Education , 81 (2), 222–241. https://doi.org/10.1080/00220973.2012.699899

*Dunn, K. E., Airola, D. T., Lo, W.-J., & Garrison, M. (2013b). What teachers think about what they can do with data: Development and validation of the data driven decision-making efficacy and anxiety inventory. Contemporary Educational Psychology, 38 (1), 87-98. https://doi.org/10.1016/j.cedpsych.2012.11.002

*Ebbeler, J., Poortman, C. L., Schildkamp, K., & Pieters, J. M. (2016). Effects of a data use intervention on educators’ use of knowledge and skills. Studies in Educational Evaluation, 48 , 19-31. https://doi.org/10.1016/j.stueduc.2015.11.002

*Ebbeler, J., Poortman, C.L., Schildkamp, K. & Pieters, J. M. (2017). The effects of a data use intervention on educators’ satisfaction and data literacy.  Educational Assessment, Evaluation and Accountability, 29 , 83–105. https://doi.org/10.1007/s11092-016-9251z

Espin, C. A., van den Bosch, R. M., van der Liende, M., Rippe, R. C., Beutick, M., Langa, A., & Mol, S. E. (2021). A systematic review of CBM professional development materials: Are teachers receiving sufficient instruction in data-based decision-making? Journal of Learning Disabilities, 54 (4), 256–268.

*Farley-Ripple, E., & Buttram, J. (2015). The development of capacity for data use: The role of teacher networks in an elementary school. Teachers College Record, 117 (4), 1–34.

*Farley-Ripple, E. N., Jennings, A. S., & Buttram, J. (2019). Toward a framework for classifying teachers’ use of assessment data. AERA Open, 5 (4). https://doi.org/10.1177/2332858419883571 .

*Farrell, C. C. (2015). Designing school systems to encourage data use and instructional improvement: A comparison of school districts and charter management organizations. Educational Administration Quarterly, 51 (3), 438–471. https://doi.org/10.1177/0013161x14539806

Filderman, M. J., Toste, J. R., Didion, L. A., Peng, P., & Clemens, N. H. (2018). Data-based decision making in reading interventions: A synthesis and meta-analysis of the effects for struggling readers. The Journal of Special Education, 52 (3), 174–187.

*Ford, T. G. (2018). Pointing teachers in the wrong direction: Understanding Louisiana elementary teachers’ use of compass high-stakes teacher evaluation data. Educational Assessment, Evaluation and Accountability , 30 (3), 251–283. https://doi.org/10.1007/s11092-018-9280-x

*Gelderblom, G., Schildkamp, K., Pieters, J., & Ehren, M. (2016). Data-based decision making for instructional improvement in primary education. International Journal of Educational Research , 80 , 1–14. https://doi.org/10.1016/j.ijer.2016.07.004

Gesel, S. A., LeJeune, L. M., Chow, J. C., Sinclair, A. C., & Lemons, C. J. (2021). A meta-analysis of the impact of professional development on teachers’ knowledge, skill, and self-efficacy in data-based decision-making. Journal of Learning Disabilities, 54 (4), 269–283.

Gummer, E., & Mandinach, E. B. (2015). Building a conceptual framework for data literacy. Teachers College Record, 117 (4), 1–22. https://www.tcrecord.org

*Hardy, I. (2015). Data, numbers and accountability: The complexity, nature and effects of data use in schools. British Journal of Educational Studies , 63 (4), 467–486. https://doi.org/10.1080/00071005.2015.1066489

*Hardy, I. (2019). Governing teachers’ work and learning through data: Australian insights. Cambridge Journal of Education , 49 (4), 501–517. https://doi.org/10.1080/0305764X.2018.1557594

*Hodges, C. A. (1996). How valid and useful are alternative assessments for decision‐making in primary grade classrooms? Reading Research and Instruction , 36 (2), 157–173. https://doi.org/10.1080/19388079709558235

Hoogland, I., Schildkamp, K., Van der Kleij, F., Heitink, M., Kippers, W., Veldkamp, B., & Dijkstra, A. M. (2016). Prerequisites for data-based decision making in the classroom: Research evidence and practical illustrations. Teaching and Teacher Education, 60 , 377–386.

*Howley, M. D., Howley, A., Henning, J. E., Gilla, M. B., & Weade, G. (2013). Intersecting domains of assessment knowledge: School typologies based on interviews with secondary teachers. Educational Assessment , 18 (1), 26–48. https://doi.org/10.1080/10627197.2013.761527

*Huffman, D., & Kalnin, J. (2003). Collaborative inquiry to make data-based decisions in schools. Teaching and Teacher Education , 19 (6), 569–580. https://doi.org/10.1016/S0742-051X(03)00054-4

*Jacobs, J., Gregory, A., Hoppey, D., & Yendol-Hoppey, D. (2009). Data literacy: Understanding teachers’ data use in a context of accountability and response to intervention. Action in Teacher Education , 31 (3), 41–55. https://doi.org/10.1080/01626620.2009.10463527

*Jimerson, J. B., & Wayman, J. (2015). Professional learning for using data: Examining teacher needs and supports. Teachers College Record, 117 (4), 1–36.

*Jimerson, J. B., Cho, V., & Wayman, J. C. (2016). Student-involved data use: Teacher practices and considerations for professional learning. Teaching and Teacher Education , 60 , 413–424. https://doi.org/10.1016/j.tate.2016.07.008

*Jimerson, J. B., Cho, V., Scroggins, K. A., Balial, R., & Robinson, R. R. (2019). How and why teachers engage students with data. Educational Studies, 45 (6), 667–691. https://doi.org/10.1080/03055698.2018.1509781

*Jimerson, J. B., Garry, V., Poortman, C. L., & Schildkamp, K. (2021). Implementation of a collaborative data use model in a United States context. Studies in Educational Evaluation , 69 , 100866. https://doi.org/10.1016/j.stueduc.2020.100866

*Jimerson, J. B. (2014). Thinking about data: Exploring the development of mental models for “data use” among teachers and school leaders. Studies in Educational Evaluation, 42 , 5–14. https://doi.org/10.1016/j.stueduc.2013.10.010

*Joo, Y. H. (2020). Promoting sustainable data-based decision-making in the Korean educational information disclosure system. Sustainability , 12 (17), 6762. https://doi.org/10.3390/su12176762

*Kallemeyn, L. M. (2014). School-level organisational routines for learning: supporting data use. Journal of Educational Administration, 52 (4), 529–548. https://doi.org/10.1108/JEA-02-2013-0025

*Kanjee, A., & Moloi, Q. (2014). South African teachers’ use of national assessment data. South African Journal of Childhood Education , 4 (2), 24. https://doi.org/10.4102/sajce.v4i2.206

Katz, L. (1993). Dispositions as Educational Goals. ERIC Digest #211.

*Kerr, K. A., Marsh, J. A., Ikemoto, G. S., Darilek, H., & Barney, H. (2006). Strategies to promote data use for instructional improvement: Actions, outcomes, and lessons from three urban districts. American Journal of Education , 112 (4), 496–520. https://doi.org/10.1086/505057

*Keuning, T., Geel, M. V., Visscher, A., Fox, J.-P., & Moolenaar, N. M. (2016). The transformation of schools’ social networks during a data-based decision making reform. Teachers College Record, 118 (9), 1–33. https://doi.org/10.1177/016146811611800908

*Keuning, T., Van Geel, M., & Visscher, A. (2017). Why a data-based decision-making intervention works in some schools and not in others. Learning Disabilities Research & Practice, 32 (1), 32-45. https://doi.org/10.1111/ldrp.12124

*Kippers, W. B., Poortman, C. L., Schildkamp, K., & Visscher, A. J. (2018). Data literacy: What do educators learn and struggle with during a data use intervention? Studies in Educational Evaluation, 56 , 21–31 . https://doi.org/10.1016/j.stueduc.2017.11.001

Lee, J. (2021). Teacher–student relationships and academic achievement in Confucian educational countries/systems from PISA 2012 perspectives. Educational Psychology, 41 (6), 764–785.

Lee, J., & Durksen, T. (2018). Dimensions of academic interest among undergraduate students: Passion, confidence, aspiration, and self-expression. Educational Psychology, 38 (2), 120–138. https://doi.org/10.1080/01443410.2017.1342770

Lee, J., & Stankov, L. (2018). Non-cognitive predictors of academic achievement: Evidence from TIMSS and PISA. Learning and Individual Differences, 65 , 50–64.

Li, L., Zhu, H., & Li, H. (2022). School leadership enhances secondary students’ achievement in rural China through teacher commitment and collaborative culture. Frontiers in Psychology, 13 , 894688.

*Lockton, M., Weddle, H., & Datnow, A. (2020). When data don’t drive: Teacher agency in data use efforts in low-performing schools. School Effectiveness and School Improvement , 31 (2), 243–265. https://doi.org/10.1080/09243453.2019.1647442

Mandinach, E. B. (2012). A perfect time for data use: Using data-driven decision making to inform practice. Educational Psychologist, 47 (2), 71–85. https://doi.org/10.1080/00461520.2012.667064

Mandinach, E. B., & Gummer, E. S. (2013a). Defining data literacy: A report on a convening of experts. Journal of Educational Research and Policy Studies, 13 (2), 6–28.

Google Scholar  

Mandinach, E. B., & Gummer, E. S. (2013b). A systemic view of implementing data literacy in educator preparation. Educational Researcher, 42 (1), 30–37. https://doi.org/10.3102/0013189x12459803

Mandinach, E. B., & Gummer, E. S. (2016). What does it mean for teachers to be data literate: Laying out the skills, knowledge, and dispositions. Teaching and Teacher Education, 60 , 366–376. https://doi.org/10.1016/j.tate.2016.07.011

Mandinach, E. B., & Schildkamp, K. (2021). Misconceptions about data-based decision making in education: An exploration of the literature. Studies in Educational Evaluation, 69 , 100842. https://doi.org/10.1016/j.stueduc.2020.100842

Mandinach, E. B., Parton, B. M., Gummer, E. S., & Anderson, R. B. (2015). Responsible data use as a skill in educators’ data literacy. Phi Delta Kappan, 96 (5), 25e28.

Mandinach, E. B. (2014). What's the difference between assessment literacy and data literacy?. In Presentation made at the data quality Campaign webinar. Retrieved from http://dataqualitycampaign.org/files/events/Data%20literacy%20event%20slides.pdf

*Marsh, J. A., & Farrell, C. C. (2015). How leaders can support teachers with data-driven decision making: A framework for understanding capacity building. Educational Management Administration & Leadership , 43 (2), 269–289. https://doi.org/10.1177/1741143214537229

*Marsh, J. A., Sloan McCombs, J., & Martorell, F. (2010). How instructional coaches support data-driven decision making: Policy implementation and effects in Florida middle schools. Educational Policy , 24 (6), 872–907. https://doi.org/10.1177/0895904809341467

*Mausethagen, S., Prøitz, T., & Skedsmo, G. (2018). Teachers’ use of knowledge sources in ‘result meetings’: Thin data and thick data use. Teachers and Teaching , 24 (1), 37–49. https://doi.org/10.1080/13540602.2017.1379986

*McDonald, J.P. (2019). Toward more effective data use in teaching. Phi Delta Kappan , 100 (6), 50–54. https://doi.org/10.1177/0031721719834029

*Michaud, R. (2016). The nature of teacher learning in collaborative data teams. The Qualitative Report, 21 (3), 529–545. Retrieved from http://nsuworks.nova.edu/tqr/vol21/iss3/7

*Militello, M., Bass, L., Jackson, K., & Wang, Y. (2013). How data are used and misused in schools: Perceptions from teachers and principals. Education Sciences , 3 (2), 98–120. https://doi.org/10.3390/educsci3020098

*Mills, M., Mockler, N., Stacey, M., & Taylor, B. (2021). Teachers’ orientations to educational research and data in England and Australia: Implications for teacher professionalism. Teaching Education, 32 (1), 77–98. https://doi.org/10.1080/10476210.2020.1843617

Mischel, W., & Shoda, Y. (1995). A cognitive-affective system theory of personality: Reconceptualizing situations, dispositions, dynamics, and invariance in personality structure. Psychological Review, 102 (2), 246–268.

Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. BMJ, 339 , b2535. https://doi.org/10.1136/bmj.b2535

*Mokhtari, K., Thoma, J., & Edwards, P. (2009). How one elementary school uses data to help raise students’ reading achievement. The Reading Teacher , 63 (4), 334–337. https://www.jstor.org/stable/30249385

*Nicholson, J., Capitelli, S., Richert, A. E., Wilson, C., & Bove, C. (2017). Teacher leaders building foundations for data-informed teacher learning in one urban elementary school. The New Educator , 13 (2), 170–189. https://doi.org/10.1080/1547688X.2016.1140856

*O’Brien, S., McNamara, G., O’Hara, J., & Brown, M. (2022). Learning by doing: Evaluating the key features of a professional development intervention for teachers in data-use, as part of whole school self-evaluation process. Professional Development in Education , 48 (2), 273–297. https://doi.org/10.1080/19415257.2020.1720778

*Omoso, E., Schildkamp, K., & Pieters, J. (2019). Data use in Kenyan secondary schools. Journal of Professional Capital and Community , 4 (3), 216–231. https://doi.org/10.1108/JPCC-11-2018-0027

*Park, V., & Datnow, A. (2009). Co-constructing distributed leadership: District and school connections in data-driven decision-making. School Leadership & Management , 29 (5), 477–494. https://doi.org/10.1080/13632430903162541

*Poortman, C. L., & Schildkamp, K. (2016). Solving student achievement problems with a data use intervention for teachers. Teaching and Teacher Education, 60 , 425-433. https://doi.org/10.1016/j.tate.2016.06.010

*Powell, S. R., Lembke, E. S., Ketterlin-Geller, L. R., Petscher, Y., Hwang, J., Bos, S. E., Cox, T., Hirt, S., Mason, E. N., Pruitt-Britton, T., Thomas, E., & Hopkins, S. (2021). Data-based individualisation in mathematics to support middle school teachers and their students with mathematics learning difficulty. Studies in Educational Evaluation, 69 , 100897. https://doi.org/10.1016/j.stueduc.2020.100897

*Prenger, R., & Schildkamp, K. (2018). Data-based decision making for teacher and student learning: A psychological perspective on the role of the teacher. Educational Psychology , 38 (6), 734–752. https://doi.org/10.1080/01443410.2018.1426834

*Reed, D. K. (2015). Middle level teachers’ perceptions of interim reading assessments: An exploratory study of data-based decision making. RMLE Online , 38 (6), 1–13. https://doi.org/10.1080/19404476.2015.11462119

*Reeves, T. D., Summers, K. H., & Grove, E. (2016). Examining the landscape of teacher learning for data use: The case of Illinois. Cogent Education , 3 (1), 1211476. https://doi.org/10.1080/2331186X.2016.1211476

*Reeves, T. D. (2017). School level and other differences in Illinois teachers’ use of data to inform instruction.  Mid-Western Educational Researcher ,  29 (4), 332–354.

Schildkamp, K. (2019). Data-based decision-making for school improvement: Research insights and gaps. Educational Research, 61 (3), 257–273. https://doi.org/10.1080/00131881.2019.1625716

*Schildkamp, K., Poortman, C., Luyten, H., & Ebbeler, J. (2017). Factors promoting and hindering data-based decision making in schools . School Effectiveness and School Improvement, 28 (2), 242–258. https://doi.org/10.1080/09243453.2016.1256901

*Schildkamp, K., Smit, M., & Blossing, U. (2019). Professional development in the use of data: From data to knowledge in data teams. Scandinavian Journal of Educational Research , 63 (3), 393–411. https://doi.org/10.1080/00313831.2017.1376350

Shavelson, R. J., Ruiz-Primo, M. A., & Wiley, E. W. (2005). Windows into the mind. Higher Education, 49 (4), 413–430.

*Snodgrass Rangel, V., Monroy, C., & Bell, E. (2016). Science teachers’ data use practices: A descriptive analysis. Education Policy Analysis Archives , 24 , 86. https://doi.org/10.14507/epaa.24.2348

*Snodgrass Rangel, V. S., Bell, E., & Monroy, C. (2019). Teachers’ sensemaking and data use implementation in science classrooms. Education and Urban Society, 51 (4), 526–554. https://doi.org/10.1177/0013124517727053

*Staman, L., Timmermans, A. C., & Visscher, A. J. (2017). Effects of a data-based decision making intervention on student achievement. Studies in Educational Evaluation, 55 , 58–67. https://doi.org/10.1016/j.stueduc.2017.07.002

Stankov, L., & Lee, J. (2008). Confidence and cognitive test performance. Journal of Educational Psychology, 100 (4), 961–976. https://doi.org/10.1037/a0012546

Sykes, G. (1999). Teacher and student learning: Strengthening their connection. In L. Darling-Hammond & G. Sykes (Eds.), Teaching as the learning profession: Handbook of policy and practice (pp. 151–179). Jossey-Bass.

Thomas, J., & Harden, A. (2008). Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Medical Research Methodology, 8 , 45. https://doi.org/10.1186/1471-2288-8-45

*Thomas, K., & Huffman, D. (2011). Navigating the challenges of helping teachers use data to inform educational decisions. Administrative Issues Journal , 1 (2). https://doi.org/10.5929/2011.1.2.7

*Thompson, C. J. (2012). Improving the use of data in early reading intervention programs in northwest Florida. Journal of Educational Research and Practice, 2 (1), 1-14.

*van Der Scheer, E. A., & Visscher, A. J. (2018). Effects of a data-based decision-making intervention for teachers on students’ mathematical achievement. Journal of Teacher Education , 69 (3), 307–320. https://doi.org/10.1177/0022487117704170

*Van Gasse, R., Vanlommel, K., Vanhoof, J., & Van Petegem, P. (2017). The impact of collaboration on teachers’ individual data use. School Effectiveness and School Improvement , 28 (3), 489–504. https://doi.org/10.1080/09243453.2017.1321555

*Van Gasse, R., Vanhoof, J., & Van Petegem, P. (2018). Instrumental, conceptual and symbolic effects of data use: The impact of collaboration and expectations. Educational Studies, 44 (5), 521–534. https://doi.org/10.1080/03055698.2017.1382328

*Van Gasse, R., Goffin, E., Vanhoof, J., & Van Petegem, P. (2021). For squad-members only! Why some teachers are more popular to interact with than others in data use. Studies in Educational Evaluation , 69 , 100881. https://doi.org/10.1016/j.stueduc.2020.100881

*Vanlommel, K., & Schildkamp, K. (2019). How do teachers make sense of data in the context of high-stakes decision making? American Educational Research Journal, 56 (3), 792–821. https://doi.org/10.3102/0002831218803891

*Vanlommel, K., Vanhoof, J., & Van Petegem, P. (2016). Data use by teachers: The impact of motivation, decision-making style, supportive relationships and reflective capacity. Educational Studies , 42 (1), 36–53. https://doi.org/10.1080/03055698.2016.1148582

*Vanlommel, K., Van Gasse, R., Vanhoof, J., & Van Petegem, P. (2021). Sorting pupils into their next educational track: How strongly do teachers rely on data-based or intuitive processes when they make the transition decision? Studies in Educational Evaluation , 69 , 100865. https://doi.org/10.1016/j.stueduc.2020.100865

*von der Embse, N. P., Kilgus, S. P., Eklund, K., Zahn, M., Peet, C., & Durango, S. (2021). Promoting effective decision making: Training educators to collect and use social–emotional skill assessment data to inform tier 2 interventions. School Psychology Review, 51 (5), 574-588. https://doi.org/10.1080/2372966X.2020.1827680

*Wachen, J., Harrison, C., & Cohen-Vogel, L. (2018). Data use as instructional reform: exploring educators’ reports of classroom practice. Leadership and Policy in Schools , 17 (2), 296–325. https://doi.org/10.1080/15700763.2016.1278244

*Wardrip, P. S., & Herman, P. (2018). ‘We’re keeping on top of the students’: Making sense of test data with more informal data in a grade-level instructional team. Teacher Development , 22 (1), 31–50. https://doi.org/10.1080/13664530.2017.1308428

Weiss, J. (2012). Data for improvement, data for accountability. Teachers College Record, 114 (11), 110307.

*Wolff, A., Wermelinger, M., & Petre, M. (2019). Exploring design principles for data literacy activities to support children’s inquiries from complex data. International Journal of Human-Computer Studies , 129 , 41–54. https://doi.org/10.1016/j.ijhcs.2019.03.006

*Young, V. M. (2006). Teachers’ use of data: Loose coupling, agenda setting, and team norms. American Journal of Education , 112 (4), 521–548. https://doi.org/10.1086/505058

*Zeuch, N., Förster, N., & Souvignier, E. (2017). Assessing teachers’ competencies to read and interpret graphs from learning progress assessment: Results from tests and interviews. Learning Disabilities Research & Practice , 32 (1), 61–70. https://doi.org/10.1111/ldrp.12126

*Zhao, X., Van den Heuvel-Panhuizen, M., & Veldhuis, M. (2016). Teachers’ use of classroom assessment techniques in primary mathematics education—An explorative study with six Chinese teachers. International Journal of STEM Education, 3 (1), 19. https://doi.org/10.1186/s40594-016-0051-2

Download references

Acknowledgements

This work was supported by the Faculty of Arts, Design & Architecture and the School of Education at UNSW, Sydney.

Open Access funding enabled and organized by CAUL and its Member Institutions.

Author information

Authors and affiliations.

School of Education, University of New South Wales, Sydney, NSW, Australia

Jihyun Lee, Dennis Alonzo, Kim Beswick, Jan Michael Vincent Abril & Adrian W. Chew

Yangon University of Education, Yangon, Myanmar

Cherry Zin Oo

You can also search for this author in PubMed   Google Scholar

Contributions

Jihyun Lee: Conceptualization; Data curation; Formal analysis; Funding acquisition; Investigation; Methodology; Supervision; Validation; Visualization; Roles/Writing—original draft; and Writing—review & editing

Dennis Alonzo: Conceptualization; Formal analysis; Funding acquisition; Investigation; Methodology; Project administration; Resources

Kim Beswick: Conceptualization; Funding acquisition; Investigation; Writing—review & editing

Jan Abril: Data curation; Validation

Adrian W. Chew: Data curation; Visualization

Cherry Zin O: Data curation; Formal analysis

Corresponding author

Correspondence to Jihyun Lee .

Ethics declarations

Conflict of interest statement.

There is no conflict of interest.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Data Management Statement

The manuscript has no data attached, as it is a systematic review.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Lee, J., Alonzo, D., Beswick, K. et al. Dimensions of teachers’ data literacy: A systematic review of literature from 1990 to 2021. Educ Asse Eval Acc (2024). https://doi.org/10.1007/s11092-024-09435-8

Download citation

Received : 06 November 2023

Accepted : 27 March 2024

Published : 06 May 2024

DOI : https://doi.org/10.1007/s11092-024-09435-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Data literacy
• Find a journal
• Publish with us
• Track your research

We use cookies to help our site work, to understand how it is used, and to tailor ads that are more relevant to you and your interests.

By accepting, you agree to cookies being stored on your device. You can view details and manage settings at any time on our cookies policy page.

Utilising open research practices to conduct a systematic review and meta-analysis of healthcare data

Claire Price, a PhD student in health sciences, has been focusing on quantifying weight loss prior to pancreatic cancer diagnosis, to help improve early detection whilst embracing the principles of open research, particularly for transparency and reproducibility. 

Pancreatic cancer is rare but has dismal survival rates meaning it’s the sixth cause of UK cancer mortality.  It often presents with non-specific symptoms. This makes early diagnosis challenging. However, because weight loss occurs in most patients with pancreatic cancer, it could be a useful diagnostic marker. Our aim was to improve its utility as a marker for pancreatic cancer by quantifying pre-diagnosis weight loss. We undertook a systematic review with meta-analysis.  

However, healthcare data must remain private and confidential. Healthcare datasets cannot be shared in their original form. Only non-identifiable data can be shared and only in a safe environment. This means it is challenging to ensure transparency in research using healthcare data. 

Our approach and challenges 

Before conducting this review CP attended an open research course at the University of Surrey (1) to improve her knowledge of open research and best practices. We registered the study protocol on PROSPERO (2), which is an international database where researchers can submit their proposals for literature reviews. This helps the research to be transparent and reproducible as well as avoiding duplication of work.  It also allows a comparison of what was planned and what was completed for the research.  We followed PRISMA guidelines which provide a framework for what information must be included when writing up the research, such as a well-defined research question together with reasons studies can be excluded from the research.  

We conducted a systematic review and meta-analysis of studies which included quantifiable information about weight loss prior to pancreatic cancer diagnosis. The search for studies to include was conducted using five different databases in November 2023. We used a tool called ROBINS-I (3) to assess possible bias in how the studies we found were conducted.  Examining possible bias is important as this may influence the results when synthesising the data from many studies and using a tool for this assessment is important so that bias is appraised in a standard and reproducible manner. 

Extracted data has been deposited in a database for openly sharing datasets (Zenodo (4)) and will be submitted as a Research Elements article which focusses on how the data was extracted from the studies. The review will be shared as a preprint to improve the speed of dissemination. It will also be submitted for publication in a peer-reviewed open access journal, increasing communication and knowledge in the area of improving the early detection of pancreatic cancer. 

Transparency of methodology enables reproducibility as well as preventing duplication of effort and cost. Depositing extracted data maximises the findability and accessibility of the data. This encourages data reuse practices which supports data minimisation. This means that only as little data as is required is collected and that the same data is not collected several times, which is especially important for potentially invasive research, such as that involving healthcare data, 

We also encountered challenges when conducting this review which were related to limited open research practices in the original studies.  Extracting data for meta-analysis was difficult due to heterogenous study types and a lack of transparency. Many original studies lacked meta-data, only presented data graphically or only reported summary statistics (e.g. as the result of a regression model) so the raw data is not provided. Authors of the original studies often could not clarify results. Without meta-data to enable data reuse this meant some studies had to be excluded from meta-analysis. 

The outcome 

Out of 6,664 original hits 30 studies were included in the review and meta-analysis. Random effects meta-analysis was conducted across three different ways of reporting weight loss (the amount of weight lost (kg), BMI change and proportion of people losing weight). We found 57.4% of people lose an average of 5.89 kg or have a BMI change of -2.54 kg/m 2 prior to pancreatic cancer diagnosis. However, there was a great deal of heterogeneity between the studies so there is limited confidence in the results and their wider applicability.   

Conclusions 

Research using healthcare records needs to be approached carefully, due to privacy and other ethical considerations. However, maximising open research practices enables data to be shared to maximise patient benefit. This avoids repetition of work and increases reproducibility. 

Original datasets that consist of healthcare records cannot be made open access. However, open research practices can still be adopted by sharing data that has been aggregated or deindentified. It is important to present meta-data to provide context for the research. This improves standardisation across study types and facilitates study combinability. 

1. Fellows. Open Research training [Internet]. [cited 2024 Apr 8]. Available from: https://www.surrey.ac.uk/library/open-research/open-research-training 

2. Price CA, Cooke D, Smith N, Wynn M, Lemanska A. Weight Loss Prior to Pancreatic Cancer Diagnosis: A systematic review and meta-analysis [Internet]. PROSPERO. 2022 [cited 2023 Jan 6]. Available from: https://www.crd.york.ac.uk/PROSPERO/display_record.php?RecordID=302985  

3. Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016 Oct 12;355:i4919. 

4. Price CA. PaCaClaire/Systematic-Review-Weight-Loss-Extracted-Data: Weight Loss Prior to Pancreatic Cancer Diagnosis - Systematic Review Extracted Data [Internet]. 2023. Available from: https://zenodo.org/records/10118729 

Share what you've read?

Featured academics.

Claire Price

Login | Register

• Editorial Team

An In-Depth Literature Review of E-Portfolio Implementation in Higher Education: Processes, Barriers, and Strategies

Authors: Hongyan Yang (The University of Tennessee, Knoxville) , Rachel Wong (The University of Tennessee, Knoxville)

Literature Review

• Harvard Citation Style
• Vancouver Citation Style
• APA Citation Style
• Download RIS
• Download BibTeX

This is an accepted article with a DOI pre-assigned that is not yet published.

This literature review examines the implementation of e-portfolios in higher education, with a focus on the implementation process, potential barriers, and strategies for overcoming challenges. This review seeks to provide instructional designers and higher education instructors with design strategies to effectively implement e-portfolios. Through an analysis of seventeen studies, we identified six common steps in the implementation process, including identifying a purpose, stakeholders, and platform, conducting workshops, creating e-portfolios, and evaluating the project. The implementation process also raised eight concerns, including concerns related to technology, policy, pedagogy, artifact quality, privacy, student motivation, academic integrity, and teacher workload. To address these concerns, existing strategies suggest that successful implementation requires training and policy support, student-centered pedagogy, criteria for assessing artifacts, privacy and data protection, feedback, anti-plagiarism measures, and shared successful models.

Keywords: literature review, e-Portfolio, implementation, higher education

Accepted on 20 Apr 2024

Peer reviewed, creative commons attribution-noncommercial-sharealike 4.0, harvard-style citation.

Yang, H & Wong, R. () 'An In-Depth Literature Review of E-Portfolio Implementation in Higher Education: Processes, Barriers, and Strategies', Issues and Trends in Learning Technologies . doi: 10.2458/itlt.5809

Show: Vancouver Citation Style | APA Citation Style

Vancouver-Style Citation

Yang, H & Wong, R. An In-Depth Literature Review of E-Portfolio Implementation in Higher Education: Processes, Barriers, and Strategies. Issues and Trends in Learning Technologies. ; doi: 10.2458/itlt.5809

Show: Harvard Citation Style | APA Citation Style

APA-Style Citation

Yang, H & Wong, R. (, ). An In-Depth Literature Review of E-Portfolio Implementation in Higher Education: Processes, Barriers, and Strategies. Issues and Trends in Learning Technologies doi: 10.2458/itlt.5809

Show: Harvard Citation Style | {% trans 'Vancouver Citation Style' %}

Non Specialist Summary

This article has no summary

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest Content
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 10, Issue 1
• Incidence and prevalence of interstitial lung diseases worldwide: a systematic literature review
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0001-9784-416X Rikisha Shah Gupta 1 , 2 ,
• Ardita Koteci 3 , 4 ,
• Ann Morgan 3 , 4 ,
• Peter M George 5 and
• Jennifer K Quint 1 , 3
• 1 National Heart and Lung Institute , Imperial College London , London , UK
• 2 Real-World Evidence , Gilead Sciences , Foster City , CA , USA
• 3 Imperial College London , London , UK
• 4 NIHR Imperial Biomedical Research Centre , London , UK
• 5 Royal Brompton and Harefield NHS Foundation Trust , London , UK
• Correspondence to Rikisha Shah Gupta; r.shah20{at}imperial.ac.uk

Interstitial lung disease (ILD) is a collective term representing a diverse group of pulmonary fibrotic and inflammatory conditions. Due to the diversity of ILD conditions, paucity of guidance and updates to diagnostic criteria over time, it has been challenging to precisely determine ILD incidence and prevalence. This systematic review provides a synthesis of published data at a global level and highlights gaps in the current knowledge base. Medline and Embase databases were searched systematically for studies reporting incidence and prevalence of various ILDs. Randomised controlled trials, case reports and conference abstracts were excluded. 80 studies were included, the most described subgroup was autoimmune-related ILD, and the most studied conditions were rheumatoid arthritis (RA)-associated ILD, systemic sclerosis associated (SSc) ILD and idiopathic pulmonary fibrosis (IPF). The prevalence of IPF was mostly established using healthcare datasets, whereas the prevalence of autoimmune ILD tended to be reported in smaller autoimmune cohorts. The prevalence of IPF ranged from 7 to 1650 per 100 000 persons. Prevalence of SSc ILD and RA ILD ranged from 26.1% to 88.1% and 0.6% to 63.7%, respectively. Significant heterogeneity was observed in the reported incidence of various ILD subtypes. This review demonstrates the challenges in establishing trends over time across regions and highlights a need to standardise ILD diagnostic criteria.PROSPERO registration number: CRD42020203035.

• Asbestos Induced Lung Disease
• Clinical Epidemiology
• Interstitial Fibrosis
• Systemic disease and lungs

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:  http://creativecommons.org/licenses/by-nc/4.0/ .

https://doi.org/10.1136/bmjresp-2022-001291

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

Interstitial lung disease (ILD) is a collective term representing a diverse group of lung conditions characterised by the presence of non-infective infiltrates, most commonly in the pulmonary interstitium and alveoli, which in certain cases manifest as architectural distortion and irreversible fibrosis. These conditions vary in their aetiology, clinical pathways, severity and prognosis. 1 Some conditions resolve completely without pharmacological intervention, whereas others, such as idiopathic pulmonary fibrosis (IPF) and non-IPF progressive fibrosing (PF) ILDs, inexorably progress to respiratory failure and premature mortality despite treatment.

Given its universally progressive nature and poor prognosis, IPF has attracted the most research attention and the current literature suggests a wide variation in disease distribution across Europe and USA. IPF prevalence varies between 0.63 and 7.6 per 100 000 persons in the USA and Europe 2 3 with a sharp increase with age.

More recently, there have been several studies investigating the incidence and prevalence of non-IPF ILDs, mainly autoimmune ILDs. Most of these reviews included studies drawn from single centres. Epidemiological data for non-IPF ILDs is inconsistent which makes it challenging to fully appreciate the ILD landscape. A recent review reported the prevalence of ILD in myositis conditions ranged from 23% in America to 50% in Asia. 4 Sambataro et al 5 reported about 20% of primary Sjogren’s syndrome patients were diagnosed with ILD. Additionally, there have been a few studies evaluating the incidence of drug induced ILD (DILD). 6–8 Guo et al 9 reported ILD incidence ranged from 4.6 to 31.5 per 100 000 persons in Europe and North America. A recent study using Global Burden of Disease data indicated the global ILD incidence in the past 10 years has risen by 51% (313.2 cases in 1990 to 207.2 per 1 00 000 cases in 2019). 10 These published estimates highlight a discernible variation in the ILD epidemiology across countries. It is unclear whether this is an ‘actual’ difference in the numbers across regions or whether the heterogeneity is driven by lack of guidelines and inconsistencies in ILD diagnostic pathways and standards of care. Likewise, while evidence suggests that the incidence of ILD has been rising over time, 9 whether this increase reflects a true increase in the disease burden, possibly related to an ageing population or whether this is due to improvements in detection, increased availability of cross-sectional imaging or coding practices over time is unknown.

This systematic review appraises the published literature on the incidence and prevalence of various ILDs over the last 6 years. We aimed to provide a comprehensive understanding of global incidence and prevalence. Specifically, we sought to identify areas where data are robust, to better appreciate the burden of ILD conditions and to comprehend the implications on healthcare utilisation and resources. We also set out to highlight areas where there remains a need for further study.

Study registration

This protocol has been drafted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols guidelines 11 and registered with the International Prospective Register of Systematic Reviews, PROSPERO ( CRD42020203035 ). Please refer to the online supplemental material for the full study protocol.

Supplemental material

Search strategy and selection criteria.

A systematic search of Medline and Embase was carried out in September 2021 to identify relevant studies investigating the incidence and prevalence of various ILDs. The search criteria were developed with support of librarian ( online supplemental figure E1 ). Due to the high volume of papers, we restricted this study period to papers published in the past 6 years. This search was limited to human studies written in English that were published between 2015 and 2021. The full search strategy and data sources included are described in online supplemental material .

Study population

Inclusion criteria included observational studies reporting the incidence and/or prevalence of individual ILDs, with study participants aged over 18 years old. Randomised controlled trials, case reports, reviews and conference abstracts were excluded. Studies which referred to DILD only were excluded because (1) there were many abstracts reporting on DILD, therefore this could be a standalone review and (2) epidemiology of DILD was a subject of a recent systematic review. 12 The first author (RG) screened all records by title and abstract; to begin with, the second reviewer (AK) independently screened 10% of all records. If there was a disagreement between RG and AK, an additional 15% were screened by AK. All studies identified as eligible for full text review were reviewed by RG, with AK reviewing 50% of eligible studies. Any disagreement was resolved through discussion with other authors, including an ILD expert. Reference of included studies were searched for additional literature.

Following full text review, RG carried out data extraction for eligible studies. AK independently extracted data for 25% of studies using the same template. RG assessed the quality for all included studies, reporting incidence and/or prevalence using a modified Newcastle Ottawa Scale (NOS). There were two NOS modified scales, one each for studies reporting prevalence and/incidence. AK independently assessed the quality of 25% of included studies. If there was a discrepancy between the data extraction and/or quality assessment conducted by RG and AK, then additional 15% were extracted and/or reviewed by AK.

It was noted that for IPF, many authors adopted what they termed ‘broad’ and ‘narrow’ case definitions. For example, Raghu et al 2 defined patients with International Classification of Disease, Ninth Revision (ICD-9) code 516.3 as a broadly defined case of IPF, and those who had this ICD-9 code alongside a claim for a surgical lung biopsy, transbronchial lung biopsy, or CT thorax as a narrowly defined case. We summarised the data using various reported case definitions. If multiple estimates were reported in a study, only the most recent estimate was included in this review.

There were two common themes around the reporting of prevalence. Studies drawn from the general population (reported prevalence per 100 000 persons) and studies drawn from multicentre or single centres (reported prevalence as the proportion of patients with ILD in the study cohort).

For this review, we have classified ILDs based on aetiology, grouped by conditions linked to environmental or occupational exposures, conditions typified by granulomatous inflammation, autoimmune ILDs and ILDs with no known cause ( online supplemental figure E2 ). 1

Evidence synthesis

The initial plan for this review was to conduct meta-analysis. However, due to high heterogeneity, we were unable to meta-analyse. Therefore, we have proceeded with data synthesis across the ILD subgroups.

Total number of included studies

The literature search yielded a total of 12 924 studies, of which 80 were included in this review. Online supplemental figure E3 demonstrates the selection process for all studies and highlights reasons for exclusion at each stage.

Although 80 unique publications were included, some papers explored the epidemiology of more than one ILD, the total count of reported estimates is 88. Half of the included publications explored autoimmune-related ILDs (n=44/88)( online supplemental figure E4 ).

Geographically, ILD publications represented all major world regions, but were predominantly from Asia (n=30, 34.1%) and Europe (n=23, 26.1%) ( figure 1 ).

• Download figure
• Open in new tab
• Download powerpoint

Geographical distribution of publications included.

Studies reporting prevalence

Eight studies reported the prevalence of IPF in general population. Prevalence of IPF was commonly reported applying ‘primary’, ‘broad’, ‘intermediate’ and/or ‘narrow’ case definitions. In the general population, the prevalence of IPF ranged from 7 to 1650 per 100 000 persons ( table 1 ). When explored within various case definitions, the prevalence for ‘broad’ cases ranged from 11 (USA, 2010) 2 to 1160 (USA, 2021) 16 ; for ‘narrow’ cases, this ranged from 7 (USA, 2010) 2 to 725 (USA, 2019). 16 There was only one study that reported IPF prevalence of 8.6% using a multicentre study setting. 19

• View inline

Studies reporting IPF prevalence per 100 000 persons by various case definitions

Twelve studies reported estimates for non-IPF ILDs in the general population ( online supplemental figure E5 ), with most of these conducted in the USA. The prevalence of systemic sclerosis (SSc) ILD in the general population ranged from 2.3 (Canada, 2018) 20 to 19 (USA, 2017) 21 per 100 000 persons. The highest SSc-ILD prevalence was reported in Medicare data which included patients aged 65 years and above. 21 22 For rheumatoid arthritis (RA) ILD, prevalence in an RA Medicare cohort was 2%. 23

Forty-six studies reported the prevalence of autoimmune-related ILD in cohorts of patients with an autoimmune condition or occupational ILD in workers with specific exposures. These studies primarily reported prevalence as a proportion, with the denominator representing patients with an autoimmune disorder or people working at a factory with exposure to certain agents, such as silica or asbestosis ( figure 2 ). Most of these estimates were drawn from cohorts at single or multiple tertiary centres, disease registries or a factory in the case of occupational ILD. Significant heterogeneity was noted in the reported prevalence of ILD associated with SSc, RA and Sjogren’s ( figure 2 ). The prevalence of ILD in SSc ranged from 26.1% (Australia, 2015) 36 to 88.1% (India, 2013). 44 Similarly, Sjogren’s ILD ranged from 1% (Sweden, 2011) 55 to 87.8% (Saudi Arabia, 2021). 56 In addition to dissimilarities in the prevalence across various regions, we also observed variation within region-specific estimates. For example, the 4 studies 47 50–52 which reported Sjogren’s ILD prevalence within China, estimated a 4-fold variation in magnitude (18.6% in 2011 47 to 78.6% in 2014). 52 Likewise, for RA ILD, there was substantial variation in the reported prevalence in Egypt (0.8% vs 63.7%). 31 32 Among the occupational-related ILDs ( figure 2 ), silicosis was the most explored condition (n=8)). Among these eight studies, there was a considerable variation in the reported prevalence of silicosis. Souza et al 61 reported an approximately 7-fold higher estimate of silicosis prevalence than that reported by Siribaddana et al (37% vs 5.6%, respectively). 65

Studies reporting non-IPF prevalence as percentage of study population. DM, dermatomyositis; HP, hypersensitivity pneumonitis; IIP, idiopathic interstitial pneumonia; ILD, interstitial lung disease; LAM, lymphangioleiomyomatosis; MCTD, mixed connective tissue disorder; multiC, multicentre; PLCH, pulmonary langerhans cell histiocytosis; PM, polymyositis; RA, rheumatoid arthritis; reg, registry; single, single centre; SSc, systemic sclerosis. Details on the study population, sample size and ILD diagnosis methods are summarised in online supplemental tables E1–E31 .

Studies reporting incidence

Significant discrepancies were observed in reported ILD incidence across subgroups and individual conditions, mainly due to differences in the study setting. Depending on the study setting and type of data source used, some authors reported an incidence rate (per 100 000 person-years), while others reported incidence proportion. Table 2 lists IPF incidence by case classification and country, and figure 3 provides a list of studies reporting incidence of non-IPF ILDs.

Published estimates of IPF incidence, stratified by various case definitions

Studies reporting ILD incidence, grouped by ILD subgroups. ICD-9-CM, International Classification of Disease, Ninth Revision, Clinical Modification; ILD, interstitial lung disease; py, person-years; RA, rheumatoid arthritis; SSc, systemic sclerosis. Ɨ Narrow silicosis definition used: Medicare beneficiaries with any claim that included ICD-9-CM code 502, pneumoconiosis due to other silica or silicates, listed in any position during 1999–2014, with at least one inpatient, skilled nursing or home health agency claim, or at least two outpatient provider claims within 365 days of each other and cases with a chest X-ray or CT scan 30 days before or 30 days after a silicosis claim. Details on the study population, sample size and ILD diagnosis methods are summarised in online supplemental tables E1–E31 .

In this review, we synthesised the evidence for the incidence and prevalence of ILDs from studies published between 2015 and 2021. Considering the changing ILD nomenclature and the desire to reflect more current estimates, in this review, we decided to restrict the study period to past 6 years. We took this conscious effort with the aim to limit the heterogeneity across reported estimates. We evaluated 39 incidence and 78 prevalence estimates for individual ILD disorders that were distributed globally. We noted an increase in the number of studies investigating non-IPF ILDs and more specifically autoimmune ILDs in recent years. There was a 6-fold rise in the autoimmune ILDs studies, in 2021 when compared with 2015 (18 vs 3 studies, respectively). This increase in non-IPF ILD studies may be related to the emergence of antifibrotic therapies for non-IPF fibrosing lung diseases. 91–93 Interestingly, the publication trend for IPF has remained unchanged.

This review revealed considerable inconsistencies in the incidence and prevalence estimated of the main ILD subgroups. The reported prevalence of IPF ranged from 7 to 1650 per 100 000 persons, 2 16 an approximately 800-fold difference across case definitions, despite most studies reporting IPF prevalence in the general population. The incidence and prevalence estimates reported by Zhang et al 16 were a notable outlier; this study was based on the USA veterans’ healthcare database which included mostly White patients aged over 70 years—the demographic in which IPF is most common. Aside from this study, the majority of studies reported a prevalence of IPF ranging from 7 to 42 per 100 000 persons across different case definitions. 2 17

Unlike prevalence, we found considerable inconsistencies in how the incidence of IPF is reported. An important factor is the lack of uniformity in reporting units. Half of the studies reported incidence using person-years, whereas others reported per 100 000 person-years. We were, therefore, unable to compare incidence estimates in a similar fashion to prevalence. It is also important to note that changes in diagnostic guidelines for IPF over the years may have made it more challenging to accurately estimate its burden and temporal trends. 94–96

For non-IPF subgroups, such as autoimmune ILDs, there were wide variations in prevalence estimates between countries and within different healthcare settings in the same country. Overall, the variation in prevalence and incidence estimates was even greater for non-IPF ILDs than IPF. This can be attributed to several factors. First, in clinical practice, it is common for the clinical presentation and serological autoantibody profiles to result in overlap syndromes. Autoimmune conditions can coexist and patients with occupational ILDs may also have autoimmune conditions. Such fluidity of diagnoses at a clinical level reflects the challenges in estimating non-IPF ILDs. Second, the denominator more frequently differs for non-IPF ILDs, resulting in lack of standardised reporting. Unlike IPF, for which there are published validated algorithms to identify ‘true’ cases in the general population. 18 24 97 For non-IPF ILDs, studies relied on disease registries or were conducted at single/multispecialist clinics.

Majority of the autoimmune-related ILD estimates were in RA and SSc ILD. When assessing SSc ILD prevalence, we observed a wide range (26.1% to 88.1%) 37 44 in reported estimates, but when studies were dichotomised into single-centre studies and multicentre studies, it became clear that the highest variability was contributed by single centre studies (SSc prevalence, 31.2%–88.1%). 43–46 Owing to a smaller number of studies reporting incidence, we were unable to observe whether the same challenge existed.

The prevalence of silicosis ranged from 5.6% 65 to 37% 61 in workers exposed to silica. Occupational ILD studies were conducted at a factory, in a neighbourhood with proximity to industries, a registry or multicentre settings. Therefore, lack of generalisability and applicability of findings only to certain populations contributed largely to the wide variabilities of these reported estimates. The geographical distribution of occupational ILD papers alludes to dominance of exposure related ILDs in low-income and middle-income countries in Asia and South America (42.8% were in Asia).

While historical diagnostic classification has been founded on underlying aetiology or clinical pathways, there is now a growing emphasis on disease behaviour. 98 99 Attention has focused on a subgroup of ILD patients who go on to develop a PF phenotype. IPF is the archetypal PF ILD but other ILDs such as chronic hypersensitivity pneumonitis (HP), SSc ILD can exhibit ‘IPF-like’ behaviour, including rapid decline in lung function and early mortality. 100 The epidemiology of PF ILD is particularly challenging to examine as accepted guidelines on definition and diagnosis have yet to be published The reported prevalence of PF ILDs (per 100 000 persons) was 19.4 in France and 57.8 in the USA. 88 89 The future direction of research will likely focus on PF ILD as a phenotype which transcends previously adhered-to diagnostic labels and is associated with poorer outcomes and increased mortality. 100 101

Among the 39 studies reporting ILD incidence ( online supplemental figure E6 ), most studies were categorised as medium risk (n=25/39, 64.1%). Two studies were categorised as high-risk primarily because of lack of information on ILD diagnosis and poor quality of reporting estimates (ie, descriptive statistics were not reported, were incomplete or did not include proper measures of dispersion).

Similarly, there were 78 prevalence assessments ( online supplemental figure E7 ) of which approximately 18% (n=14/78) were categorised as high risk, 64.1% (n=50/78) as medium risk and 18% (n=14/76) as low risk. Most studies assessed as high risk were studies reporting autoimmune ILDs, mainly because of ILD diagnosis, single-centre studies or small sample size. Most of the studies reporting prevalence based on large healthcare datasets or disease registries were classified as low risk.

There are several strengths of this systematic review. We have provided an assessment of the incidence and prevalence of several ILD conditions globally and have grouped ILDs based on their aetiology to allow the appraisal of incidence and/prevalence at a disease level with as much granularity as possible. This review underlines the need for standardisation of diagnostic classifications for non-IPF ILDs—the narrower estimates for IPF provide the evidence that clear and consistent diagnostic guidelines are of great clinical utility. Guidelines have recently emerged for the diagnosis of HP 102 103 which we envisage will further improve the epidemiological reporting of this important condition, although incorporation of guidelines into routine clinical practice and then into epidemiological estimates takes time. Cross-specialty guideline groups will undoubtedly improve standardisation of reporting for autoimmune driven ILDs.

It is possible that genetic differences between individuals from different ethnic backgrounds may play a role in the global variability in incidence and prevalence. For example, the MUC5B promoter polymorphism (rs35705950) is the dominant risk factor for IPF 104 and is also a key risk factor for other ILDs such as RA. 105 This gain of function polymorphism is frequent in those of European decent but almost completely absent in those of African ancestry. 106 As more research is performed unravelling the complex interplay between genetics and environment in the development of ILD, it is likely that genetic variability will be found to play an important role in the global variability of ILD.

Despite the strengths, there are limitations to this systematic review. The certainty of the ILD case definition varied across studies. It was not always possible to be sure of how reliable the ascertainment method was. However, we attempted to reflect the differences in the ILD diagnostic methods in our risk of bias quality assessment. Along with the uncertainty in the diagnosis of ILD, there were different disease definitions used across studies. Therefore, in this review due to high heterogeneity, in how ILD was defined, we were unable to perform a meta-analysis. In this review, we have only included studies reporting ILD estimates in general populations, registries or populations with a specific disorder of interest. For single-centre studies reporting incidence and/or prevalence of autoimmune or exposure ILDs, the estimates were not generalisable and this has been reflected in the risk of bias quality assessment score. This review is limited to English publications only. However, due to high volume of papers found with the study period, we are confident it has a minimal effect on the overall conclusion. 107

This review highlights the lack of uniformity in the published estimates of incidence and prevalence of ILD conditions. In addition, there is a dissimilarity in disease definitions across the studies and geographical regions. Owing to these discrepancies, we were unable to derive estimates for the global incidence and prevalence of ILD and moreover unable to confirm whether there has been a ‘true’ increase in ILD incidence over time. Revisions to diagnostic criteria have augmented the challenges of estimating incidence and prevalence of individual ILD conditions and determining the drivers for temporal trends in incidence. Improving our estimates of the burden of fibrosing lung conditions is essential for future health service planning, a need that has been heightened by the development of new antifibrotic treatments. Guidelines have recently emerged for non-IPF ILDs, we envisage this may improve the epidemiological reporting for future research. There is a fundamental need to standardise ILD diagnosis, disease definitions and reporting in order to provide the data which will drive the provision of a consistently high level of care for these patients across the globe. 108

Ethics statements

Patient consent for publication.

Not applicable.

• ↵ American Thoracic Society/European Respiratory Society international multidisciplinary consensus classification of the idiopathic interstitial pneumonias. This joint statement of the American Thoracic Society (ATS), and the European Respiratory Society (ERS) was adopted by the ATS board of directors, June 2001 and by the ERS Executive Committee, June 2001 . Am J Respir Crit Care Med 2002 ; 165 : 277 – 304 . doi:10.1164/ajrccm.165.2.ats01 OpenUrl CrossRef PubMed
• Hou Q , et al
• Wang Y-X , et al
• Sambataro D ,
• Sambataro G ,
• Pignataro F , et al
• Roubille C ,
• Micheletto L , et al
• Salliot C ,
• van der Heijde D
• Xia S , et al
• Global Burden of Disease Collaborative Network
• McKenzie JE ,
• Bossuyt PM , et al
• Weatherley N ,
• Swift AJ , et al
• Madotto F ,
• Caminati A , et al
• Raimundo K ,
• Broder MS , et al
• Myong J-P ,
• Kim H-R , et al
• Lee JS , et al
• Hopkins RB ,
• Fell C , et al
• Strongman H ,
• Kaunisto J ,
• Kelloniemi K ,
• Sutinen E , et al
• Quansah K ,
• Hassan S , et al
• Wallace L ,
• Patnaik P , et al
• Solomon JJ ,
• Olson AL , et al
• Sparks JA ,
• Cho S-K , et al
• Kang EH , et al
• Choi C-B , et al
• Kronzer VL ,
• Westerlind H ,
• Alfredsson L , et al
• Nikiphorou E , et al
• Benavidez F , et al
• Wu N , et al
• Xu D , et al
• Elfishawi MM ,
• ElArousy MH , et al
• Md Yusof MY ,
• Darby M , et al
• Duarte AC ,
• Porter JC ,
• McFarlane IM ,
• Bhamra MS , et al
• Morrisroe K ,
• Huq M , et al
• Fairley JL ,
• Proudman S , et al
• Noviani M ,
• Saffari SE ,
• Tan JL , et al
• Vandecasteele E ,
• Melsens K ,
• Vanhaecke A , et al
• Simeón-Aznar CP ,
• Fonollosa-Plá V ,
• Tolosa-Vilella C , et al
• Sánchez-Cano D ,
• Ortego-Centeno N ,
• Callejas JL , et al
• Lescoat A ,
• Huang S , et al
• Tanguy M , et al
• Janardana R ,
• Surin AK , et al
• Tomiyama F ,
• Watanabe R ,
• Ishii T , et al
• Wangkaew S ,
• Euathrongchit J ,
• Wattanawittawas P , et al
• Wang L , et al
• Dominique S ,
• Schmidt J , et al
• Leong R-L , et al
• Zhang X-W ,
• He J , et al
• Zhang X-Y , et al
• Qiu M , et al
• Kampolis CF ,
• Fragkioudaki S ,
• Mavragani CP , et al
• Manfredi A ,
• Sebastiani M ,
• Cerri S , et al
• AlQahtani BS ,
• AlHamad EH , et al
• Kvarnström M ,
• Ottosson V ,
• Nordmark B , et al
• Nilsson AM ,
• Aaltonen HL ,
• Olsson P , et al
• Poinen-Rughooputh S ,
• Rughooputh MS ,
• Guo Y , et al
• Requena-Mullor M ,
• Alarcón-Rodríguez R ,
• Parrón-Carreño T , et al
• van Tongeren M ,
• Gusso AM , et al
• Ehrlich R ,
• Fielding K , et al
• Dimitriadis C , et al
• Silanun K ,
• Chaiear N ,
• Rechaipichitkul W
• Siribaddana AD ,
• Wickramasekera K ,
• Palipana WM , et al
• Kim Y , et al
• Wickramatillake BA ,
• Fernando MA ,
• Braillard Poccard A , et al
• Ishizuka M ,
• Huang H-L ,
• Lin P-Y , et al
• Olaosebikan H ,
• Adeyeye O ,
• Akintayo R , et al
• Duchemann B ,
• Annesi-Maesano I ,
• Jacobe de Naurois C , et al
• Reiseter S ,
• Gunnarsson R ,
• Mogens Aaløkken T , et al
• Üzmezoğlu BA , et al
• Coquart N ,
• Cadelis G ,
• Tressières B , et al
• Kim D , et al
• Belbasis L ,
• Evangelou E
• Kim Y-K , et al
• Gjonbrataj J ,
• Bahn YE , et al
• Han B , et al
• DeBono NL ,
• Logar-Henderson C , et al
• Szeszenia-Dąbrowska N ,
• Świątkowska B ,
• Sobala W , et al
• Thomsen RW ,
• Flachs EM , et al
• Murofushi KN ,
• Gosho M , et al
• Tamaki T , et al
• Fernández Pérez ER ,
• Raimundo K , et al
• Yoo SH , et al
• Larrieu S ,
• Boussel L , et al
• Patnaik P ,
• Hartmann N , et al
• Flaherty KR ,
• Brown KK , et al
• Highland KB ,
• Distler O ,
• Kuwana M , et al
• Kreuter M , et al
• Collard HR ,
• Egan JJ , et al
• Rochwerg B ,
• Zhang Y , et al
• Remy-Jardin M ,
• Myers JL , et al
• Urbania T ,
• Husson G , et al
• Ryerson CJ ,
• Travis WD ,
• Costabel U ,
• Hansell DM , et al
• George PM ,
• Spagnolo P ,
• Martinez FJ ,
• Walsh SLF , et al
• Lynch DA , et al
• Seibold MA ,
• Speer MC , et al
• Ebstein E , et al
• Genomes Project C ,
• Brooks LD , et al
• Nussbaumer-Streit B ,
• Klerings I ,
• Dobrescu AI , et al
• Richeldi L , et al

Supplementary materials

Supplementary data.

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

• Data supplement 1

Twitter @DrPeter_George

Contributors RG, AM, PMG and JKQ developed the research question. RG, AM, PMG and JKQ developed the study protocol. RG developed the search strategy with input from AM and JKQ. RG screened the studies for inclusion, extracted the data from included studies and carried out quality assessment of the data. AK was the secondary reviewer for screening, data extraction and quality assessment. PMG supported with the understanding of various ILD diseases and their clinical pathways. All authors interpreted the review results. RG drafted the manuscript. All authors read, commented on and approved the manuscript.

Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Map disclaimer The inclusion of any map (including the depiction of any boundaries therein), or of any geographic or locational reference, does not imply the expression of any opinion whatsoever on the part of BMJ concerning the legal status of any country, territory, jurisdiction or area or of its authorities. Any such expression remains solely that of the relevant source and is not endorsed by BMJ. Maps are provided without any warranty of any kind, either express or implied.

Competing interests RG is a current employee of Gilead Sciences, outside the submitted work. JKQ has received grants from The Health Foundation, MRC, GSK, Bayer, BI, British Lung Foundation, IQVIA, Chiesi AZ, Insmed and Asthma UK. JKQ has received personal fees for advisory board participation or speaking fees from GlaxoSmithKline, Boehringer Ingelheim, AstraZeneca, Bayer and Insmed. PMG has received grants from the MRC, Boehringer Ingelheim and Roche Pharmaceuticals and personal fees from Boehringer Ingelheim, Roche Pharmaceuticals, Teva, Cippla, AZ and Brainomix. AK and AM have nothing to disclose.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF:

Help | Advanced Search

Computer Science > Cryptography and Security

Title: large language models for cyber security: a systematic literature review.

Abstract: The rapid advancement of Large Language Models (LLMs) has opened up new opportunities for leveraging artificial intelligence in various domains, including cybersecurity. As the volume and sophistication of cyber threats continue to grow, there is an increasing need for intelligent systems that can automatically detect vulnerabilities, analyze malware, and respond to attacks. In this survey, we conduct a comprehensive review of the literature on the application of LLMs in cybersecurity (LLM4Security). By comprehensively collecting over 30K relevant papers and systematically analyzing 127 papers from top security and software engineering venues, we aim to provide a holistic view of how LLMs are being used to solve diverse problems across the cybersecurity domain. Through our analysis, we identify several key findings. First, we observe that LLMs are being applied to a wide range of cybersecurity tasks, including vulnerability detection, malware analysis, network intrusion detection, and phishing detection. Second, we find that the datasets used for training and evaluating LLMs in these tasks are often limited in size and diversity, highlighting the need for more comprehensive and representative datasets. Third, we identify several promising techniques for adapting LLMs to specific cybersecurity domains, such as fine-tuning, transfer learning, and domain-specific pre-training. Finally, we discuss the main challenges and opportunities for future research in LLM4Security, including the need for more interpretable and explainable models, the importance of addressing data privacy and security concerns, and the potential for leveraging LLMs for proactive defense and threat hunting. Overall, our survey provides a comprehensive overview of the current state-of-the-art in LLM4Security and identifies several promising directions for future research.

Submission history

Access paper:.

• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .