literature review empirical research

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Dissertation
• What is a Literature Review? | Guide, Template, & Examples

What is a Literature Review? | Guide, Template, & Examples

Published on 22 February 2022 by Shona McCombes . Revised on 7 June 2022.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research.

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarise sources – it analyses, synthesises, and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Be assured that you'll submit flawless writing. Upload your document to correct all your mistakes.

Table of contents

Why write a literature review, examples of literature reviews, step 1: search for relevant literature, step 2: evaluate and select sources, step 3: identify themes, debates and gaps, step 4: outline your literature review’s structure, step 5: write your literature review, frequently asked questions about literature reviews, introduction.

• Quick Run-through
• Step 1 & 2

When you write a dissertation or thesis, you will have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and scholarly context
• Develop a theoretical framework and methodology for your research
• Position yourself in relation to other researchers and theorists
• Show how your dissertation addresses a gap or contributes to a debate

You might also have to write a literature review as a stand-alone assignment. In this case, the purpose is to evaluate the current state of research and demonstrate your knowledge of scholarly debates around a topic.

The content will look slightly different in each case, but the process of conducting a literature review follows the same steps. We’ve written a step-by-step guide that you can follow below.

The only proofreading tool specialized in correcting academic writing

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Correct my document today

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research objectives and questions .

If you are writing a literature review as a stand-alone assignment, you will have to choose a focus and develop a central question to direct your search. Unlike a dissertation research question, this question has to be answerable without collecting original data. You should be able to answer it based only on a review of existing publications.

Make a list of keywords

Start by creating a list of keywords related to your research topic. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list if you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can use boolean operators to help narrow down your search:

Read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

To identify the most important publications on your topic, take note of recurring citations. If the same authors, books or articles keep appearing in your reading, make sure to seek them out.

You probably won’t be able to read absolutely everything that has been written on the topic – you’ll have to evaluate which sources are most relevant to your questions.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models and methods? Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• How does the publication contribute to your understanding of the topic? What are its key insights and arguments?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible, and make sure you read any landmark studies and major theories in your field of research.

You can find out how many times an article has been cited on Google Scholar – a high citation count means the article has been influential in the field, and should certainly be included in your literature review.

The scope of your review will depend on your topic and discipline: in the sciences you usually only review recent literature, but in the humanities you might take a long historical perspective (for example, to trace how a concept has changed in meaning over time).

Remember that you can use our template to summarise and evaluate sources you’re thinking about using!

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It’s important to keep track of your sources with references to avoid plagiarism . It can be helpful to make an annotated bibliography, where you compile full reference information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

You can use our free APA Reference Generator for quick, correct, consistent citations.

To begin organising your literature review’s argument and structure, you need to understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly-visual platforms like Instagram and Snapchat – this is a gap that you could address in your own research.

There are various approaches to organising the body of a literature review. You should have a rough idea of your strategy before you start writing.

Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarising sources in order.

Try to analyse patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organise your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text, your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

If you are writing the literature review as part of your dissertation or thesis, reiterate your central problem or research question and give a brief summary of the scholarly context. You can emphasise the timeliness of the topic (“many recent studies have focused on the problem of x”) or highlight a gap in the literature (“while there has been much research on x, few researchers have taken y into consideration”).

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, make sure to follow these tips:

• Summarise and synthesise: give an overview of the main points of each source and combine them into a coherent whole.
• Analyse and interpret: don’t just paraphrase other researchers – add your own interpretations, discussing the significance of findings in relation to the literature as a whole.
• Critically evaluate: mention the strengths and weaknesses of your sources.
• Write in well-structured paragraphs: use transitions and topic sentences to draw connections, comparisons and contrasts.

In the conclusion, you should summarise the key findings you have taken from the literature and emphasise their significance.

If the literature review is part of your dissertation or thesis, reiterate how your research addresses gaps and contributes new knowledge, or discuss how you have drawn on existing theories and methods to build a framework for your research. This can lead directly into your methodology section.

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a dissertation , thesis, research paper , or proposal .

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarise yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your  dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

McCombes, S. (2022, June 07). What is a Literature Review? | Guide, Template, & Examples. Scribbr. Retrieved 14 May 2024, from https://www.scribbr.co.uk/thesis-dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, how to write a dissertation proposal | a step-by-step guide, what is a theoretical framework | a step-by-step guide, what is a research methodology | steps & tips.

Get science-backed answers as you write with Paperpal's Research feature

What is a Literature Review? How to Write It (with Examples)

A literature review is a critical analysis and synthesis of existing research on a particular topic. It provides an overview of the current state of knowledge, identifies gaps, and highlights key findings in the literature. 1 The purpose of a literature review is to situate your own research within the context of existing scholarship, demonstrating your understanding of the topic and showing how your work contributes to the ongoing conversation in the field. Learning how to write a literature review is a critical tool for successful research. Your ability to summarize and synthesize prior research pertaining to a certain topic demonstrates your grasp on the topic of study, and assists in the learning process. 

Table of Contents

• What is the purpose of literature review? 
• a. Habitat Loss and Species Extinction: 
• b. Range Shifts and Phenological Changes: 
• c. Ocean Acidification and Coral Reefs: 
• d. Adaptive Strategies and Conservation Efforts: 

How to write a good literature review 

• Choose a Topic and Define the Research Question: 
• Decide on the Scope of Your Review: 
• Select Databases for Searches: 
• Conduct Searches and Keep Track: 
• Review the Literature: 
• Organize and Write Your Literature Review: 
• How to write a literature review faster with Paperpal? 
• Frequently asked questions 

What is a literature review?

A well-conducted literature review demonstrates the researcher’s familiarity with the existing literature, establishes the context for their own research, and contributes to scholarly conversations on the topic. One of the purposes of a literature review is also to help researchers avoid duplicating previous work and ensure that their research is informed by and builds upon the existing body of knowledge.

What is the purpose of literature review?

A literature review serves several important purposes within academic and research contexts. Here are some key objectives and functions of a literature review: 2  

1. Contextualizing the Research Problem: The literature review provides a background and context for the research problem under investigation. It helps to situate the study within the existing body of knowledge. 

2. Identifying Gaps in Knowledge: By identifying gaps, contradictions, or areas requiring further research, the researcher can shape the research question and justify the significance of the study. This is crucial for ensuring that the new research contributes something novel to the field. 

Find academic papers related to your research topic faster. Try Research on Paperpal  

3. Understanding Theoretical and Conceptual Frameworks: Literature reviews help researchers gain an understanding of the theoretical and conceptual frameworks used in previous studies. This aids in the development of a theoretical framework for the current research. 

4. Providing Methodological Insights: Another purpose of literature reviews is that it allows researchers to learn about the methodologies employed in previous studies. This can help in choosing appropriate research methods for the current study and avoiding pitfalls that others may have encountered. 

5. Establishing Credibility: A well-conducted literature review demonstrates the researcher’s familiarity with existing scholarship, establishing their credibility and expertise in the field. It also helps in building a solid foundation for the new research. 

6. Informing Hypotheses or Research Questions: The literature review guides the formulation of hypotheses or research questions by highlighting relevant findings and areas of uncertainty in existing literature. 

Literature review example

Let’s delve deeper with a literature review example: Let’s say your literature review is about the impact of climate change on biodiversity. You might format your literature review into sections such as the effects of climate change on habitat loss and species extinction, phenological changes, and marine biodiversity. Each section would then summarize and analyze relevant studies in those areas, highlighting key findings and identifying gaps in the research. The review would conclude by emphasizing the need for further research on specific aspects of the relationship between climate change and biodiversity. The following literature review template provides a glimpse into the recommended literature review structure and content, demonstrating how research findings are organized around specific themes within a broader topic. 

Literature Review on Climate Change Impacts on Biodiversity:

Climate change is a global phenomenon with far-reaching consequences, including significant impacts on biodiversity. This literature review synthesizes key findings from various studies: 

a. Habitat Loss and Species Extinction:

Climate change-induced alterations in temperature and precipitation patterns contribute to habitat loss, affecting numerous species (Thomas et al., 2004). The review discusses how these changes increase the risk of extinction, particularly for species with specific habitat requirements. 

b. Range Shifts and Phenological Changes:

Observations of range shifts and changes in the timing of biological events (phenology) are documented in response to changing climatic conditions (Parmesan & Yohe, 2003). These shifts affect ecosystems and may lead to mismatches between species and their resources. 

c. Ocean Acidification and Coral Reefs:

The review explores the impact of climate change on marine biodiversity, emphasizing ocean acidification’s threat to coral reefs (Hoegh-Guldberg et al., 2007). Changes in pH levels negatively affect coral calcification, disrupting the delicate balance of marine ecosystems. 

d. Adaptive Strategies and Conservation Efforts:

Recognizing the urgency of the situation, the literature review discusses various adaptive strategies adopted by species and conservation efforts aimed at mitigating the impacts of climate change on biodiversity (Hannah et al., 2007). It emphasizes the importance of interdisciplinary approaches for effective conservation planning. 

Strengthen your literature review with factual insights. Try Research on Paperpal for free!    

Writing a literature review involves summarizing and synthesizing existing research on a particular topic. A good literature review format should include the following elements. 

Introduction: The introduction sets the stage for your literature review, providing context and introducing the main focus of your review. 

• Opening Statement: Begin with a general statement about the broader topic and its significance in the field. 
• Scope and Purpose: Clearly define the scope of your literature review. Explain the specific research question or objective you aim to address. 
• Organizational Framework: Briefly outline the structure of your literature review, indicating how you will categorize and discuss the existing research. 
• Significance of the Study: Highlight why your literature review is important and how it contributes to the understanding of the chosen topic. 
• Thesis Statement: Conclude the introduction with a concise thesis statement that outlines the main argument or perspective you will develop in the body of the literature review. 

Body: The body of the literature review is where you provide a comprehensive analysis of existing literature, grouping studies based on themes, methodologies, or other relevant criteria. 

• Organize by Theme or Concept: Group studies that share common themes, concepts, or methodologies. Discuss each theme or concept in detail, summarizing key findings and identifying gaps or areas of disagreement. 
• Critical Analysis: Evaluate the strengths and weaknesses of each study. Discuss the methodologies used, the quality of evidence, and the overall contribution of each work to the understanding of the topic. 
• Synthesis of Findings: Synthesize the information from different studies to highlight trends, patterns, or areas of consensus in the literature. 
• Identification of Gaps: Discuss any gaps or limitations in the existing research and explain how your review contributes to filling these gaps. 
• Transition between Sections: Provide smooth transitions between different themes or concepts to maintain the flow of your literature review. 

Write and Cite as you go with Paperpal Research. Start now for free.   

Conclusion: The conclusion of your literature review should summarize the main findings, highlight the contributions of the review, and suggest avenues for future research. 

• Summary of Key Findings: Recap the main findings from the literature and restate how they contribute to your research question or objective. 
• Contributions to the Field: Discuss the overall contribution of your literature review to the existing knowledge in the field. 
• Implications and Applications: Explore the practical implications of the findings and suggest how they might impact future research or practice. 
• Recommendations for Future Research: Identify areas that require further investigation and propose potential directions for future research in the field. 
• Final Thoughts: Conclude with a final reflection on the importance of your literature review and its relevance to the broader academic community. 

Conducting a literature review

Conducting a literature review is an essential step in research that involves reviewing and analyzing existing literature on a specific topic. It’s important to know how to do a literature review effectively, so here are the steps to follow: 1  

Choose a Topic and Define the Research Question:

• Select a topic that is relevant to your field of study. 
• Clearly define your research question or objective. Determine what specific aspect of the topic do you want to explore? 

Decide on the Scope of Your Review:

• Determine the timeframe for your literature review. Are you focusing on recent developments, or do you want a historical overview? 
• Consider the geographical scope. Is your review global, or are you focusing on a specific region? 
• Define the inclusion and exclusion criteria. What types of sources will you include? Are there specific types of studies or publications you will exclude? 

Select Databases for Searches:

• Identify relevant databases for your field. Examples include PubMed, IEEE Xplore, Scopus, Web of Science, and Google Scholar. 
• Consider searching in library catalogs, institutional repositories, and specialized databases related to your topic. 

Conduct Searches and Keep Track:

• Develop a systematic search strategy using keywords, Boolean operators (AND, OR, NOT), and other search techniques. 
• Record and document your search strategy for transparency and replicability. 
• Keep track of the articles, including publication details, abstracts, and links. Use citation management tools like EndNote, Zotero, or Mendeley to organize your references. 

Review the Literature:

• Evaluate the relevance and quality of each source. Consider the methodology, sample size, and results of studies. 
• Organize the literature by themes or key concepts. Identify patterns, trends, and gaps in the existing research. 
• Summarize key findings and arguments from each source. Compare and contrast different perspectives. 
• Identify areas where there is a consensus in the literature and where there are conflicting opinions. 
• Provide critical analysis and synthesis of the literature. What are the strengths and weaknesses of existing research? 

Organize and Write Your Literature Review:

• Literature review outline should be based on themes, chronological order, or methodological approaches. 
• Write a clear and coherent narrative that synthesizes the information gathered. 
• Use proper citations for each source and ensure consistency in your citation style (APA, MLA, Chicago, etc.). 
• Conclude your literature review by summarizing key findings, identifying gaps, and suggesting areas for future research. 

Whether you’re exploring a new research field or finding new angles to develop an existing topic, sifting through hundreds of papers can take more time than you have to spare. But what if you could find science-backed insights with verified citations in seconds? That’s the power of Paperpal’s new Research feature!  

How to write a literature review faster with Paperpal?

Paperpal, an AI writing assistant, integrates powerful academic search capabilities within its writing platform. With the Research feature, you get 100% factual insights, with citations backed by 250M+ verified research articles, directly within your writing interface with the option to save relevant references in your Citation Library. By eliminating the need to switch tabs to find answers to all your research questions, Paperpal saves time and helps you stay focused on your writing.   

Here’s how to use the Research feature:  

• Ask a question: Get started with a new document on paperpal.com. Click on the “Research” feature and type your question in plain English. Paperpal will scour over 250 million research articles, including conference papers and preprints, to provide you with accurate insights and citations. 
• Review and Save: Paperpal summarizes the information, while citing sources and listing relevant reads. You can quickly scan the results to identify relevant references and save these directly to your built-in citations library for later access. 
• Cite with Confidence: Paperpal makes it easy to incorporate relevant citations and references into your writing, ensuring your arguments are well-supported by credible sources. This translates to a polished, well-researched literature review. 

The literature review sample and detailed advice on writing and conducting a review will help you produce a well-structured report. But remember that a good literature review is an ongoing process, and it may be necessary to revisit and update it as your research progresses. By combining effortless research with an easy citation process, Paperpal Research streamlines the literature review process and empowers you to write faster and with more confidence. Try Paperpal Research now and see for yourself.  

Frequently asked questions

A literature review is a critical and comprehensive analysis of existing literature (published and unpublished works) on a specific topic or research question and provides a synthesis of the current state of knowledge in a particular field. A well-conducted literature review is crucial for researchers to build upon existing knowledge, avoid duplication of efforts, and contribute to the advancement of their field. It also helps researchers situate their work within a broader context and facilitates the development of a sound theoretical and conceptual framework for their studies.

Literature review is a crucial component of research writing, providing a solid background for a research paper’s investigation. The aim is to keep professionals up to date by providing an understanding of ongoing developments within a specific field, including research methods, and experimental techniques used in that field, and present that knowledge in the form of a written report. Also, the depth and breadth of the literature review emphasizes the credibility of the scholar in his or her field.  

Before writing a literature review, it’s essential to undertake several preparatory steps to ensure that your review is well-researched, organized, and focused. This includes choosing a topic of general interest to you and doing exploratory research on that topic, writing an annotated bibliography, and noting major points, especially those that relate to the position you have taken on the topic. 

Literature reviews and academic research papers are essential components of scholarly work but serve different purposes within the academic realm. 3 A literature review aims to provide a foundation for understanding the current state of research on a particular topic, identify gaps or controversies, and lay the groundwork for future research. Therefore, it draws heavily from existing academic sources, including books, journal articles, and other scholarly publications. In contrast, an academic research paper aims to present new knowledge, contribute to the academic discourse, and advance the understanding of a specific research question. Therefore, it involves a mix of existing literature (in the introduction and literature review sections) and original data or findings obtained through research methods. 

Literature reviews are essential components of academic and research papers, and various strategies can be employed to conduct them effectively. If you want to know how to write a literature review for a research paper, here are four common approaches that are often used by researchers.  Chronological Review: This strategy involves organizing the literature based on the chronological order of publication. It helps to trace the development of a topic over time, showing how ideas, theories, and research have evolved.  Thematic Review: Thematic reviews focus on identifying and analyzing themes or topics that cut across different studies. Instead of organizing the literature chronologically, it is grouped by key themes or concepts, allowing for a comprehensive exploration of various aspects of the topic.  Methodological Review: This strategy involves organizing the literature based on the research methods employed in different studies. It helps to highlight the strengths and weaknesses of various methodologies and allows the reader to evaluate the reliability and validity of the research findings.  Theoretical Review: A theoretical review examines the literature based on the theoretical frameworks used in different studies. This approach helps to identify the key theories that have been applied to the topic and assess their contributions to the understanding of the subject.  It’s important to note that these strategies are not mutually exclusive, and a literature review may combine elements of more than one approach. The choice of strategy depends on the research question, the nature of the literature available, and the goals of the review. Additionally, other strategies, such as integrative reviews or systematic reviews, may be employed depending on the specific requirements of the research.

The literature review format can vary depending on the specific publication guidelines. However, there are some common elements and structures that are often followed. Here is a general guideline for the format of a literature review:  Introduction:   Provide an overview of the topic.  Define the scope and purpose of the literature review.  State the research question or objective.  Body:   Organize the literature by themes, concepts, or chronology.  Critically analyze and evaluate each source.  Discuss the strengths and weaknesses of the studies.  Highlight any methodological limitations or biases.  Identify patterns, connections, or contradictions in the existing research.  Conclusion:   Summarize the key points discussed in the literature review.  Highlight the research gap.  Address the research question or objective stated in the introduction.  Highlight the contributions of the review and suggest directions for future research.

Both annotated bibliographies and literature reviews involve the examination of scholarly sources. While annotated bibliographies focus on individual sources with brief annotations, literature reviews provide a more in-depth, integrated, and comprehensive analysis of existing literature on a specific topic. The key differences are as follows: 

References 

• Denney, A. S., & Tewksbury, R. (2013). How to write a literature review.  Journal of criminal justice education ,  24 (2), 218-234. 
• Pan, M. L. (2016).  Preparing literature reviews: Qualitative and quantitative approaches . Taylor & Francis. 
• Cantero, C. (2019). How to write a literature review.  San José State University Writing Center . 

Paperpal is an AI writing assistant that help academics write better, faster with real-time suggestions for in-depth language and grammar correction. Trained on millions of research manuscripts enhanced by professional academic editors, Paperpal delivers human precision at machine speed.  

Try it for free or upgrade to  Paperpal Prime , which unlocks unlimited access to premium features like academic translation, paraphrasing, contextual synonyms, consistency checks and more. It’s like always having a professional academic editor by your side! Go beyond limitations and experience the future of academic writing.  Get Paperpal Prime now at just US$19 a month!

Related Reads:

• Empirical Research: A Comprehensive Guide for Academics 
• How to Write a Scientific Paper in 10 Steps 
• How Long Should a Chapter Be?
• How to Use Paperpal to Generate Emails & Cover Letters?

6 Tips for Post-Doc Researchers to Take Their Career to the Next Level

Self-plagiarism in research: what it is and how to avoid it, you may also like, how to write a high-quality conference paper, how paperpal’s research feature helps you develop and..., how paperpal is enhancing academic productivity and accelerating..., how to write a successful book chapter for..., academic editing: how to self-edit academic text with..., 4 ways paperpal encourages responsible writing with ai, what are scholarly sources and where can you..., how to write a hypothesis types and examples , measuring academic success: definition & strategies for excellence, what is academic writing: tips for students.

Research Methods

• Getting Started
• Literature Review Research
• Research Design
• Research Design By Discipline
• SAGE Research Methods
• Teaching with SAGE Research Methods

Literature Review

• What is a Literature Review?
• What is NOT a Literature Review?
• Purposes of a Literature Review
• Types of Literature Reviews
• Literature Reviews vs. Systematic Reviews
• Systematic vs. Meta-Analysis

Literature Review  is a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works.

Also, we can define a literature review as the collected body of scholarly works related to a topic:

• Summarizes and analyzes previous research relevant to a topic
• Includes scholarly books and articles published in academic journals
• Can be an specific scholarly paper or a section in a research paper

The objective of a Literature Review is to find previous published scholarly works relevant to an specific topic

• Help gather ideas or information
• Keep up to date in current trends and findings
• Help develop new questions

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Helps focus your own research questions or problems
• Discovers relationships between research studies/ideas.
• Suggests unexplored ideas or populations
• Identifies major themes, concepts, and researchers on a topic.
• Tests assumptions; may help counter preconceived ideas and remove unconscious bias.
• Identifies critical gaps, points of disagreement, or potentially flawed methodology or theoretical approaches.
• Indicates potential directions for future research.

All content in this section is from Literature Review Research from Old Dominion University 

Keep in mind the following, a literature review is NOT:

Not an essay 

Not an annotated bibliography  in which you summarize each article that you have reviewed.  A literature review goes beyond basic summarizing to focus on the critical analysis of the reviewed works and their relationship to your research question.

Not a research paper   where you select resources to support one side of an issue versus another.  A lit review should explain and consider all sides of an argument in order to avoid bias, and areas of agreement and disagreement should be highlighted.

A literature review serves several purposes. For example, it

• provides thorough knowledge of previous studies; introduces seminal works.
• helps focus one’s own research topic.
• identifies a conceptual framework for one’s own research questions or problems; indicates potential directions for future research.
• suggests previously unused or underused methodologies, designs, quantitative and qualitative strategies.
• identifies gaps in previous studies; identifies flawed methodologies and/or theoretical approaches; avoids replication of mistakes.
• helps the researcher avoid repetition of earlier research.
• suggests unexplored populations.
• determines whether past studies agree or disagree; identifies controversy in the literature.
• tests assumptions; may help counter preconceived ideas and remove unconscious bias.

As Kennedy (2007) notes*, it is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the original studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally that become part of the lore of field. In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews.

Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are several approaches to how they can be done, depending upon the type of analysis underpinning your study. Listed below are definitions of types of literature reviews:

Argumentative Review      This form examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to to make summary claims of the sort found in systematic reviews.

Integrative Review      Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication.

Historical Review      Few things rest in isolation from historical precedent. Historical reviews are focused on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review      A review does not always focus on what someone said [content], but how they said it [method of analysis]. This approach provides a framework of understanding at different levels (i.e. those of theory, substantive fields, research approaches and data collection and analysis techniques), enables researchers to draw on a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection and data analysis, and helps highlight many ethical issues which we should be aware of and consider as we go through our study.

Systematic Review      This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyse data from the studies that are included in the review. Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?"

Theoretical Review      The purpose of this form is to concretely examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review help establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

* Kennedy, Mary M. "Defining a Literature."  Educational Researcher  36 (April 2007): 139-147.

All content in this section is from The Literature Review created by Dr. Robert Larabee USC

Robinson, P. and Lowe, J. (2015),  Literature reviews vs systematic reviews.  Australian and New Zealand Journal of Public Health, 39: 103-103. doi: 10.1111/1753-6405.12393

What's in the name? The difference between a Systematic Review and a Literature Review, and why it matters . By Lynn Kysh from University of Southern California

Systematic review or meta-analysis?

A  systematic review  answers a defined research question by collecting and summarizing all empirical evidence that fits pre-specified eligibility criteria.

A  meta-analysis  is the use of statistical methods to summarize the results of these studies.

Systematic reviews, just like other research articles, can be of varying quality. They are a significant piece of work (the Centre for Reviews and Dissemination at York estimates that a team will take 9-24 months), and to be useful to other researchers and practitioners they should have:

• clearly stated objectives with pre-defined eligibility criteria for studies
• explicit, reproducible methodology
• a systematic search that attempts to identify all studies
• assessment of the validity of the findings of the included studies (e.g. risk of bias)
• systematic presentation, and synthesis, of the characteristics and findings of the included studies

Not all systematic reviews contain meta-analysis. 

Meta-analysis is the use of statistical methods to summarize the results of independent studies. By combining information from all relevant studies, meta-analysis can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review.  More information on meta-analyses can be found in  Cochrane Handbook, Chapter 9 .

A meta-analysis goes beyond critique and integration and conducts secondary statistical analysis on the outcomes of similar studies.  It is a systematic review that uses quantitative methods to synthesize and summarize the results.

An advantage of a meta-analysis is the ability to be completely objective in evaluating research findings.  Not all topics, however, have sufficient research evidence to allow a meta-analysis to be conducted.  In that case, an integrative review is an appropriate strategy. 

Some of the content in this section is from Systematic reviews and meta-analyses: step by step guide created by Kate McAllister.

• << Previous: Getting Started
• Next: Research Design >>
• Last Updated: Aug 21, 2023 4:07 PM
• URL: https://guides.lib.udel.edu/researchmethods

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• CBE Life Sci Educ
• v.21(3); Fall 2022

Literature Reviews, Theoretical Frameworks, and Conceptual Frameworks: An Introduction for New Biology Education Researchers

Julie a. luft.

† Department of Mathematics, Social Studies, and Science Education, Mary Frances Early College of Education, University of Georgia, Athens, GA 30602-7124

Sophia Jeong

‡ Department of Teaching & Learning, College of Education & Human Ecology, Ohio State University, Columbus, OH 43210

Robert Idsardi

§ Department of Biology, Eastern Washington University, Cheney, WA 99004

Grant Gardner

∥ Department of Biology, Middle Tennessee State University, Murfreesboro, TN 37132

Associated Data

To frame their work, biology education researchers need to consider the role of literature reviews, theoretical frameworks, and conceptual frameworks as critical elements of the research and writing process. However, these elements can be confusing for scholars new to education research. This Research Methods article is designed to provide an overview of each of these elements and delineate the purpose of each in the educational research process. We describe what biology education researchers should consider as they conduct literature reviews, identify theoretical frameworks, and construct conceptual frameworks. Clarifying these different components of educational research studies can be helpful to new biology education researchers and the biology education research community at large in situating their work in the broader scholarly literature.

INTRODUCTION

Discipline-based education research (DBER) involves the purposeful and situated study of teaching and learning in specific disciplinary areas ( Singer et al. , 2012 ). Studies in DBER are guided by research questions that reflect disciplines’ priorities and worldviews. Researchers can use quantitative data, qualitative data, or both to answer these research questions through a variety of methodological traditions. Across all methodologies, there are different methods associated with planning and conducting educational research studies that include the use of surveys, interviews, observations, artifacts, or instruments. Ensuring the coherence of these elements to the discipline’s perspective also involves situating the work in the broader scholarly literature. The tools for doing this include literature reviews, theoretical frameworks, and conceptual frameworks. However, the purpose and function of each of these elements is often confusing to new education researchers. The goal of this article is to introduce new biology education researchers to these three important elements important in DBER scholarship and the broader educational literature.

The first element we discuss is a review of research (literature reviews), which highlights the need for a specific research question, study problem, or topic of investigation. Literature reviews situate the relevance of the study within a topic and a field. The process may seem familiar to science researchers entering DBER fields, but new researchers may still struggle in conducting the review. Booth et al. (2016b) highlight some of the challenges novice education researchers face when conducting a review of literature. They point out that novice researchers struggle in deciding how to focus the review, determining the scope of articles needed in the review, and knowing how to be critical of the articles in the review. Overcoming these challenges (and others) can help novice researchers construct a sound literature review that can inform the design of the study and help ensure the work makes a contribution to the field.

The second and third highlighted elements are theoretical and conceptual frameworks. These guide biology education research (BER) studies, and may be less familiar to science researchers. These elements are important in shaping the construction of new knowledge. Theoretical frameworks offer a way to explain and interpret the studied phenomenon, while conceptual frameworks clarify assumptions about the studied phenomenon. Despite the importance of these constructs in educational research, biology educational researchers have noted the limited use of theoretical or conceptual frameworks in published work ( DeHaan, 2011 ; Dirks, 2011 ; Lo et al. , 2019 ). In reviewing articles published in CBE—Life Sciences Education ( LSE ) between 2015 and 2019, we found that fewer than 25% of the research articles had a theoretical or conceptual framework (see the Supplemental Information), and at times there was an inconsistent use of theoretical and conceptual frameworks. Clearly, these frameworks are challenging for published biology education researchers, which suggests the importance of providing some initial guidance to new biology education researchers.

Fortunately, educational researchers have increased their explicit use of these frameworks over time, and this is influencing educational research in science, technology, engineering, and mathematics (STEM) fields. For instance, a quick search for theoretical or conceptual frameworks in the abstracts of articles in Educational Research Complete (a common database for educational research) in STEM fields demonstrates a dramatic change over the last 20 years: from only 778 articles published between 2000 and 2010 to 5703 articles published between 2010 and 2020, a more than sevenfold increase. Greater recognition of the importance of these frameworks is contributing to DBER authors being more explicit about such frameworks in their studies.

Collectively, literature reviews, theoretical frameworks, and conceptual frameworks work to guide methodological decisions and the elucidation of important findings. Each offers a different perspective on the problem of study and is an essential element in all forms of educational research. As new researchers seek to learn about these elements, they will find different resources, a variety of perspectives, and many suggestions about the construction and use of these elements. The wide range of available information can overwhelm the new researcher who just wants to learn the distinction between these elements or how to craft them adequately.

Our goal in writing this paper is not to offer specific advice about how to write these sections in scholarly work. Instead, we wanted to introduce these elements to those who are new to BER and who are interested in better distinguishing one from the other. In this paper, we share the purpose of each element in BER scholarship, along with important points on its construction. We also provide references for additional resources that may be beneficial to better understanding each element. Table 1 summarizes the key distinctions among these elements.

Comparison of literature reviews, theoretical frameworks, and conceptual reviews

This article is written for the new biology education researcher who is just learning about these different elements or for scientists looking to become more involved in BER. It is a result of our own work as science education and biology education researchers, whether as graduate students and postdoctoral scholars or newly hired and established faculty members. This is the article we wish had been available as we started to learn about these elements or discussed them with new educational researchers in biology.

LITERATURE REVIEWS

Purpose of a literature review.

A literature review is foundational to any research study in education or science. In education, a well-conceptualized and well-executed review provides a summary of the research that has already been done on a specific topic and identifies questions that remain to be answered, thus illustrating the current research project’s potential contribution to the field and the reasoning behind the methodological approach selected for the study ( Maxwell, 2012 ). BER is an evolving disciplinary area that is redefining areas of conceptual emphasis as well as orientations toward teaching and learning (e.g., Labov et al. , 2010 ; American Association for the Advancement of Science, 2011 ; Nehm, 2019 ). As a result, building comprehensive, critical, purposeful, and concise literature reviews can be a challenge for new biology education researchers.

Building Literature Reviews

There are different ways to approach and construct a literature review. Booth et al. (2016a) provide an overview that includes, for example, scoping reviews, which are focused only on notable studies and use a basic method of analysis, and integrative reviews, which are the result of exhaustive literature searches across different genres. Underlying each of these different review processes are attention to the s earch process, a ppraisa l of articles, s ynthesis of the literature, and a nalysis: SALSA ( Booth et al. , 2016a ). This useful acronym can help the researcher focus on the process while building a specific type of review.

However, new educational researchers often have questions about literature reviews that are foundational to SALSA or other approaches. Common questions concern determining which literature pertains to the topic of study or the role of the literature review in the design of the study. This section addresses such questions broadly while providing general guidance for writing a narrative literature review that evaluates the most pertinent studies.

The literature review process should begin before the research is conducted. As Boote and Beile (2005 , p. 3) suggested, researchers should be “scholars before researchers.” They point out that having a good working knowledge of the proposed topic helps illuminate avenues of study. Some subject areas have a deep body of work to read and reflect upon, providing a strong foundation for developing the research question(s). For instance, the teaching and learning of evolution is an area of long-standing interest in the BER community, generating many studies (e.g., Perry et al. , 2008 ; Barnes and Brownell, 2016 ) and reviews of research (e.g., Sickel and Friedrichsen, 2013 ; Ziadie and Andrews, 2018 ). Emerging areas of BER include the affective domain, issues of transfer, and metacognition ( Singer et al. , 2012 ). Many studies in these areas are transdisciplinary and not always specific to biology education (e.g., Rodrigo-Peiris et al. , 2018 ; Kolpikova et al. , 2019 ). These newer areas may require reading outside BER; fortunately, summaries of some of these topics can be found in the Current Insights section of the LSE website.

In focusing on a specific problem within a broader research strand, a new researcher will likely need to examine research outside BER. Depending upon the area of study, the expanded reading list might involve a mix of BER, DBER, and educational research studies. Determining the scope of the reading is not always straightforward. A simple way to focus one’s reading is to create a “summary phrase” or “research nugget,” which is a very brief descriptive statement about the study. It should focus on the essence of the study, for example, “first-year nonmajor students’ understanding of evolution,” “metacognitive prompts to enhance learning during biochemistry,” or “instructors’ inquiry-based instructional practices after professional development programming.” This type of phrase should help a new researcher identify two or more areas to review that pertain to the study. Focusing on recent research in the last 5 years is a good first step. Additional studies can be identified by reading relevant works referenced in those articles. It is also important to read seminal studies that are more than 5 years old. Reading a range of studies should give the researcher the necessary command of the subject in order to suggest a research question.

Given that the research question(s) arise from the literature review, the review should also substantiate the selected methodological approach. The review and research question(s) guide the researcher in determining how to collect and analyze data. Often the methodological approach used in a study is selected to contribute knowledge that expands upon what has been published previously about the topic (see Institute of Education Sciences and National Science Foundation, 2013 ). An emerging topic of study may need an exploratory approach that allows for a description of the phenomenon and development of a potential theory. This could, but not necessarily, require a methodological approach that uses interviews, observations, surveys, or other instruments. An extensively studied topic may call for the additional understanding of specific factors or variables; this type of study would be well suited to a verification or a causal research design. These could entail a methodological approach that uses valid and reliable instruments, observations, or interviews to determine an effect in the studied event. In either of these examples, the researcher(s) may use a qualitative, quantitative, or mixed methods methodological approach.

Even with a good research question, there is still more reading to be done. The complexity and focus of the research question dictates the depth and breadth of the literature to be examined. Questions that connect multiple topics can require broad literature reviews. For instance, a study that explores the impact of a biology faculty learning community on the inquiry instruction of faculty could have the following review areas: learning communities among biology faculty, inquiry instruction among biology faculty, and inquiry instruction among biology faculty as a result of professional learning. Biology education researchers need to consider whether their literature review requires studies from different disciplines within or outside DBER. For the example given, it would be fruitful to look at research focused on learning communities with faculty in STEM fields or in general education fields that result in instructional change. It is important not to be too narrow or too broad when reading. When the conclusions of articles start to sound similar or no new insights are gained, the researcher likely has a good foundation for a literature review. This level of reading should allow the researcher to demonstrate a mastery in understanding the researched topic, explain the suitability of the proposed research approach, and point to the need for the refined research question(s).

The literature review should include the researcher’s evaluation and critique of the selected studies. A researcher may have a large collection of studies, but not all of the studies will follow standards important in the reporting of empirical work in the social sciences. The American Educational Research Association ( Duran et al. , 2006 ), for example, offers a general discussion about standards for such work: an adequate review of research informing the study, the existence of sound and appropriate data collection and analysis methods, and appropriate conclusions that do not overstep or underexplore the analyzed data. The Institute of Education Sciences and National Science Foundation (2013) also offer Common Guidelines for Education Research and Development that can be used to evaluate collected studies.

Because not all journals adhere to such standards, it is important that a researcher review each study to determine the quality of published research, per the guidelines suggested earlier. In some instances, the research may be fatally flawed. Examples of such flaws include data that do not pertain to the question, a lack of discussion about the data collection, poorly constructed instruments, or an inadequate analysis. These types of errors result in studies that are incomplete, error-laden, or inaccurate and should be excluded from the review. Most studies have limitations, and the author(s) often make them explicit. For instance, there may be an instructor effect, recognized bias in the analysis, or issues with the sample population. Limitations are usually addressed by the research team in some way to ensure a sound and acceptable research process. Occasionally, the limitations associated with the study can be significant and not addressed adequately, which leaves a consequential decision in the hands of the researcher. Providing critiques of studies in the literature review process gives the reader confidence that the researcher has carefully examined relevant work in preparation for the study and, ultimately, the manuscript.

A solid literature review clearly anchors the proposed study in the field and connects the research question(s), the methodological approach, and the discussion. Reviewing extant research leads to research questions that will contribute to what is known in the field. By summarizing what is known, the literature review points to what needs to be known, which in turn guides decisions about methodology. Finally, notable findings of the new study are discussed in reference to those described in the literature review.

Within published BER studies, literature reviews can be placed in different locations in an article. When included in the introductory section of the study, the first few paragraphs of the manuscript set the stage, with the literature review following the opening paragraphs. Cooper et al. (2019) illustrate this approach in their study of course-based undergraduate research experiences (CUREs). An introduction discussing the potential of CURES is followed by an analysis of the existing literature relevant to the design of CUREs that allows for novel student discoveries. Within this review, the authors point out contradictory findings among research on novel student discoveries. This clarifies the need for their study, which is described and highlighted through specific research aims.

A literature reviews can also make up a separate section in a paper. For example, the introduction to Todd et al. (2019) illustrates the need for their research topic by highlighting the potential of learning progressions (LPs) and suggesting that LPs may help mitigate learning loss in genetics. At the end of the introduction, the authors state their specific research questions. The review of literature following this opening section comprises two subsections. One focuses on learning loss in general and examines a variety of studies and meta-analyses from the disciplines of medical education, mathematics, and reading. The second section focuses specifically on LPs in genetics and highlights student learning in the midst of LPs. These separate reviews provide insights into the stated research question.

Suggestions and Advice

A well-conceptualized, comprehensive, and critical literature review reveals the understanding of the topic that the researcher brings to the study. Literature reviews should not be so big that there is no clear area of focus; nor should they be so narrow that no real research question arises. The task for a researcher is to craft an efficient literature review that offers a critical analysis of published work, articulates the need for the study, guides the methodological approach to the topic of study, and provides an adequate foundation for the discussion of the findings.

In our own writing of literature reviews, there are often many drafts. An early draft may seem well suited to the study because the need for and approach to the study are well described. However, as the results of the study are analyzed and findings begin to emerge, the existing literature review may be inadequate and need revision. The need for an expanded discussion about the research area can result in the inclusion of new studies that support the explanation of a potential finding. The literature review may also prove to be too broad. Refocusing on a specific area allows for more contemplation of a finding.

It should be noted that there are different types of literature reviews, and many books and articles have been written about the different ways to embark on these types of reviews. Among these different resources, the following may be helpful in considering how to refine the review process for scholarly journals:

• Booth, A., Sutton, A., & Papaioannou, D. (2016a). Systemic approaches to a successful literature review (2nd ed.). Los Angeles, CA: Sage. This book addresses different types of literature reviews and offers important suggestions pertaining to defining the scope of the literature review and assessing extant studies.
• Booth, W. C., Colomb, G. G., Williams, J. M., Bizup, J., & Fitzgerald, W. T. (2016b). The craft of research (4th ed.). Chicago: University of Chicago Press. This book can help the novice consider how to make the case for an area of study. While this book is not specifically about literature reviews, it offers suggestions about making the case for your study.
• Galvan, J. L., & Galvan, M. C. (2017). Writing literature reviews: A guide for students of the social and behavioral sciences (7th ed.). Routledge. This book offers guidance on writing different types of literature reviews. For the novice researcher, there are useful suggestions for creating coherent literature reviews.

THEORETICAL FRAMEWORKS

Purpose of theoretical frameworks.

As new education researchers may be less familiar with theoretical frameworks than with literature reviews, this discussion begins with an analogy. Envision a biologist, chemist, and physicist examining together the dramatic effect of a fog tsunami over the ocean. A biologist gazing at this phenomenon may be concerned with the effect of fog on various species. A chemist may be interested in the chemical composition of the fog as water vapor condenses around bits of salt. A physicist may be focused on the refraction of light to make fog appear to be “sitting” above the ocean. While observing the same “objective event,” the scientists are operating under different theoretical frameworks that provide a particular perspective or “lens” for the interpretation of the phenomenon. Each of these scientists brings specialized knowledge, experiences, and values to this phenomenon, and these influence the interpretation of the phenomenon. The scientists’ theoretical frameworks influence how they design and carry out their studies and interpret their data.

Within an educational study, a theoretical framework helps to explain a phenomenon through a particular lens and challenges and extends existing knowledge within the limitations of that lens. Theoretical frameworks are explicitly stated by an educational researcher in the paper’s framework, theory, or relevant literature section. The framework shapes the types of questions asked, guides the method by which data are collected and analyzed, and informs the discussion of the results of the study. It also reveals the researcher’s subjectivities, for example, values, social experience, and viewpoint ( Allen, 2017 ). It is essential that a novice researcher learn to explicitly state a theoretical framework, because all research questions are being asked from the researcher’s implicit or explicit assumptions of a phenomenon of interest ( Schwandt, 2000 ).

Selecting Theoretical Frameworks

Theoretical frameworks are one of the most contemplated elements in our work in educational research. In this section, we share three important considerations for new scholars selecting a theoretical framework.

The first step in identifying a theoretical framework involves reflecting on the phenomenon within the study and the assumptions aligned with the phenomenon. The phenomenon involves the studied event. There are many possibilities, for example, student learning, instructional approach, or group organization. A researcher holds assumptions about how the phenomenon will be effected, influenced, changed, or portrayed. It is ultimately the researcher’s assumption(s) about the phenomenon that aligns with a theoretical framework. An example can help illustrate how a researcher’s reflection on the phenomenon and acknowledgment of assumptions can result in the identification of a theoretical framework.

In our example, a biology education researcher may be interested in exploring how students’ learning of difficult biological concepts can be supported by the interactions of group members. The phenomenon of interest is the interactions among the peers, and the researcher assumes that more knowledgeable students are important in supporting the learning of the group. As a result, the researcher may draw on Vygotsky’s (1978) sociocultural theory of learning and development that is focused on the phenomenon of student learning in a social setting. This theory posits the critical nature of interactions among students and between students and teachers in the process of building knowledge. A researcher drawing upon this framework holds the assumption that learning is a dynamic social process involving questions and explanations among students in the classroom and that more knowledgeable peers play an important part in the process of building conceptual knowledge.

It is important to state at this point that there are many different theoretical frameworks. Some frameworks focus on learning and knowing, while other theoretical frameworks focus on equity, empowerment, or discourse. Some frameworks are well articulated, and others are still being refined. For a new researcher, it can be challenging to find a theoretical framework. Two of the best ways to look for theoretical frameworks is through published works that highlight different frameworks.

When a theoretical framework is selected, it should clearly connect to all parts of the study. The framework should augment the study by adding a perspective that provides greater insights into the phenomenon. It should clearly align with the studies described in the literature review. For instance, a framework focused on learning would correspond to research that reported different learning outcomes for similar studies. The methods for data collection and analysis should also correspond to the framework. For instance, a study about instructional interventions could use a theoretical framework concerned with learning and could collect data about the effect of the intervention on what is learned. When the data are analyzed, the theoretical framework should provide added meaning to the findings, and the findings should align with the theoretical framework.

A study by Jensen and Lawson (2011) provides an example of how a theoretical framework connects different parts of the study. They compared undergraduate biology students in heterogeneous and homogeneous groups over the course of a semester. Jensen and Lawson (2011) assumed that learning involved collaboration and more knowledgeable peers, which made Vygotsky’s (1978) theory a good fit for their study. They predicted that students in heterogeneous groups would experience greater improvement in their reasoning abilities and science achievements with much of the learning guided by the more knowledgeable peers.

In the enactment of the study, they collected data about the instruction in traditional and inquiry-oriented classes, while the students worked in homogeneous or heterogeneous groups. To determine the effect of working in groups, the authors also measured students’ reasoning abilities and achievement. Each data-collection and analysis decision connected to understanding the influence of collaborative work.

Their findings highlighted aspects of Vygotsky’s (1978) theory of learning. One finding, for instance, posited that inquiry instruction, as a whole, resulted in reasoning and achievement gains. This links to Vygotsky (1978) , because inquiry instruction involves interactions among group members. A more nuanced finding was that group composition had a conditional effect. Heterogeneous groups performed better with more traditional and didactic instruction, regardless of the reasoning ability of the group members. Homogeneous groups worked better during interaction-rich activities for students with low reasoning ability. The authors attributed the variation to the different types of helping behaviors of students. High-performing students provided the answers, while students with low reasoning ability had to work collectively through the material. In terms of Vygotsky (1978) , this finding provided new insights into the learning context in which productive interactions can occur for students.

Another consideration in the selection and use of a theoretical framework pertains to its orientation to the study. This can result in the theoretical framework prioritizing individuals, institutions, and/or policies ( Anfara and Mertz, 2014 ). Frameworks that connect to individuals, for instance, could contribute to understanding their actions, learning, or knowledge. Institutional frameworks, on the other hand, offer insights into how institutions, organizations, or groups can influence individuals or materials. Policy theories provide ways to understand how national or local policies can dictate an emphasis on outcomes or instructional design. These different types of frameworks highlight different aspects in an educational setting, which influences the design of the study and the collection of data. In addition, these different frameworks offer a way to make sense of the data. Aligning the data collection and analysis with the framework ensures that a study is coherent and can contribute to the field.

New understandings emerge when different theoretical frameworks are used. For instance, Ebert-May et al. (2015) prioritized the individual level within conceptual change theory (see Posner et al. , 1982 ). In this theory, an individual’s knowledge changes when it no longer fits the phenomenon. Ebert-May et al. (2015) designed a professional development program challenging biology postdoctoral scholars’ existing conceptions of teaching. The authors reported that the biology postdoctoral scholars’ teaching practices became more student-centered as they were challenged to explain their instructional decision making. According to the theory, the biology postdoctoral scholars’ dissatisfaction in their descriptions of teaching and learning initiated change in their knowledge and instruction. These results reveal how conceptual change theory can explain the learning of participants and guide the design of professional development programming.

The communities of practice (CoP) theoretical framework ( Lave, 1988 ; Wenger, 1998 ) prioritizes the institutional level , suggesting that learning occurs when individuals learn from and contribute to the communities in which they reside. Grounded in the assumption of community learning, the literature on CoP suggests that, as individuals interact regularly with the other members of their group, they learn about the rules, roles, and goals of the community ( Allee, 2000 ). A study conducted by Gehrke and Kezar (2017) used the CoP framework to understand organizational change by examining the involvement of individual faculty engaged in a cross-institutional CoP focused on changing the instructional practice of faculty at each institution. In the CoP, faculty members were involved in enhancing instructional materials within their department, which aligned with an overarching goal of instituting instruction that embraced active learning. Not surprisingly, Gehrke and Kezar (2017) revealed that faculty who perceived the community culture as important in their work cultivated institutional change. Furthermore, they found that institutional change was sustained when key leaders served as mentors and provided support for faculty, and as faculty themselves developed into leaders. This study reveals the complexity of individual roles in a COP in order to support institutional instructional change.

It is important to explicitly state the theoretical framework used in a study, but elucidating a theoretical framework can be challenging for a new educational researcher. The literature review can help to identify an applicable theoretical framework. Focal areas of the review or central terms often connect to assumptions and assertions associated with the framework that pertain to the phenomenon of interest. Another way to identify a theoretical framework is self-reflection by the researcher on personal beliefs and understandings about the nature of knowledge the researcher brings to the study ( Lysaght, 2011 ). In stating one’s beliefs and understandings related to the study (e.g., students construct their knowledge, instructional materials support learning), an orientation becomes evident that will suggest a particular theoretical framework. Theoretical frameworks are not arbitrary , but purposefully selected.

With experience, a researcher may find expanded roles for theoretical frameworks. Researchers may revise an existing framework that has limited explanatory power, or they may decide there is a need to develop a new theoretical framework. These frameworks can emerge from a current study or the need to explain a phenomenon in a new way. Researchers may also find that multiple theoretical frameworks are necessary to frame and explore a problem, as different frameworks can provide different insights into a problem.

Finally, it is important to recognize that choosing “x” theoretical framework does not necessarily mean a researcher chooses “y” methodology and so on, nor is there a clear-cut, linear process in selecting a theoretical framework for one’s study. In part, the nonlinear process of identifying a theoretical framework is what makes understanding and using theoretical frameworks challenging. For the novice scholar, contemplating and understanding theoretical frameworks is essential. Fortunately, there are articles and books that can help:

• Creswell, J. W. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). Los Angeles, CA: Sage. This book provides an overview of theoretical frameworks in general educational research.
• Ding, L. (2019). Theoretical perspectives of quantitative physics education research. Physical Review Physics Education Research , 15 (2), 020101-1–020101-13. This paper illustrates how a DBER field can use theoretical frameworks.
• Nehm, R. (2019). Biology education research: Building integrative frameworks for teaching and learning about living systems. Disciplinary and Interdisciplinary Science Education Research , 1 , ar15. https://doi.org/10.1186/s43031-019-0017-6 . This paper articulates the need for studies in BER to explicitly state theoretical frameworks and provides examples of potential studies.
• Patton, M. Q. (2015). Qualitative research & evaluation methods: Integrating theory and practice . Sage. This book also provides an overview of theoretical frameworks, but for both research and evaluation.

CONCEPTUAL FRAMEWORKS

Purpose of a conceptual framework.

A conceptual framework is a description of the way a researcher understands the factors and/or variables that are involved in the study and their relationships to one another. The purpose of a conceptual framework is to articulate the concepts under study using relevant literature ( Rocco and Plakhotnik, 2009 ) and to clarify the presumed relationships among those concepts ( Rocco and Plakhotnik, 2009 ; Anfara and Mertz, 2014 ). Conceptual frameworks are different from theoretical frameworks in both their breadth and grounding in established findings. Whereas a theoretical framework articulates the lens through which a researcher views the work, the conceptual framework is often more mechanistic and malleable.

Conceptual frameworks are broader, encompassing both established theories (i.e., theoretical frameworks) and the researchers’ own emergent ideas. Emergent ideas, for example, may be rooted in informal and/or unpublished observations from experience. These emergent ideas would not be considered a “theory” if they are not yet tested, supported by systematically collected evidence, and peer reviewed. However, they do still play an important role in the way researchers approach their studies. The conceptual framework allows authors to clearly describe their emergent ideas so that connections among ideas in the study and the significance of the study are apparent to readers.

Constructing Conceptual Frameworks

Including a conceptual framework in a research study is important, but researchers often opt to include either a conceptual or a theoretical framework. Either may be adequate, but both provide greater insight into the research approach. For instance, a research team plans to test a novel component of an existing theory. In their study, they describe the existing theoretical framework that informs their work and then present their own conceptual framework. Within this conceptual framework, specific topics portray emergent ideas that are related to the theory. Describing both frameworks allows readers to better understand the researchers’ assumptions, orientations, and understanding of concepts being investigated. For example, Connolly et al. (2018) included a conceptual framework that described how they applied a theoretical framework of social cognitive career theory (SCCT) to their study on teaching programs for doctoral students. In their conceptual framework, the authors described SCCT, explained how it applied to the investigation, and drew upon results from previous studies to justify the proposed connections between the theory and their emergent ideas.

In some cases, authors may be able to sufficiently describe their conceptualization of the phenomenon under study in an introduction alone, without a separate conceptual framework section. However, incomplete descriptions of how the researchers conceptualize the components of the study may limit the significance of the study by making the research less intelligible to readers. This is especially problematic when studying topics in which researchers use the same terms for different constructs or different terms for similar and overlapping constructs (e.g., inquiry, teacher beliefs, pedagogical content knowledge, or active learning). Authors must describe their conceptualization of a construct if the research is to be understandable and useful.

There are some key areas to consider regarding the inclusion of a conceptual framework in a study. To begin with, it is important to recognize that conceptual frameworks are constructed by the researchers conducting the study ( Rocco and Plakhotnik, 2009 ; Maxwell, 2012 ). This is different from theoretical frameworks that are often taken from established literature. Researchers should bring together ideas from the literature, but they may be influenced by their own experiences as a student and/or instructor, the shared experiences of others, or thought experiments as they construct a description, model, or representation of their understanding of the phenomenon under study. This is an exercise in intellectual organization and clarity that often considers what is learned, known, and experienced. The conceptual framework makes these constructs explicitly visible to readers, who may have different understandings of the phenomenon based on their prior knowledge and experience. There is no single method to go about this intellectual work.

Reeves et al. (2016) is an example of an article that proposed a conceptual framework about graduate teaching assistant professional development evaluation and research. The authors used existing literature to create a novel framework that filled a gap in current research and practice related to the training of graduate teaching assistants. This conceptual framework can guide the systematic collection of data by other researchers because the framework describes the relationships among various factors that influence teaching and learning. The Reeves et al. (2016) conceptual framework may be modified as additional data are collected and analyzed by other researchers. This is not uncommon, as conceptual frameworks can serve as catalysts for concerted research efforts that systematically explore a phenomenon (e.g., Reynolds et al. , 2012 ; Brownell and Kloser, 2015 ).

Sabel et al. (2017) used a conceptual framework in their exploration of how scaffolds, an external factor, interact with internal factors to support student learning. Their conceptual framework integrated principles from two theoretical frameworks, self-regulated learning and metacognition, to illustrate how the research team conceptualized students’ use of scaffolds in their learning ( Figure 1 ). Sabel et al. (2017) created this model using their interpretations of these two frameworks in the context of their teaching.

Conceptual framework from Sabel et al. (2017) .

A conceptual framework should describe the relationship among components of the investigation ( Anfara and Mertz, 2014 ). These relationships should guide the researcher’s methods of approaching the study ( Miles et al. , 2014 ) and inform both the data to be collected and how those data should be analyzed. Explicitly describing the connections among the ideas allows the researcher to justify the importance of the study and the rigor of the research design. Just as importantly, these frameworks help readers understand why certain components of a system were not explored in the study. This is a challenge in education research, which is rooted in complex environments with many variables that are difficult to control.

For example, Sabel et al. (2017) stated: “Scaffolds, such as enhanced answer keys and reflection questions, can help students and instructors bridge the external and internal factors and support learning” (p. 3). They connected the scaffolds in the study to the three dimensions of metacognition and the eventual transformation of existing ideas into new or revised ideas. Their framework provides a rationale for focusing on how students use two different scaffolds, and not on other factors that may influence a student’s success (self-efficacy, use of active learning, exam format, etc.).

In constructing conceptual frameworks, researchers should address needed areas of study and/or contradictions discovered in literature reviews. By attending to these areas, researchers can strengthen their arguments for the importance of a study. For instance, conceptual frameworks can address how the current study will fill gaps in the research, resolve contradictions in existing literature, or suggest a new area of study. While a literature review describes what is known and not known about the phenomenon, the conceptual framework leverages these gaps in describing the current study ( Maxwell, 2012 ). In the example of Sabel et al. (2017) , the authors indicated there was a gap in the literature regarding how scaffolds engage students in metacognition to promote learning in large classes. Their study helps fill that gap by describing how scaffolds can support students in the three dimensions of metacognition: intelligibility, plausibility, and wide applicability. In another example, Lane (2016) integrated research from science identity, the ethic of care, the sense of belonging, and an expertise model of student success to form a conceptual framework that addressed the critiques of other frameworks. In a more recent example, Sbeglia et al. (2021) illustrated how a conceptual framework influences the methodological choices and inferences in studies by educational researchers.

Sometimes researchers draw upon the conceptual frameworks of other researchers. When a researcher’s conceptual framework closely aligns with an existing framework, the discussion may be brief. For example, Ghee et al. (2016) referred to portions of SCCT as their conceptual framework to explain the significance of their work on students’ self-efficacy and career interests. Because the authors’ conceptualization of this phenomenon aligned with a previously described framework, they briefly mentioned the conceptual framework and provided additional citations that provided more detail for the readers.

Within both the BER and the broader DBER communities, conceptual frameworks have been used to describe different constructs. For example, some researchers have used the term “conceptual framework” to describe students’ conceptual understandings of a biological phenomenon. This is distinct from a researcher’s conceptual framework of the educational phenomenon under investigation, which may also need to be explicitly described in the article. Other studies have presented a research logic model or flowchart of the research design as a conceptual framework. These constructions can be quite valuable in helping readers understand the data-collection and analysis process. However, a model depicting the study design does not serve the same role as a conceptual framework. Researchers need to avoid conflating these constructs by differentiating the researchers’ conceptual framework that guides the study from the research design, when applicable.

Explicitly describing conceptual frameworks is essential in depicting the focus of the study. We have found that being explicit in a conceptual framework means using accepted terminology, referencing prior work, and clearly noting connections between terms. This description can also highlight gaps in the literature or suggest potential contributions to the field of study. A well-elucidated conceptual framework can suggest additional studies that may be warranted. This can also spur other researchers to consider how they would approach the examination of a phenomenon and could result in a revised conceptual framework.

It can be challenging to create conceptual frameworks, but they are important. Below are two resources that could be helpful in constructing and presenting conceptual frameworks in educational research:

• Maxwell, J. A. (2012). Qualitative research design: An interactive approach (3rd ed.). Los Angeles, CA: Sage. Chapter 3 in this book describes how to construct conceptual frameworks.
• Ravitch, S. M., & Riggan, M. (2016). Reason & rigor: How conceptual frameworks guide research . Los Angeles, CA: Sage. This book explains how conceptual frameworks guide the research questions, data collection, data analyses, and interpretation of results.

CONCLUDING THOUGHTS

Literature reviews, theoretical frameworks, and conceptual frameworks are all important in DBER and BER. Robust literature reviews reinforce the importance of a study. Theoretical frameworks connect the study to the base of knowledge in educational theory and specify the researcher’s assumptions. Conceptual frameworks allow researchers to explicitly describe their conceptualization of the relationships among the components of the phenomenon under study. Table 1 provides a general overview of these components in order to assist biology education researchers in thinking about these elements.

It is important to emphasize that these different elements are intertwined. When these elements are aligned and complement one another, the study is coherent, and the study findings contribute to knowledge in the field. When literature reviews, theoretical frameworks, and conceptual frameworks are disconnected from one another, the study suffers. The point of the study is lost, suggested findings are unsupported, or important conclusions are invisible to the researcher. In addition, this misalignment may be costly in terms of time and money.

Conducting a literature review, selecting a theoretical framework, and building a conceptual framework are some of the most difficult elements of a research study. It takes time to understand the relevant research, identify a theoretical framework that provides important insights into the study, and formulate a conceptual framework that organizes the finding. In the research process, there is often a constant back and forth among these elements as the study evolves. With an ongoing refinement of the review of literature, clarification of the theoretical framework, and articulation of a conceptual framework, a sound study can emerge that makes a contribution to the field. This is the goal of BER and education research.

Supplementary Material

• Allee, V. (2000). Knowledge networks and communities of learning . OD Practitioner , 32 ( 4 ), 4–13. [ Google Scholar ]
• Allen, M. (2017). The Sage encyclopedia of communication research methods (Vols. 1–4 ). Los Angeles, CA: Sage. 10.4135/9781483381411 [ CrossRef ] [ Google Scholar ]
• American Association for the Advancement of Science. (2011). Vision and change in undergraduate biology education: A call to action . Washington, DC. [ Google Scholar ]
• Anfara, V. A., Mertz, N. T. (2014). Setting the stage . In Anfara, V. A., Mertz, N. T. (eds.), Theoretical frameworks in qualitative research (pp. 1–22). Sage. [ Google Scholar ]
• Barnes, M. E., Brownell, S. E. (2016). Practices and perspectives of college instructors on addressing religious beliefs when teaching evolution . CBE—Life Sciences Education , 15 ( 2 ), ar18. https://doi.org/10.1187/cbe.15-11-0243 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Boote, D. N., Beile, P. (2005). Scholars before researchers: On the centrality of the dissertation literature review in research preparation . Educational Researcher , 34 ( 6 ), 3–15. 10.3102/0013189x034006003 [ CrossRef ] [ Google Scholar ]
• Booth, A., Sutton, A., Papaioannou, D. (2016a). Systemic approaches to a successful literature review (2nd ed.). Los Angeles, CA: Sage. [ Google Scholar ]
• Booth, W. C., Colomb, G. G., Williams, J. M., Bizup, J., Fitzgerald, W. T. (2016b). The craft of research (4th ed.). Chicago, IL: University of Chicago Press. [ Google Scholar ]
• Brownell, S. E., Kloser, M. J. (2015). Toward a conceptual framework for measuring the effectiveness of course-based undergraduate research experiences in undergraduate biology . Studies in Higher Education , 40 ( 3 ), 525–544. https://doi.org/10.1080/03075079.2015.1004234 [ Google Scholar ]
• Connolly, M. R., Lee, Y. G., Savoy, J. N. (2018). The effects of doctoral teaching development on early-career STEM scholars’ college teaching self-efficacy . CBE—Life Sciences Education , 17 ( 1 ), ar14. https://doi.org/10.1187/cbe.17-02-0039 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Cooper, K. M., Blattman, J. N., Hendrix, T., Brownell, S. E. (2019). The impact of broadly relevant novel discoveries on student project ownership in a traditional lab course turned CURE . CBE—Life Sciences Education , 18 ( 4 ), ar57. https://doi.org/10.1187/cbe.19-06-0113 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Creswell, J. W. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). Los Angeles, CA: Sage. [ Google Scholar ]
• DeHaan, R. L. (2011). Education research in the biological sciences: A nine decade review (Paper commissioned by the NAS/NRC Committee on the Status, Contributions, and Future Directions of Discipline Based Education Research) . Washington, DC: National Academies Press. Retrieved May 20, 2022, from www7.nationalacademies.org/bose/DBER_Mee ting2_commissioned_papers_page.html [ Google Scholar ]
• Ding, L. (2019). Theoretical perspectives of quantitative physics education research . Physical Review Physics Education Research , 15 ( 2 ), 020101. [ Google Scholar ]
• Dirks, C. (2011). The current status and future direction of biology education research . Paper presented at: Second Committee Meeting on the Status, Contributions, and Future Directions of Discipline-Based Education Research, 18–19 October (Washington, DC). Retrieved May 20, 2022, from http://sites.nationalacademies.org/DBASSE/BOSE/DBASSE_071087 [ Google Scholar ]
• Duran, R. P., Eisenhart, M. A., Erickson, F. D., Grant, C. A., Green, J. L., Hedges, L. V., Schneider, B. L. (2006). Standards for reporting on empirical social science research in AERA publications: American Educational Research Association . Educational Researcher , 35 ( 6 ), 33–40. [ Google Scholar ]
• Ebert-May, D., Derting, T. L., Henkel, T. P., Middlemis Maher, J., Momsen, J. L., Arnold, B., Passmore, H. A. (2015). Breaking the cycle: Future faculty begin teaching with learner-centered strategies after professional development . CBE—Life Sciences Education , 14 ( 2 ), ar22. https://doi.org/10.1187/cbe.14-12-0222 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Galvan, J. L., Galvan, M. C. (2017). Writing literature reviews: A guide for students of the social and behavioral sciences (7th ed.). New York, NY: Routledge. https://doi.org/10.4324/9781315229386 [ Google Scholar ]
• Gehrke, S., Kezar, A. (2017). The roles of STEM faculty communities of practice in institutional and departmental reform in higher education . American Educational Research Journal , 54 ( 5 ), 803–833. https://doi.org/10.3102/0002831217706736 [ Google Scholar ]
• Ghee, M., Keels, M., Collins, D., Neal-Spence, C., Baker, E. (2016). Fine-tuning summer research programs to promote underrepresented students’ persistence in the STEM pathway . CBE—Life Sciences Education , 15 ( 3 ), ar28. https://doi.org/10.1187/cbe.16-01-0046 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Institute of Education Sciences & National Science Foundation. (2013). Common guidelines for education research and development . Retrieved May 20, 2022, from www.nsf.gov/pubs/2013/nsf13126/nsf13126.pdf
• Jensen, J. L., Lawson, A. (2011). Effects of collaborative group composition and inquiry instruction on reasoning gains and achievement in undergraduate biology . CBE—Life Sciences Education , 10 ( 1 ), 64–73. https://doi.org/10.1187/cbe.19-05-0098 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Kolpikova, E. P., Chen, D. C., Doherty, J. H. (2019). Does the format of preclass reading quizzes matter? An evaluation of traditional and gamified, adaptive preclass reading quizzes . CBE—Life Sciences Education , 18 ( 4 ), ar52. https://doi.org/10.1187/cbe.19-05-0098 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Labov, J. B., Reid, A. H., Yamamoto, K. R. (2010). Integrated biology and undergraduate science education: A new biology education for the twenty-first century? CBE—Life Sciences Education , 9 ( 1 ), 10–16. https://doi.org/10.1187/cbe.09-12-0092 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Lane, T. B. (2016). Beyond academic and social integration: Understanding the impact of a STEM enrichment program on the retention and degree attainment of underrepresented students . CBE—Life Sciences Education , 15 ( 3 ), ar39. https://doi.org/10.1187/cbe.16-01-0070 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Lave, J. (1988). Cognition in practice: Mind, mathematics and culture in everyday life . New York, NY: Cambridge University Press. [ Google Scholar ]
• Lo, S. M., Gardner, G. E., Reid, J., Napoleon-Fanis, V., Carroll, P., Smith, E., Sato, B. K. (2019). Prevailing questions and methodologies in biology education research: A longitudinal analysis of research in CBE — Life Sciences Education and at the Society for the Advancement of Biology Education Research . CBE—Life Sciences Education , 18 ( 1 ), ar9. https://doi.org/10.1187/cbe.18-08-0164 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Lysaght, Z. (2011). Epistemological and paradigmatic ecumenism in “Pasteur’s quadrant:” Tales from doctoral research . In Official Conference Proceedings of the Third Asian Conference on Education in Osaka, Japan . Retrieved May 20, 2022, from http://iafor.org/ace2011_offprint/ACE2011_offprint_0254.pdf
• Maxwell, J. A. (2012). Qualitative research design: An interactive approach (3rd ed.). Los Angeles, CA: Sage. [ Google Scholar ]
• Miles, M. B., Huberman, A. M., Saldaña, J. (2014). Qualitative data analysis (3rd ed.). Los Angeles, CA: Sage. [ Google Scholar ]
• Nehm, R. (2019). Biology education research: Building integrative frameworks for teaching and learning about living systems . Disciplinary and Interdisciplinary Science Education Research , 1 , ar15. https://doi.org/10.1186/s43031-019-0017-6 [ Google Scholar ]
• Patton, M. Q. (2015). Qualitative research & evaluation methods: Integrating theory and practice . Los Angeles, CA: Sage. [ Google Scholar ]
• Perry, J., Meir, E., Herron, J. C., Maruca, S., Stal, D. (2008). Evaluating two approaches to helping college students understand evolutionary trees through diagramming tasks . CBE—Life Sciences Education , 7 ( 2 ), 193–201. https://doi.org/10.1187/cbe.07-01-0007 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Posner, G. J., Strike, K. A., Hewson, P. W., Gertzog, W. A. (1982). Accommodation of a scientific conception: Toward a theory of conceptual change . Science Education , 66 ( 2 ), 211–227. [ Google Scholar ]
• Ravitch, S. M., Riggan, M. (2016). Reason & rigor: How conceptual frameworks guide research . Los Angeles, CA: Sage. [ Google Scholar ]
• Reeves, T. D., Marbach-Ad, G., Miller, K. R., Ridgway, J., Gardner, G. E., Schussler, E. E., Wischusen, E. W. (2016). A conceptual framework for graduate teaching assistant professional development evaluation and research . CBE—Life Sciences Education , 15 ( 2 ), es2. https://doi.org/10.1187/cbe.15-10-0225 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Reynolds, J. A., Thaiss, C., Katkin, W., Thompson, R. J. Jr. (2012). Writing-to-learn in undergraduate science education: A community-based, conceptually driven approach . CBE—Life Sciences Education , 11 ( 1 ), 17–25. https://doi.org/10.1187/cbe.11-08-0064 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Rocco, T. S., Plakhotnik, M. S. (2009). Literature reviews, conceptual frameworks, and theoretical frameworks: Terms, functions, and distinctions . Human Resource Development Review , 8 ( 1 ), 120–130. https://doi.org/10.1177/1534484309332617 [ Google Scholar ]
• Rodrigo-Peiris, T., Xiang, L., Cassone, V. M. (2018). A low-intensity, hybrid design between a “traditional” and a “course-based” research experience yields positive outcomes for science undergraduate freshmen and shows potential for large-scale application . CBE—Life Sciences Education , 17 ( 4 ), ar53. https://doi.org/10.1187/cbe.17-11-0248 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Sabel, J. L., Dauer, J. T., Forbes, C. T. (2017). Introductory biology students’ use of enhanced answer keys and reflection questions to engage in metacognition and enhance understanding . CBE—Life Sciences Education , 16 ( 3 ), ar40. https://doi.org/10.1187/cbe.16-10-0298 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Sbeglia, G. C., Goodridge, J. A., Gordon, L. H., Nehm, R. H. (2021). Are faculty changing? How reform frameworks, sampling intensities, and instrument measures impact inferences about student-centered teaching practices . CBE—Life Sciences Education , 20 ( 3 ), ar39. https://doi.org/10.1187/cbe.20-11-0259 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Schwandt, T. A. (2000). Three epistemological stances for qualitative inquiry: Interpretivism, hermeneutics, and social constructionism . In Denzin, N. K., Lincoln, Y. S. (Eds.), Handbook of qualitative research (2nd ed., pp. 189–213). Los Angeles, CA: Sage. [ Google Scholar ]
• Sickel, A. J., Friedrichsen, P. (2013). Examining the evolution education literature with a focus on teachers: Major findings, goals for teacher preparation, and directions for future research . Evolution: Education and Outreach , 6 ( 1 ), 23. https://doi.org/10.1186/1936-6434-6-23 [ Google Scholar ]
• Singer, S. R., Nielsen, N. R., Schweingruber, H. A. (2012). Discipline-based education research: Understanding and improving learning in undergraduate science and engineering . Washington, DC: National Academies Press. [ Google Scholar ]
• Todd, A., Romine, W. L., Correa-Menendez, J. (2019). Modeling the transition from a phenotypic to genotypic conceptualization of genetics in a university-level introductory biology context . Research in Science Education , 49 ( 2 ), 569–589. https://doi.org/10.1007/s11165-017-9626-2 [ Google Scholar ]
• Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes . Cambridge, MA: Harvard University Press. [ Google Scholar ]
• Wenger, E. (1998). Communities of practice: Learning as a social system . Systems Thinker , 9 ( 5 ), 2–3. [ Google Scholar ]
• Ziadie, M. A., Andrews, T. C. (2018). Moving evolution education forward: A systematic analysis of literature to identify gaps in collective knowledge for teaching . CBE—Life Sciences Education , 17 ( 1 ), ar11. https://doi.org/10.1187/cbe.17-08-0190 [ PMC free article ] [ PubMed ] [ Google Scholar ]

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: May 15, 2024 9:53 AM
• URL: https://libguides.usc.edu/writingguide

Harvey Cushing/John Hay Whitney Medical Library

• Collections
• Research Help

YSN Doctoral Programs: Steps in Conducting a Literature Review

• Biomedical Databases
• Global (Public Health) Databases
• Soc. Sci., History, and Law Databases
• Grey Literature
• Trials Registers
• Data and Statistics
• Public Policy
• Google Tips
• Recommended Books
• Steps in Conducting a Literature Review

What is a literature review?

A literature review is an integrated analysis -- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question.  That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.

A literature review may be a stand alone work or the introduction to a larger research paper, depending on the assignment.  Rely heavily on the guidelines your instructor has given you.

Why is it important?

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Discovers relationships between research studies/ideas.
• Identifies major themes, concepts, and researchers on a topic.
• Identifies critical gaps and points of disagreement.
• Discusses further research questions that logically come out of the previous studies.

APA7 Style resources

APA Style Blog - for those harder to find answers

1. Choose a topic. Define your research question.

Your literature review should be guided by your central research question.  The literature represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.

• Make sure your research question is not too broad or too narrow.  Is it manageable?
• Begin writing down terms that are related to your question. These will be useful for searches later.
• If you have the opportunity, discuss your topic with your professor and your class mates.

2. Decide on the scope of your review

How many studies do you need to look at? How comprehensive should it be? How many years should it cover? 

• This may depend on your assignment.  How many sources does the assignment require?

3. Select the databases you will use to conduct your searches.

Make a list of the databases you will search. 

Where to find databases:

• use the tabs on this guide
• Find other databases in the Nursing Information Resources web page
• More on the Medical Library web page
• ... and more on the Yale University Library web page

4. Conduct your searches to find the evidence. Keep track of your searches.

• Use the key words in your question, as well as synonyms for those words, as terms in your search. Use the database tutorials for help.
• Save the searches in the databases. This saves time when you want to redo, or modify, the searches. It is also helpful to use as a guide is the searches are not finding any useful results.
• Review the abstracts of research studies carefully. This will save you time.
• Use the bibliographies and references of research studies you find to locate others.
• Check with your professor, or a subject expert in the field, if you are missing any key works in the field.
• Ask your librarian for help at any time.
• Use a citation manager, such as EndNote as the repository for your citations. See the EndNote tutorials for help.

Review the literature

Some questions to help you analyze the research:

• What was the research question of the study you are reviewing? What were the authors trying to discover?
• Was the research funded by a source that could influence the findings?
• What were the research methodologies? Analyze its literature review, the samples and variables used, the results, and the conclusions.
• Does the research seem to be complete? Could it have been conducted more soundly? What further questions does it raise?
• If there are conflicting studies, why do you think that is?
• How are the authors viewed in the field? Has this study been cited? If so, how has it been analyzed?

Tips: 

• Review the abstracts carefully.  
• Keep careful notes so that you may track your thought processes during the research process.
• Create a matrix of the studies for easy analysis, and synthesis, across all of the studies.
• << Previous: Recommended Books
• Last Updated: Jan 4, 2024 10:52 AM
• URL: https://guides.library.yale.edu/YSNDoctoral

• Meriam Library

SWRK 330 - Social Work Research Methods

• Literature Reviews and Empirical Research
• Databases and Search Tips
• Article Citations
• Scholarly Journal Evaulation
• Statistical Sources
• Books and eBooks

What is a Literature Review?

Empirical research.

• Annotated Bibliographies

A literature review  summarizes and discusses previous publications  on a topic.

It should also:

explore past research and its strengths and weaknesses.

be used to validate the target and methods you have chosen for your proposed research.

consist of books and scholarly journals that provide research examples of populations or settings similar to your own, as well as community resources to document the need for your proposed research.

The literature review does not present new  primary  scholarship. 

be completed in the correct citation format requested by your professor  (see the  C itations Tab)

Access Purdue  OWL's Social Work Literature Review Guidelines here .  

Empirical Research  is  research  that is based on experimentation or observation, i.e. Evidence. Such  research  is often conducted to answer a specific question or to test a hypothesis (educated guess).

How do you know if a study is empirical? Read the subheadings within the article, book, or report and look for a description of the research "methodology."  Ask yourself: Could I recreate this study and test these results?

These are some key features to look for when identifying empirical research.

NOTE:  Not all of these features will be in every empirical research article, some may be excluded, use this only as a guide.

• Statement of methodology
• Research questions are clear and measurable
• Individuals, group, subjects which are being studied are identified/defined
• Data is presented regarding the findings
• Controls or instruments such as surveys or tests were conducted
• There is a literature review
• There is discussion of the results included
• Citations/references are included

See also Empirical Research Guide

• << Previous: Citations
• Next: Annotated Bibliographies >>
• Last Updated: Feb 6, 2024 8:38 AM
• URL: https://libguides.csuchico.edu/SWRK330

Meriam Library | CSU, Chico

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

2.3 Reviewing the Research Literature

Learning objectives.

• Define the research literature in psychology and give examples of sources that are part of the research literature and sources that are not.
• Describe and use several methods for finding previous research on a particular research idea or question.

Reviewing the research literature means finding, reading, and summarizing the published research relevant to your question. An empirical research report written in American Psychological Association (APA) style always includes a written literature review, but it is important to review the literature early in the research process for several reasons.

• It can help you turn a research idea into an interesting research question.
• It can tell you if a research question has already been answered.
• It can help you evaluate the interestingness of a research question.
• It can give you ideas for how to conduct your own study.
• It can tell you how your study fits into the research literature.

What Is the Research Literature?

The research literature in any field is all the published research in that field. The research literature in psychology is enormous—including millions of scholarly articles and books dating to the beginning of the field—and it continues to grow. Although its boundaries are somewhat fuzzy, the research literature definitely does not include self-help and other pop psychology books, dictionary and encyclopedia entries, websites, and similar sources that are intended mainly for the general public. These are considered unreliable because they are not reviewed by other researchers and are often based on little more than common sense or personal experience. Wikipedia contains much valuable information, but the fact that its authors are anonymous and its content continually changes makes it unsuitable as a basis of sound scientific research. For our purposes, it helps to define the research literature as consisting almost entirely of two types of sources: articles in professional journals, and scholarly books in psychology and related fields.

Professional Journals

Professional journals are periodicals that publish original research articles. There are thousands of professional journals that publish research in psychology and related fields. They are usually published monthly or quarterly in individual issues, each of which contains several articles. The issues are organized into volumes, which usually consist of all the issues for a calendar year. Some journals are published in hard copy only, others in both hard copy and electronic form, and still others in electronic form only.

Most articles in professional journals are one of two basic types: empirical research reports and review articles. Empirical research reports describe one or more new empirical studies conducted by the authors. They introduce a research question, explain why it is interesting, review previous research, describe their method and results, and draw their conclusions. Review articles summarize previously published research on a topic and usually present new ways to organize or explain the results. When a review article is devoted primarily to presenting a new theory, it is often referred to as a theoretical article .

Figure 2.6 Small Sample of the Thousands of Professional Journals That Publish Research in Psychology and Related Fields

Most professional journals in psychology undergo a process of peer review . Researchers who want to publish their work in the journal submit a manuscript to the editor—who is generally an established researcher too—who in turn sends it to two or three experts on the topic. Each reviewer reads the manuscript, writes a critical review, and sends the review back to the editor along with his or her recommendations. The editor then decides whether to accept the article for publication, ask the authors to make changes and resubmit it for further consideration, or reject it outright. In any case, the editor forwards the reviewers’ written comments to the researchers so that they can revise their manuscript accordingly. Peer review is important because it ensures that the work meets basic standards of the field before it can enter the research literature.

Scholarly Books

Scholarly books are books written by researchers and practitioners mainly for use by other researchers and practitioners. A monograph is written by a single author or a small group of authors and usually gives a coherent presentation of a topic much like an extended review article. Edited volumes have an editor or a small group of editors who recruit many authors to write separate chapters on different aspects of the same topic. Although edited volumes can also give a coherent presentation of the topic, it is not unusual for each chapter to take a different perspective or even for the authors of different chapters to openly disagree with each other. In general, scholarly books undergo a peer review process similar to that used by professional journals.

Literature Search Strategies

Using psycinfo and other databases.

The primary method used to search the research literature involves using one or more electronic databases. These include Academic Search Premier, JSTOR, and ProQuest for all academic disciplines, ERIC for education, and PubMed for medicine and related fields. The most important for our purposes, however, is PsycINFO , which is produced by the APA. PsycINFO is so comprehensive—covering thousands of professional journals and scholarly books going back more than 100 years—that for most purposes its content is synonymous with the research literature in psychology. Like most such databases, PsycINFO is usually available through your college or university library.

PsycINFO consists of individual records for each article, book chapter, or book in the database. Each record includes basic publication information, an abstract or summary of the work, and a list of other works cited by that work. A computer interface allows entering one or more search terms and returns any records that contain those search terms. (These interfaces are provided by different vendors and therefore can look somewhat different depending on the library you use.) Each record also contains lists of keywords that describe the content of the work and also a list of index terms. The index terms are especially helpful because they are standardized. Research on differences between women and men, for example, is always indexed under “Human Sex Differences.” Research on touching is always indexed under the term “Physical Contact.” If you do not know the appropriate index terms, PsycINFO includes a thesaurus that can help you find them.

Given that there are nearly three million records in PsycINFO, you may have to try a variety of search terms in different combinations and at different levels of specificity before you find what you are looking for. Imagine, for example, that you are interested in the question of whether women and men differ in terms of their ability to recall experiences from when they were very young. If you were to enter “memory for early experiences” as your search term, PsycINFO would return only six records, most of which are not particularly relevant to your question. However, if you were to enter the search term “memory,” it would return 149,777 records—far too many to look through individually. This is where the thesaurus helps. Entering “memory” into the thesaurus provides several more specific index terms—one of which is “early memories.” While searching for “early memories” among the index terms returns 1,446 records—still too many too look through individually—combining it with “human sex differences” as a second search term returns 37 articles, many of which are highly relevant to the topic.

Depending on the vendor that provides the interface to PsycINFO, you may be able to save, print, or e-mail the relevant PsycINFO records. The records might even contain links to full-text copies of the works themselves. (PsycARTICLES is a database that provides full-text access to articles in all journals published by the APA.) If not, and you want a copy of the work, you will have to find out if your library carries the journal or has the book and the hard copy on the library shelves. Be sure to ask a librarian if you need help.

Using Other Search Techniques

In addition to entering search terms into PsycINFO and other databases, there are several other techniques you can use to search the research literature. First, if you have one good article or book chapter on your topic—a recent review article is best—you can look through the reference list of that article for other relevant articles, books, and book chapters. In fact, you should do this with any relevant article or book chapter you find. You can also start with a classic article or book chapter on your topic, find its record in PsycINFO (by entering the author’s name or article’s title as a search term), and link from there to a list of other works in PsycINFO that cite that classic article. This works because other researchers working on your topic are likely to be aware of the classic article and cite it in their own work. You can also do a general Internet search using search terms related to your topic or the name of a researcher who conducts research on your topic. This might lead you directly to works that are part of the research literature (e.g., articles in open-access journals or posted on researchers’ own websites). The search engine Google Scholar is especially useful for this purpose. A general Internet search might also lead you to websites that are not part of the research literature but might provide references to works that are. Finally, you can talk to people (e.g., your instructor or other faculty members in psychology) who know something about your topic and can suggest relevant articles and book chapters.

What to Search For

When you do a literature review, you need to be selective. Not every article, book chapter, and book that relates to your research idea or question will be worth obtaining, reading, and integrating into your review. Instead, you want to focus on sources that help you do four basic things: (a) refine your research question, (b) identify appropriate research methods, (c) place your research in the context of previous research, and (d) write an effective research report. Several basic principles can help you find the most useful sources.

First, it is best to focus on recent research, keeping in mind that what counts as recent depends on the topic. For newer topics that are actively being studied, “recent” might mean published in the past year or two. For older topics that are receiving less attention right now, “recent” might mean within the past 10 years. You will get a feel for what counts as recent for your topic when you start your literature search. A good general rule, however, is to start with sources published in the past five years. The main exception to this rule would be classic articles that turn up in the reference list of nearly every other source. If other researchers think that this work is important, even though it is old, then by all means you should include it in your review.

Second, you should look for review articles on your topic because they will provide a useful overview of it—often discussing important definitions, results, theories, trends, and controversies—giving you a good sense of where your own research fits into the literature. You should also look for empirical research reports addressing your question or similar questions, which can give you ideas about how to operationally define your variables and collect your data. As a general rule, it is good to use methods that others have already used successfully unless you have good reasons not to. Finally, you should look for sources that provide information that can help you argue for the interestingness of your research question. For a study on the effects of cell phone use on driving ability, for example, you might look for information about how widespread cell phone use is, how frequent and costly motor vehicle crashes are, and so on.

How many sources are enough for your literature review? This is a difficult question because it depends on how extensively your topic has been studied and also on your own goals. One study found that across a variety of professional journals in psychology, the average number of sources cited per article was about 50 (Adair & Vohra, 2003). This gives a rough idea of what professional researchers consider to be adequate. As a student, you might be assigned a much lower minimum number of references to use, but the principles for selecting the most useful ones remain the same.

Key Takeaways

• The research literature in psychology is all the published research in psychology, consisting primarily of articles in professional journals and scholarly books.
• Early in the research process, it is important to conduct a review of the research literature on your topic to refine your research question, identify appropriate research methods, place your question in the context of other research, and prepare to write an effective research report.
• There are several strategies for finding previous research on your topic. Among the best is using PsycINFO, a computer database that catalogs millions of articles, books, and book chapters in psychology and related fields.
• Practice: Use the techniques discussed in this section to find 10 journal articles and book chapters on one of the following research ideas: memory for smells, aggressive driving, the causes of narcissistic personality disorder, the functions of the intraparietal sulcus, or prejudice against the physically handicapped.

Adair, J. G., & Vohra, N. (2003). The explosion of knowledge, references, and citations: Psychology’s unique response to a crisis. American Psychologist, 58 , 15–23.

Research Methods in Psychology Copyright © 2016 by University of Minnesota is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Advertisement

Open government data: A systematic literature review of empirical research

• Research Paper
• Open access
• Published: 20 September 2022
• Volume 32 , pages 2381–2404, ( 2022 )

Cite this article

You have full access to this open access article

• Bernd W. Wirtz   ORCID: orcid.org/0000-0003-1480-8513 1 ,
• Jan C. Weyerer 1 ,
• Marcel Becker 1 &
• Wilhelm M. Müller 1  

7995 Accesses

21 Citations

2 Altmetric

Explore all metrics

Open government data (OGD) holds great potential for firms and the digital economy as a whole and has attracted increasing interest in research and practice in recent years. Governments and organizations worldwide are struggling in exploiting the full potential of OGD and require a comprehensive understanding of this phenomenon. Although scientific debates in OGD research are intense and heterogeneous, the field lacks theoretical integration of OGD topics and their systematic consideration in the context of the digital economy. In addition, OGD has been widely neglected by information systems (IS) research, which promises great potential for advancing our knowledge of the OGD concept and its role in the digital economy. To fill in this gap, this study conducts a systematic literature review of 169 empirical OGD studies. In doing so, we develop a theoretical review framework of Antecedents, Decisions, Outcomes (ADO) to unify and grasp the accumulating isolated evidence on OGD in context of the digital economy and provide a theory-informed research agenda to tap the potential of IS research for OGD. Our findings reveal six related key topic clusters of OGD research and substantial gaps, opening up prospective research avenues and particularly outlining how IS research can inform and advance OGD research.

Similar content being viewed by others

Criteria for Good Qualitative Research: A Comprehensive Review

Digital transformation: a review, synthesis and opportunities for future research

Environmental-, social-, and governance-related factors for business investment and sustainability: a scientometric review of global trends

Avoid common mistakes on your manuscript.

Introduction

In the age of the digital economy, data have become a new currency and an indispensable asset for organizations. Data constitutes the foundation of innovative technologies and applications (e.g., AI and IoT) and data-driven insights and management are vital for organizational success. The advancing digitalization in the public sector over the last decade has led to large amounts of data, making the public sector one of the main producers of data in the digital economy. A substantial part of this data pool is freely provided to the public and is commonly referred to as open government data (OGD) (Kim, 2018 ; Lim, 2021 ). As the number and worldwide development of OGD initiatives continue to advance in light of its great importance (Attard et al., 2015 ; Piotrowski, 2017 ), the widely unexplored relationship between OGD and the digital economy becomes of increasing interest.

On the one hand, the digital economy itself constitutes an important driver of OGD adoption and the successful implementation of OGD programs, as IT firms, for instance, supply public administration with mission-critical tangible (e.g., hardware and software), human (e.g., IT consultants), and intangible (e.g., IT and data know how) IT resources. On the other hand, OGD constitutes a new source of innovation and economic growth for the digital economy. OGD offers the potential to create innovation and to increase economic value sustainably - for both the public and the private business sectors. It may serve organizations as a free and meaningful complementary data source in developing new products or services, as well as in improving business intelligence, R&D, and business processes (Magalhaes & Roseira, 2020 ). Thus, OGD and the digital economy are characterized by a reciprocal relationship, in which both sides benefit from each other.

While the public value of OGD in terms of leveling up the transparency of governmental activities, the political participation of citizens and the collaboration between governments and external stakeholders is well-documented (Lee et al., 2019 ; Ruijer et al., 2017 ), its great opportunities and importance for the digital economy and commercial use have been widely neglected. According to the World Wide Web Foundation ( 2017 ), the impact of OGD on the economy even in the top ten countries worldwide remains rather low, averaging four out of ten on their assessment scale. A recent survey of 178 U.S. firms on the use OGD further reveals that the frequency of application varies across different forms of use, ranging from 9% (data to fact) to 44% (data to service) (Magalhaes & Roseira, 2020 ). These figures indicate that firms and the digital economy as a whole seem to struggle in using OGD and exploiting its full potential. This is also reflected in the current research landscape, in which the OGD concept has been predominantly examined in public administration and public management research, while receiving little attention in the field of information systems (IS) and digital business research.

Given its relevance for the digital economy and close relatedness to information systems and various associated research streams (e.g., big data analytics, AI and IoT), it is essential to frame OGD more broadly in the context of the digital economy and build a bridge to IS and digital business research. The stronger involvement of the latter promises great potential for further advancing the OGD concept and filling in the gap pertaining to its role in the digital economy and commercial use, as demanded in the literature (Magalhaes & Roseira, 2020 ). In order to better familiarize the IS and digital business research community with OGD and meaningfully involve it in the scholarly discussion, it is essential to first convey a broad understanding of the concept, its research landscape, and specific starting points for potential research endeavours.

As the role of the digital economy in OGD initiatives and the value potential of OGD is influenced by the antecedents of OGD programs (e.g., sophistication of governmental data infrastructures), the decisions and actions taken by the government for implementing OGD (e.g., strategic positioning and scope of governmental OGD activity), as well as the achieved outcomes and impacts (e.g., efficiency gains through and acceptance of OGD), it seems particularly promising to examine OGD and its relevance for the digital economy along these dimensions.

The research field of OGD has been on the rise over the last decade. While the number and heterogeneity of contributions are increasing, comprehensive literature reviews remain scarce in the context of open government (Tai, 2021 ), in particular from an IS perspective. Most importantly, OGD research lacks theoretical foundation and integration of OGD topics (Hassan & Twinomurinzi, 2018 ), as well as their systematic examination in the context of the digital economy. Taken together, the literature fails to provide a theoretical framework combining theoretical and empirical insights on OGD with regard to its antecedents, decisions, and outcomes, in which the concept is framed more broadly in the context of the digital economy, and which yields a research agenda that meaningfully involves the field of IS and digital business research. To fill in this gap, we conduct a systematic literature review to address the following research questions: (1) what do we know about the antecedents, decisions, and outcomes of OGD and their relation in the context of the digital economy, and (2) how can IS and digital business research inform OGD research in this connection?

To answer these research questions, the remainder of the study is structured as follows: The next section discusses definitional issues of OGD, delineating it from the closely related concepts of open government and open data. We then present an overview of prior literature reviews related to OGD and illustrate their shortcomings and implications for the study at hand. Subsequently, we describe the methodological approach and results of the systematic review of OGD literature and develop an overarching theoretical framework to integrate and synthesize thematic clusters of OGD research. Based on this, we derive a research agenda for future research on OGD providing concrete research avenues for IS and digital business research. In the final section, the findings and implications are discussed in the context of prior research and the digital economy.

Defining open government data between the poles of open government and open data

OGD is closely related to other concepts, in particular, open government and open data. Although it may be viewed as a hybrid of both of these more general concepts (Sayogo et al., 2014 ), the extensive number of dedicated OGD studies in recent years indicates not only the increasing scholarly interest but also that OGD has established itself as a distinct concept and research stream separate from its superordinates, open government and open data. This also becomes apparent when looking at differentiated definitions of each concept. To begin with, open government is generally defined as “a multilateral, political, and social process, which includes in particular transparent, collaborative, and participatory action by government and administration” (Wirtz & Birkmeyer, 2015 , p. 382). Although OGD can be viewed as a manifestation thereof underlying the same principles of transparency, collaboration and participation (Wirtz et al., 2019 ), it sets itself apart from the general concept through its data character and thus its inherently closer link to information systems.

This data characteristic is – besides the openness – the common denominator of the OGD and the open data concept and separates both from the open government concept. A widely used definition of open data refers to data that “can be freely used, modified, and shared by anyone for any purpose” (Open Knowledge Foundation, 2021 ). The definition of open government and its delineation from open data has been subject to many debates in the literature (Bogdanović-Dinić et al., 2014 ; Karkin & Yavuz, 2017 ; Kim, 2018 ; Sayogo et al., 2014 ). Although some earlier approaches use both terms synonymously (Janssen et al., 2012 ; Veljković et al., 2014 ), there is meanwhile consensus in the literature that OGD constitutes a subform of open data and the special distinguishing mark is that OGD is data collected by means of public funding and/or provided by public sector organizations (Borgesius et al., 2015 ; Kim, 2018 ; Lim, 2021 ). Accordingly, OGD is defined as “non-confidential, non-privacy-restricted data collected using public funding that is made freely available for anyone to download” (Lim, 2021 , p. 1) or put more simple as “[p]ublic sector information made available to the public as open data” (Kim, 2018 , p. 20). Thus, its government relatedness is the decisive element distinguishing it from open data.

For a better understanding of the scope and nature of OGD, the OECD (Ubaldi, 2013 ) has developed a typology of OGD, distinguishing between seven major categories: (1) business data (e.g., chamber of commerce information and official business information with regard to company or industry data), (2) registers and data pertaining to patents, trademarks, and public tenders, (3) geographic data (e.g., topographic and address data), (4) legal data (e.g., court decisions, legislation data), (5) meteorological data (e.g., weather and climate data), (6) social data (e.g., population, employment, and public health data), and (7) transport data (e.g., vehicle registrations, traffic, and public transport data). This typology underlines the particularities of OGD and indicates its various application opportunities and value for businesses.

Prior literature reviews on OGD

The widespread scientific interest in OGD is reflected in a large number of studies, which have been the motivator and starting point for various overview studies. With a view to placing our systematic literature review in the existing field of literature reviews and determining its potential contribution to future OGD research, we first identified and analyzed the thematically relevant set of previous literature reviews. We systematically searched for literature reviews in different databases, including EBSCO (including Academic Search Premier, Business Source Premier, and EconLit with Full Text), Web of Science, ScienceDirect, ProQuest, and Google Scholar. This yielded a total of twelve dedicated literature reviews that were obtained for further analysis. To determine the scientific added value of our study, it is important to contrast the core structure and key topics of these literature reviews briefly and concisely, see Table  1 .

The literature reviews identified can be classified into three clusters: (1) reviews treating OGD as a side aspect, (2) reviews focusing on a specific aspect of OGD literature, and (3) reviews with a general approach towards OGD literature. The first cluster contains four out of twelve reviews identified. These reviews do not clearly distinguish between open government, open data, and OGD, and thus mix OGD studies in their analysis with those from one of the other research streams. To begin with, Hossain et al. ( 2016 ) provide a general systematization of the research field of open data, addressing OGD as one of five subareas and deriving corresponding research implications. The other three reviews in this cluster focus on OGD including OGD studies as a subset in their analyses . While Wirtz and Birkmeyer ( 2015 ) concentrate on the development of an integrative framework to better understand open government in general, Criado et al. ( 2018 ) attempt to explain the phenomenon of open government by means of a comprehensive analysis of existing literature and provide a comprehensive overview without deriving overly specific research implications. Likewise, Tai ( 2021 ) also provides a comprehensive review of open government research, focusing on three aspects, namely its conceptual development, its use and implementation, as well as the impacts or outcomes of open government initiatives. However, an integrated consideration as applied by the above-mentioned reviews in the first cluster confounds a clear picture of OGD research and carries the risk of arriving at undifferentiated and ultimately inaccurate conclusions. Therefore, it is essential to conduct review studies that are solely dedicated to the field of OGD, as is the case with the second and third cluster of review studies.

The second cluster is the largest and is composed of six out of twelve literature reviews identified. These reviews analyze a certain segment of OGD literature depending on a selected subtopic. The work of Attard et al. ( 2015 ) clearly focuses on the description of OGD initiatives and their respective components. They are less concerned with mapping and structuring the literature as a whole but rather with analyzing OGD initiatives and related approaches. In contrast, Ruijer and Martinius ( 2017 ) set their focus more specifically by examining literature and deriving specific research implications in relation to the democratic impact of OGD. Safarov et al. ( 2017 ) have a different emphasis by orienting their literature evaluation and systematization towards the development of an OGD utilization framework and pointing out utilization-specific research opportunities. Moreover, the literature review of Haini et al. ( 2020 ) has a special view upon studies concerning influence factors of OGD adoption in public sector organizations, identifying 16 influence factors and classifying them according to three dimensions (i.e., technological, organizational, and environmental). In contrast, Purwanto et al. ( 2020 ) focus on the citizen perspective in their review and analyze studies that deal with drivers of and barriers to citizen engagement with OGD. They identify seven groups of drivers and three categories of barriers, developing a conceptual model of citizen engagement with OGD. Finally, Francey and Mettler ( 2021 ) review case studies and examine empirical evidence on the effects of OGD, deriving nine stylized facts. While all of the studies in the second cluster provide valuable insights into the field of OGD, they only do so for the respective subtopic analyzed. None of these reviews systematizes the entire field of research and identifies the implications for further necessary research. Although Safarov et al. ( 2017 ) make a well-conceived attempt to broadly analyze and systematize based on their grouping along four key topics and the further subdivision thereof, their findings still remain specific in that they are primarily concerned with the utilization of OGD. Thus, the reviews in this cluster do not allow to make profound comparisons among subtopics within the field or to draw general conclusions in order to improve our understanding of relationships among subtopics and the state of research as a whole. This can only be achieved by reviews with a comprehensive perspective, like those in the third cluster of our literature review analysis.

This cluster is the smallest and comprises only two reviews, indicating the lack of reviews with a comprehensive focus on OGD research. These approaches are most relevant to our study because they likewise address the OGD topic as a whole. In doing so, Zuiderwijk et al. ( 2014 ) examine individual studies in relation to their topic and theoretical foundation. They offer a brief outlook on potential fields of research related to the three core topics they identified, including theory and development; policies, use, and innovation; as well as infrastructures and technology. Saxena’s ( 2018 ) systematic literature review likewise classifies OGD studies into three general clusters, i.e. theoretical and conceptual research, applied research, and user-focused research. Despite their valuable contributions both studies lack theoretical foundation and integration of the clusters. Moreover, both reviews each propose a very general taxonomy to structure research. Both taxonomies contain three clusters and appear to be little differentiated given the heterogeneity of the current research landscape. Paired with their purely descriptive nature of analysis, they only provide basic research implications that lack thematic specification and thoroughness.

The above-mentioned studies in each cluster constitute a thorough selection of OGD-related literature reviews in peer-reviewed journals. However, a literature search in the databases of AIS, IEEE, and ACM shows that several literature reviews on OGD have also been published in conference proceedings, which also should be acknowledged at this point. These contributions can also be classified according to the proposed clusters and are subject to the same shortcomings and criticism. While the broad and very early approach of Novais et al. ( 2013 ) can be assigned to the third cluster of reviews, all of the other review attempts belong to the second cluster, as they focus on specific aspects in connection with OGD, in particular, barriers or problems associated with OGD implementation and development (Bachtiar et al., 2020 ; Crusoe & Melin, 2018 ; Neto et al., 2018 ; Roa et al., 2019 ), but also challenges and opportunities associated with OGD (Hassan & Twinomurinzi, 2018 ), or the impact of civil servants’ behavioral factors on the opening of government data (Kleiman et al., 2020 ).

Overall, the analysis of literature reviews confirms the conceptual autonomy of OGD and its independent research stream (emphasized in the above-mentioned definitional considerations), since eight out of twelve reviews are specifically dedicated to OGD. Our findings further show that previous review approaches lack theoretical integration of OGD issues and do not consider them in the context of the digital economy. Accordingly, they do not provide answers to our research questions of what we know about the antecedents, decisions, and outcomes of OGD and their relation in connection with the digital economy and how IS and digital business research can inform OGD research in this respect. Given the increasing importance of OGD and the digital economy as well as their reciprocal relationship, it is essential for the further development and a better understanding of the OGD concept to systematically theorize and synthesize the respective body of knowledge. Our systematic literature review goes beyond prior literature reviews and addresses their shortcomings by developing a theoretical review framework of antecedents, decisions, and outcomes of OGD, elaborating them in relation to the digital economy and deriving a theory-informed research agenda to tap the potential of IS and digital business research for OGD.

Methodology of the systematic literature review

Literature selection.

The literature review is based on established methodological recommendations regarding a general literature review’s overall structure and the related process of identification and selection of relevant studies (Tranfield et al., 2003 ; Webster & Watson, 2002 ). In order to comprehensively and systematically search for and select relevant studies, we followed further procedural guidelines according to the well-established PRISMA flow process adhering to its individual steps of identification, screening, eligibility, and final inclusion (Liberati et al., 2009 ).

To identify relevant records from established and relevant academic databases, we initially conducted a title, abstract, and subject search in different databases, including EBSCO (including Academic Search Premier, Business Source Premier, and EconLit with Full Text), Web of Science, ScienceDirect, and ProQuest . The search included the terms “open government data”, “data openness”, and “open data” in combination with “government” and “governance”. For the purpose of scientific rigor and quality, the search was limited to articles published in peer-reviewed academic journals in English language (Wang et al., 2019 ). Subsequent to the identification and elimination of duplicate records, editorial notes, and comments, the retrieved articles were first screened regarding title and abstract to determine and exclude irrelevant studies. The remaining articles were then subjected to a full-text review to exclude any studies that were not empirical and whose thematic focus was not clearly attributed to the field of OGD. This initial literature approach resulted in a total of 125 articles conforming to the selection criteria. To complement this set of literature with meaningful conference papers, we likewise searched the databases of AIS, IEEE, and ACM, yielding another 37 relevant articles. To minimize the risk of missing relevant studies, we finally screened the Google Scholar database using the same search terms with attention to the same criteria, since Google Scholar is the most comprehensive database (Gusenbauer, 2019 ; Martín-Martín et al., 2020 ) and is considered to be especially useful for identifying influential studies within specific fields of research (Martín-Martín et al., 2017 ; Zientek et al., 2018 ). In this way, seven additional eligible studies were identified and added to the selection, resulting in a final set of 169 relevant studies from the overall literature search, which represents the basis of the following preparation and analysis. Similarly to the entire selection process and assessment of eligibility, the further review, coding, and classification of the literature was performed by two reviewers. They were supported by a third reviewer who took a mediating role to assist once again in case of disagreement. The analysis of the literature consisted of two steps. The first step of our approach comprised the identification of key topic clusters in the literature by means of a bottom-up coding approach in order to determine what kind of topics are actually prevalent in the literature without constraining the result to certain areas. The second step referred to the theoretical integration of these clusters by means of a framework-based approach. In the following, we explain the methodological procedures underlying these two steps of analysis in more detail.

Identification of key topic clusters

In this first step of the analysis, the individual studies were assigned to individual clusters according to their respective content and thematic structure. Due to the thematic complexity of OGD and the associated heterogeneity of research, as well as different foci of the individual studies, the development and final formulation of the individual key topic clusters were designed and refined through a stepwise systematic coding process. This coding process relied on the approach of Saldaña ( 2013 ) and incorporated techniques of initial coding and pattern coding. Initial coding is an open form of coding, in which qualitative information is broken down into discrete aspects. While initial coding is the first step of analysis and serves “as a starting point to provide the researcher with analytic leads for further exploration” (Saldaña, 2013 , p. 101), pattern coding takes the analysis to a higher and more abstract level by refining the codes developed in the initial coding step and merging them into superordinate categories. The openness of this two-step approach already indicates that it follows an inductive procedure without a predefined coding scheme. This means that the formed concepts or categories emerge from the given data, which is characteristic for a bottom-up approach (Urquhart, 2013 ). Following this procedure, relevant information from the respective studies was initially coded. The resulting codes were then carefully and repeatedly examined to determine patterns in terms of similarities, correlations, and dissimilarities. The respective key topic clusters were then compared regarding their overall degree of similarity or distinction and refined, if necessary, in order to achieve optimum accuracy and consistency. This procedure yielded a final set of six key topic clusters, including (1) general/conceptual development (OGD theory), (2) drivers/barriers (OGD antecedents), (3) adoption/usage/implementation (OGD decisions), (4) success/performance/value (OGD outcomes), (5) acceptance/satisfaction/trust in government (OGD impacts), and (6) policies/regulation/law (OGD governance). The literature was then analyzed and structured according to these key topic clusters and a number of other classification criteria, including study type, method of analysis, data collection, and research perspective. The results of this step of analysis are depicted in the overview and evolution of the OGD literature.

Theoretical integration of key topic clusters

The second step referred to the theoretical integration of these clusters and thus their arrangement in a common complex of meaning. Here, we applied a framework-based approach (Paul & Criado, 2020 ), developing an overarching theoretical review framework that organizes the theoretical relationships among the identified thematic clusters of OGD in terms of a relationship map (Watson & Webster, 2020 ). This framework-based approach to literature was informed by previous literature reviews (Kessler & Chakrabarti, 1996 ; Lane et al., 2006 ; Raisch & Birkinshaw, 2008 ) and is particularly based on the antecedents, decisions, and outcomes (ADO) framework by Paul and Benito ( 2018 ), which is regarded as “an excellent framework to organize the findings (i.e., constructs and its ensuing relationships) of past research in a structured assembly” (Lim et al., 2021 , p. 537). The ADO framework approach appeared to be particularly suitable as it provides overarching and general theoretically linked dimensions to which the specific clusters could be meaningfully assigned. Thus, the framework-based approach, i.e. the predefined dimensions of the ADO framework and their relationships given by prior literature, provides an established but at the same time only rough grid, which is specified with the core clusters identified by means of the bottom-up method in the first step of the analysis. The theoretical review framework developed in this second step of analysis and the corresponding theoretical integration of the core clusters in the context of the digital economy are presented in the synthesis of OGD literature. The framework finally also serves as a point of reference for deriving the theory-informed research agenda for IS and digital business research (Fig. 1 ).

Development of open government data research

Overview and evolution of the OGD literature

To provide a better understanding of the extent and evolution of the empirical OGD literature, this section gives a brief overview of its general development and current state. To begin with, Fig.  1 illustrates the distribution of qualitative and quantitative empirical OGD studies over the last 10 years.

Considering that OGD has evolved as an independent research stream out of general open government and open data research, it is not surprising that empirical research on OGD developed with a certain time lag in comparison to both of these more general research streams. Although OGD-related research was initially, in particular, an integral part of open government research, the first empirical and dedicated OGD studies appeared in 2011. Academic interest has increased significantly since 2014 and, measured by the number of empirical studies, of which a total of 107 (about 63%) studies are of a qualitative and 62 (about 37%) are of a quantitative design, remains high. The peak in 2016 and 2017 is due to a comparatively greater number of pertinent conferences and respective publications in these years. The decline in 2020 may be a result of the coronavirus pandemic, which has disrupted and delayed research projects and funding in general (Callaway et al., 2020 ).

Corresponding to the allocation of qualitative and quantitative empirical studies, the majority of the studies apply qualitative content analyses based on either an individual or comparative approach (61.54%). The application of quantitative methods is consequently lower in total, whereby publications using methods of complex empirical research, such as regression analysis and structural equation modeling, with a combined share of 18.93%, number even fewer, as opposed to publications based on descriptive statistics (19.53%). Figure  2 depicts the distribution of the applied methods of analysis.

Number of studies according to applied method of analysis

Table  2 presents the identified key topic clusters and provides selected descriptive statistics how these key topics have been approached in terms of study type, data collection, and research perspective.

Table  2 shows that the largest share of the research focuses on the key topic (4) OGD outcomes and accounts for 28.99% of the literature reviewed, which is not surprising given the extensive impact of OGD on different performance and success levels. The key topic, with an almost equal number of assigned studies, is the group (3) OGD decisions with 28.40%, followed by the key topics (2) OGD antecedents with 15.98%, and (1) OGD theory with 11.24%. While the share of studies in key topic (6) OGD governance remains in the double-digit percentage range (10.65%), the level of scientific interest measured by the number of publications within key topic (5) OGD impacts is significantly lower (4.73%). Furthermore, like the overall distribution of qualitative and quantitative empirical research occurs the composition with regard to the individual key topic clusters, so that the number of qualitative studies clearly predominates in each key topic. Notably, key topic (5) OGD impacts constitutes an exception, where the exact opposite is the case. This pattern can be explained by the fact that research on OGD is still at a relatively early stage.

In summary, the analysis reveals the great scope and heterogeneity of the research landscape of OGD in terms of research focus and methodology. The pronounced imbalance between qualitative and quantitative studies in favor of the former indicates that OGD is still an emerging field of research. Given this emergent state of research, quantitative empirical studies are essential to confirm causality of theoretical relationships and effects of evolving issues proposed by conceptual or qualitative research, and to address associated concerns of validity. In particular, little empirical robust knowledge is available in the areas of acceptance/satisfaction/trust in government, policies/regulation/law general/conceptual development, and drivers/barriers. This also holds when it comes to understanding the user perspective in the context of OGD, which is generally neglected in the field, but in particular in these areas. A remarkable exception to this pattern is the area of acceptance/satisfaction/trust in government, which has so far only focused on the user perspective, while disregarding the provider perspective. However, this would be especially important in view of the struggling implementation and diffusion of OGD in several public organizations. The user perspective so far has also strongly emphasized the individual level (e.g., citizens) and should increasingly consider the organizational level (e.g., firms) for a better understanding of the role of OGD in the digital economy.

Synthesis of the OGD literature

The synthesis of the OGD literature is based on the theoretical review framework and theoretically integrates the previously identified key topic clusters with reference to the digital economy. Figure  3 depicts the review framework and the theoretical relationships among the identified key topic clusters.

Overarching theoretical review framework

The framework may serve as a thematic relationship map of empirical OGD research, particularly illustrating the associations among antecedents, decisions, and outcomes of OGD, as well respective focus areas of research and neglected topics. The antecedents in terms of the drivers and barriers explain the reasons for a certain behavior, while decisions determine the forms of behavior (i.e. adoption, usage, or implementation of OGD), and outcomes comprise the assessments that result from decisions and the associated behavior (i.e., success, performance, and value or acceptance, satisfaction, and trust in government) (Lim et al., 2021 ). All these processes take place in a governance and regulatory setting, in which policies, regulation, and law may affect this process in terms of institutional moderators. These layers underlie the general and conceptual development of OGD, which is the overarching object of action and knowledge, and thus constitutes the point of reference for all other elements in the framework. The synthesis of OGD literature is conducted along these dimensions in the following.

General conceptual development of OGD

Perspectives on ogd.

Regarding the general conceptual development of ‘Open Government Data’, various studies contrast four ways of perceiving the term in recent years (Alexopoulos et al., 2018 ; Gonzalez-Zapata & Heeks, 2015 ; Jetzek et al., 2013 ): (1) the bureaucratic perspective conceiving OGD as a bureaucratic mechanism to enhance information quality, effectiveness and efficiency of government policy making, and legitimacy of polices (cf. Alexopoulos et al., 2018 ; Gonzalez-Zapata & Heeks 2015 ), (2) the technological perspective conceiving OGD as a technological innovation of public administration building up a data infrastructure to host a freely available public database of accurate, complete, and timely public sector data (cf. McNutt et al., 2016 ; Meijer, 2015 ), (3) the political perspective conceiving OGD as a part of government accountability to the citizens, thus providing insights into government affairs, transparency of governmental action, and the option for civic participation in policymaking (cf. Zhao & Fan, 2018 ; Meijer, 2015 ), and (4) the economic perspective conceiving OGD as source of economic value creation, providing several opportunities for the commercialization of these data in new goods and services (cf. McBride et al., 2019 ; Zhao & Fan, 2018 ; Berrone et al., 2017 ).

The digital economy’s role in the OGD ecosystem

Against the background of the OGD ecosystem model presented by Dawes et al. ( 2016 ), these perspectives of the literature can be interpreted as to portray four fields of stakeholder interactions in OGD settings. In this context, the bureaucratic perspective focuses on the interaction between the policymakers and the implementing authorities by surveilling the effects (increase in the quality of information, the effectiveness of administrative action, the legitimacy of public policy) (cf. Alexopoulos et al., 2018 ), while the political perspective regards OGD as a means for democratic processes and decision-making, as it investigates the role of OGD in government accountability, transparency, and citizen participation. Correspondingly, the technological perspective portrays the interaction between OGD providers (public authorities) and OGD intermediaries (i.e., the digital economy) by stating the technical characteristics of the data infrastructure. The economic perspective, however, lays its focus upon the creation of value for the OGD customers, i.e., the citizens, by investigating how OGD yields public value to them. Against this background, firms of the digital economy assume an intermediary function matching technical data supply from the government with information demand of the OGD customers. In a nutshell, the task of digital firms in the OGD ecosystem is to access the data supplied by the government, to gather the information contained in OGD by electronic data processing and analytics software, and to commercialize this information in their products and services. Figure  4 outlines the OGD ecosystem and sketches the role of the digital economy as a data intermediary facilitating the interaction between the executive government authorities and the citizens.

Open government data ecosystem (based on Dawes et al., 2016 and Kassen, 2013 )

Scope of government activity and the digital economy

Besides the general role of the digital economy, both the scope of digital business opportunities and the business approach are crucial to the digital economy. In this context, the literature raises interesting points regarding the scope of government activity in data-based service provision. Some studies find evidence for governments simply providing public data and setting the legal and technical framework by data formats and access rights, while leaving further processing and marketizing of these data completely to interested stakeholders, like NGOs, companies, or private citizens (cf. Alexopoulos et al., 2018 ; Berrone et al., 2017 ; Dawes et al., 2016 ; McNutt et al., 2016 ). However, another strand of OGD literature finds more complex forms of governmental open data platforms, providing data via APIs and data-based apps that enable the user to filter and manipulate the chosen data set and to embed the data in other data processing programs (cf. McBride et al., 2019 ; Zhao & Fan, 2018 ; Berrone et al., 2017 ). In this case, government provides OGD products and services on its own in competition to possible private sector offerings. In this context, contemporary OGD research presents a spectrum of government involvement in the presentation and processing of publicly accessible data by presenting diverging roles of government in OGD programs, i.e. data provision and standard-setting versus data service platform hosting. Consequently, the scope of government activity and the sophistication of governmental data infrastructures for the compilation, analysis, and provision of public sector data significantly influences the economic margin and targets of digital private business with OGD. Besides the theoretical setting of the digital economy’s role in the OGD ecosystem, the antecedents of OGD programs, the decisions and actions taken by the government for OGD implementation, as well as the achieved outcomes and impacts also determine the position of the digital economy in OGD programs and how to create value from public sector data.

The following subsection provides a synthesis of the findings of previous research on the antecedents, decisions, and outcomes of OGD with special reference to the digital economy, elaborating their significance for IS and digital business research (Table  1 in the online Appendix summarizes these findings). The representative studies presented in the following subsection (and in Table  1 in the online Appendix) were selected due to their high resonance in scientific research (high Google Scholar citation score) and their publication in particularly influential scientific, peer-reviewed journals (high journal impact score).

Antecedents, decisions, and outcomes of OGD and the digital economy

Ogd antecedents: drivers and barriers.

When considering the antecedents and determinants of OGD programs, previous studies more often refer to barriers emerging from the OGD ecosystem (cf. Barry & Bannister, 2014 ; Janssen et al., 2012 ; Ruijer et al., 2017 ), rather than the drivers and enablers (cf. Young, 2020 ; Zhenbin et al., 2020 ; Susha et al., 2015 ). For the factors triggering or fostering OGD policies, the findings of previous studies distinguish among political and social factors, operational and technical properties of agency equipment, or economic opportunities for OGD usage. In case of political and social OGD determinants, political and social demand for transparency and accountability (Barry & Bannister, 2014 ; Janssen et al., 2012 ; Zhenbin et al., 2020 ) is perceived as a major trigger for OGD programs alongside with increasing citizen engagement and participation in government affairs (Young, 2020 ; Welch et al., 2016 ). Regarding the operational and technical drivers, previous studies highlight the importance of a cultural anchorage of electronic data processing and sharing in public administration (Zhenbin et al., 2020 , Yang et al., 2015 ) in combination with a well-developed data infrastructure within the agency operated by qualified specialists (Young, 2020 ; Welch et al. 2016 ). In this context, economic pressure arises from a large share of private companies providing public services to the citizens for profit. Studies such as Young ( 2020 ) find that the opportunity to augment extant or create new public services by using public sector data bears opportunities to create new sources for economic growth (cf. Young, 2020 ; Zhenbin et al., 2020 ; Susha et al., 2015 ). This is even more the case if the national economy possesses the resources for exploiting the information contained in public sector data (high GDP) and exhibits a large productivity in providing ICT services (high share of the IT industry) (cf. Young, 2020 ; Susha et al., 2015 ). In this context, the state of the digital economy as well as the maturity of governmental data infrastructures appear as drivers for both the successful implementation of OGD programs and the successful exploitation of these data in public services. Consequently, IT firms thus function as software and hardware suppliers to public administration in digitally underdeveloped economies, while they assume the role of a private sector competitor in the delivery of public services in digitally advanced countries.

Barriers to implementing an OGD policy emerge from problems with (1) data compilation on the part of the government or the executive agencies (institutional constraints), with (2) data access caused by technical failures or dysfunctional data portals (technical constraints), or with (3) data application on the part of the citizens (societal barriers). Accordingly, data compilation barriers refer to factors that hinder the respective agencies to collect, compile, or transfer suitable data due to legal constraints (Yang et al., 2015 ; Barry & Bannister, 2014 ), due to the complexity of the organizational structures of government agencies (Ruijer et al., 2017 ; Welch et al., 2016 ; Yang et al., 2015 ), and/or due to the lack of their data management capacities and capabilities (Ruijer et al., 2017 ; Young, 2020 ). In contrast, data access barriers emerge from the properties of the data infrastructure. Major impediments in data access arise from a lack in system interoperability if governmental software and data formats are not compatible with its civic counterparts (Smith & Sandberg, 2018 ; Barry & Bannister, 2014 ) or from a lack in technical support and constant updating of data platforms due to staff shortages (Janssen et al., 2012 ). Furthermore, the literature also finds that the introduction of registered access to public data creates another great obstacle for OGD as most people are unwilling to register officially on public data platforms for occasional data access (cf.Barry & Bannister, 2014 ; Ruijer et al., 2017 ). Regarding the obstacles emerging from the properties of the user, i.e. the citizens, previous research argues that the success of OGD programs is to be attached to the ability of society to make use of the published data. Obstacles emerge from the inability of the users to achieve a practical use of these data; this might either be due to the societal inability of information processing (e.g., low ICT equipment, low levels of education, low income, etc.) (Barry & Bannister, 2014 ; Ruijer et al., 2017 ), or due to the uselessness of the provided data such that the citizens cannot apply the information to achieve any value (Smith & Sandberg, 2018 ; Janssen et al., 2012 ). Considering these findings, all barriers provide starting points for digital business to step in and solve the issue. In case of data compilation constraints, IT firms adapt solutions from private sector products and services to provide a customized data infrastructure to public authorities aiming to publish their data. To overcome data access barriers, private IT firms host government data for public retrieval as business partners of public authorities and provide the information via their own data services and applications. Finally, to solve data application barriers, the digital economy provides IT specialists and data analysts processing government data and create a useful summary and analysis of OGD for the citizens.

OGD decisions: Adoption, usage, and implementation

Although the relevant drivers and obstacles open corresponding business opportunities for the digital economy, actual policy decisions regarding the adoption of OGD measures, as well as their implementation and subsequent use, are of crucial importance for business practice. As stated before, government activity in providing data-based applications to its citizens is of major importance for the type of digital business. Accordingly, previous research analyzed the decisions regarding OGD policy and strategy as well as the intensity of governmental OGD activities (Gascó-Hernandez et al., 2018 ; Dawes et al., 2016 ). Depending on the scope of governmental data processing and data-based service provision, Dawes et al. ( 2016 ) propose a spectrum of OGD policies presenting three archetypes of OGD strategy, starting with (1) the data-oriented OGD policy aiming at the provision of accurate, unbiased datasets from public sector entities without any further service features (cf. Wang & Lo, 2016 ; Yang & Wu, 2016 ), followed by (2) the intermediate program-oriented OGD policy providing public data via an OGD platform displaying basic data analysis features and APIs (cf. Chatfield & Reddick, 2017 ; Parycek et al., 2014 ), ending up with (3) the use- and user-oriented OGD policy focusing on the creation of public value by embedding public sector data within data-based public services (Gascó-Hernandez et al., 2018 ).

Despite these strategic considerations, governmental adoption decisions also have a major impact upon the organizational and technical preparations to get public administration ready for OGD (Chatfield & Reddick, 2017 ; Yang & Wu, 2016 ; Parycek et al., 2014 ). Closely connected to the strategic setting is the scope of publication permissions from high-level authorities ranging from data publication restrictions to the support of interactive data services. Furthermore, the government’s adoption decisions also shape the maturity of the authorities’ data infrastructure by defining the technical capacity as well as the interoperability and connectivity to citizen devices (Bonina & Eaton, 2020 ; Wang & Lo, 2016 ). Consequently, the ex-ante decisions regarding the adoption of OGD measures also define the way of doing business with OGD. In this regard, the strategic positioning of governmental OGD activities directly determines the scope of the intermediary role of the digital economy. In case of a data-oriented OGD program relying upon a mediocre public data infrastructure, the intermediary role of the digital economy achieves its climax as the government acts as a proper data provider, leaving data analysis, application, and embedment in public services completely to digital firms. However, privatization of data-based public services diminishes if the OGD program place special emphasis upon the user. For a user-oriented OGD program equipped with a well-developed public data infrastructure, utilizing OGD for providing data-based public services is completely in the hands of the government, whereas IT firms provide IT expertise and software solutions to the authorities.

Besides the determining character of ex-ante decisions for digital business with OGD, the ex-post decisions of the government flanking the OGD program also provide opportunities for the digital economy. Linked to the strategic setting of the OGD program is the decision for the target group and user profile of the program (Smith & Sandberg, 2018 ; Parycek et al., 2014 ). Depending on the respective policy intensity, government must decide whether (1) to grant general access for the average citizen in case of a user-oriented approach, or (2) to grant licensed commercial access enabling the embedment of OGD in the products and services offered by private IT firms in case of a program-oriented OGD approach, or (3) to grant access only to IT specialists for retrieving information via data analytics in case of a data-oriented OGD approach.

Furthermore, previous research also investigated the ensuing decisions concerning the interface design and the related features of OGD portals (Wirtz et al., 2019 ; Chatfield & Reddick, 2017 ). Accordingly, OGD portals diverge in the scope of the provided datasets, in the scope of the OGD interface as well as the scope of data service functions, ranging from mere data downloads from government websites to data service hubs created by OGD platforms. As a result, the user profile targeted by the OGD program as well as the design and features of the OGD interface shape business approaches for OGD. Accordingly, IT firms seek to gather, process, and capture value by commercializing OGD in products and services for the citizens in case of a licensed access and a low scope of OGD data service features, responding to the demand of proper data processing on the demand side of the OGD ecosystem. In case of limited specialist access and a high scope of data service functions, IT firms switch towards offering data analytics services to the authorities involved, equivalently responding to the demand of supply-sided data processing and analytics (cf. Bonina & Eaton, 2020 ).

Another relevant field for government decisions flanking the implementation of OGD programs refers to the creation of IT skills and technical expertise required for data management by public authorities (Gascó-Hernandez et al., 2018 ; Wirtz et al., 2019 ; Yang & Wu, 2016 ). Regarding the timescale and the addressees of these measures, current research distinguishes between short- to mid-term educational measures for public employees developing OGD skills and capabilities (cf. Safarov, 2019 ; Yang & Wu 2016 ) and long-term educational measures, increasing common IT knowledge among the population (cf. Gascó-Hernandez et al., 2018 ; Wirtz et al., 2018 ). Short- to mid-term OGD skill development is associated with a variety of options, ranging from internal IT trainings with the respective authorities (Yang & Wu, 2016 ) to joint ventures with the digital economy (Safarov, 2019 ). This decision area thus offers several linkages to digital business, spanning from the provision of training programs for public administration to learning-on-the-job in collaborative partnerships for OGD processing and evaluation. Regarding long-term public IT schooling, the government aims at building up IT skills and capabilities among the population in order to gain skilled employees for public administration (cf. Gascó-Hernandez et al., 2018 ). As a result, private-sector IT companies sell their know-how and IT expertise to educational institutions as mentoring partners for IT practice. All in all, the digital economy assumes the role of a catalyst in the field of digital education and training of the people - as trainers and administrative partners in the short term and as mentors in the long run.

OGD outcomes: Success, performance, and value

Finally, it is of crucial importance not only to the government and public administration whether an OGD program pays off in terms of efficiency, citizen satisfaction, and trust in government. For the digital economy, the question is whether accessing and utilizing OGD provides access to new products and services as well as whether OGD can create new markets for data-based public services. Regarding the outcomes achieved by OGD implementation, most studies refer to the internal effects upon the performance of public administration, such as efficiency gains in administrative procedures and public service provision (Mergel et al., 2018 ; Worthy, 2015 ), transparency of political decisions and policy-making (Wang & Shepherd, 2020 ; Marjanovic & Cecez-Kecmanovic, 2017 ; Jetzek et al., 2014 ), or behavioral effects upon public employees (Marjanovic & Cecez-Kecmanovic, 2017 ; Worthy, 2015 ).

In contrast to these specific administrative and political issues, some studies also refer to spill-over effects upon the interaction of citizens with public authorities (interaction effects), the distribution of information among the population (information effects), as well as the innovation of public services by utilizing OGD (commercialization/innovation effects). Considering interaction effects upon the participation and involvement of citizens into public affairs, previous studies observe a positive effect in citizen engagement in case of OGD programs. Although there is evidence of negative OGD effects upon the polarization in political debates due to different interpretations of government data (cf. Worthy, 2015 ), most studies report positive effects, such as public service innovation through co-creation with citizens and IT firms or synergy effects due to simplified data sharing in collaborations between government agencies and external service providers (Ruijer & Meijer, 2020 ; Máchová & Lněnička, 2017 ; Jetzek et al., 2014 ). Having this mind, interaction effects of OGD programs enable the digital economy to serve as a moderator, facilitating the interaction between government and citizens by easing information processing on the part of the citizens and communication to the citizens on the part of public administration. Furthermore, IT firms relying upon big data analytics might experience competitive advantages in comparison to their international competitors as the cost for gathering public sector data decreases significantly. Consequently, citizen engagement and data sharing provide economic growth potentials to the digital economy. This is also in line with the commercialization and innovation effects observed by several studies (Jetzek et al., 2019 ; Mergel et al., 2018 ; Jetzek et al., 2014 ). Accordingly, previous research finds evidence for OGD spillover effects to the private sector, as implementing OGD enables digital firms to access new information at lower cost, and to generate a footage in the public sector by developing new markets for data-based products and public services.

Acceptance, satisfaction, and trust in government

Considering the consequences on technology acceptance and citizen satisfaction triggered by OGD, previous research observes a positive impact fostered by several preconditions. In case of technology acceptance, studies find that a positive impact relies upon (1) sufficiently intense Internet usage among the population (Gonzálvez-Gallego et al., 2020 ; Afful-Dadzie & Afful-Dadzie, 2017 ), (2) the awareness of individual benefits that emerge when using and applying OGD (Zuiderwijk et al., 2015 ; De Kool & Bekkers, 2014 ), and (3) the degree of OGD usage obligation in G2C interactions (Gonzálvez-Gallego et al., 2020 ; Zuiderwijk et al., 2015 ). Considering citizen satisfaction, broad acceptance and public support of OGD and its application appear as necessary conditions alongside with a sufficiently high information quality, system quality, and service quality (cf. Gonzálvez-Gallego et al., 2020 ). Hence, the maturity of a country’s digital economy directly moderates the impact of OGD on technology acceptance and citizen satisfaction. This is due to developed digital economies displaying both a widespread use of ICT devices and their intensive usage, as well as common IT knowledge among the people. In addition, resident digital firms are in a much better position to support a well-functioning public data infrastructure in the case of an advanced IT industry.

In summary, it can be stated that from the perspective of public administration, the digital economy constitutes both a driver of OGD adoption and a warrant for successfully implementing an OGD program. From the perspective of the digital economy, however, OGD represents a new source of economic growth and business model innovation based upon the development of new resources, i.e., public sector data, and new business opportunities emerging during OGD adoption and implementation.

• Research agenda

The preceding identification of OGD key topic clusters and their synthesis into a theoretical framework with special reference to the digital economy has revealed significant points of connection to IS and digital business research and enables us to develop a theory-informed research agenda for the latter. Although the prior literature review emphasized particularly the core dimensions of the ADO framework, the findings also yield implications for the key topics (1) OGD theory and (6) OGD governance.

(1) OGD theory: General/conceptual development

As the OGD ecosystem theorizes that firms of the digital economy assume an intermediary function matching technical data supply from public authorities with the demand for information on the part of the citizens, empirical research needs to verify how this assumption holds true in practice. Furthermore, future research needs to clarify the impact of government activity and OGD infrastructure maturity upon the business models of related IT firms. Consequently, McBride et al. ( 2019 ) postulate the need for further empirical research, which would enable comparison and differentiation of individual OGD services in their emergence, orientation, and goals. McBride et al. ( 2019 ) consider this especially with regard to data platforms and OGD services, which increasingly evolve from different sources. This corresponds with the implications pointed out by other researchers who identify further needs for empirical research on the characteristics of OGD sources in connection with different national contexts (Alexopoulos et al., 2018 ), data platforms collaboratively developed in joint ventures with IT firms (Meijer & Potjer, 2018 ), and the changes in the OGD portals’ datasets over time (Di Wang et al., 2018 ). Hence, more empirical research is needed, especially case studies regarding the economic OGD perspective, to determine the scope of involvement of private IT firms in OGD programs in general as well as their function within the whole OGD ecosystem in practice. Despite that, the scope of government activity in data-based service provision needs further investigation regarding its impact on the business approach of the digital economy. Consequently, the following questions may guide further research in this direction: How are firms of the digital economy involved in contemporary OGD programs? What is the function/business of digital IT firms in respective OGD programs? How does the scope of governmental OGD activity alter the business model of digital firms?

(2) OGD antecedents: Drivers/barriers

Considering external OGD drivers and barriers, the preceding analysis of OGD research revealed the productivity of the IT industry, as well as the GDP share of the digital economy as key drivers of successful OGD programs. Thus, establishing a causal linkage between the size of the IT industry, the share of the digital economy, and the maturity of OGD programs appears as a suitable goal for further empirical research. Linked to this idea is also the idea of Shao and Saxena ( 2019 ) raising the question of how a society’s cultural characteristics and traditional values act as drivers and/or barriers to the intentions of administrative implementation and the participation of external actors within OGD initiatives. Consequently, the following research questions appear as a good starting point for analyzing OGD drivers and barriers emerging from the digital environment: Does a high productivity of IT firms and large share of the digital economy increase the success of OGD initiatives? Which socioeconomic, demographic, and cultural characteristics of the economy drive or impede OGD implementation?

Turning towards drivers and barriers from inside public authorities, Zhenbin et al. ( 2020 ), for instance, name the need to further investigate which specific drivers influence the motivation of government agencies to engage in OGD development and public service innovation. This has been similarly formulated by Fan and Zhao ( 2017 ), who, in addition to examining the question of which influences generally exert pressure on the internal, organizational orientation in relation to OGD activities, also emphasize the need for further research on the extensive influence of the media. With regard to policy constraints, Young ( 2020 ) identifies the risk within public institutions of intentionally withholding data/information that could be detrimental to the publisher and postulates the need to investigate more closely the existence of these barriers and their potentially negative consequences in the future. Considering the findings from the qualitative literature synthesis, the question arises as to whether collaboration with private IT companies results in a reduction of barriers or an activation of drivers within the agency. This could be empirically determined and investigated in particular by means of interviews and questionnaires. Possible research questions in this direction would be: To what extent do data access, data processing, and data application in public services improve due to collaboration with private IT companies? To what extent do intensive G2B interactions regarding OGD contribute to its successful implementation?

(3) OGD decisions: Adoption/usage/implementation

While synthesizing the findings of previous studies, it became clear that the strategic positioning in OGD adoption, the target groups for OGD usage, as well as the organizational OGD readiness for OGD implementation have a significant impact on the orientation of the corresponding OGD business models. In light of these findings, two promising directions of research emerge for the IS research community investigating OGD in the context of the digital economy: (1) the empirical verification of the assumed correlation between the user-orientation of governmental OGD initiatives and the predominant customer alignment of IT firms’ OGD business models, and (2) the case-study-based investigation of the causal relationship between OGD access barriers and the share of the digital economy in providing data-based public services. Overall, the need for further, user-focused research is obvious and acknowledged. For example, there is a need to identify the types of datasets users of OGD require in order to enable even more active participation and usage (Chorley, 2017 ) and to understand how external users can be motivated to become permanent participants in OGD, while respecting their job situation and other cultural influences (Hermanto et al., 2018 ). Smith and Sandberg ( 2018 ) also point out that instead of the usual data-centric research, more user-centric OGD research is needed in future. In this context, the established theories of IS and digital business research such as the Technology Acceptance Model (TAM), the Unified Theory of Acceptance and Use of Technology (UTAUT), and the DeLone-McLean IS Success Model become particularly important for the further development of this field of research. Accordingly, the following research questions may guide scholars in conducting further research concerning the OGD adoption and usage approaches: How can IS theories and explanatory models, in particular, the TAM, UTAUT, and IS Success Model be applied in the context of OGD research and theory development to explain acceptance, adoption and usage behavior? How does governmental customization of OGD alter the value proposition and customer composition of OGD business models? What is the impact of OGD access restrictions on the business practices of the IT firms involved?

Another interesting avenue for further research connecting OGD to IS and digital business studies is the topic of building up relevant OGD skills and educational support. The findings from the literature synthesis suggest the digital economy to serve as a catalyst in digital education providing skills and knowledge in the short run, and innovative spirit and educational support in the long run. In this respect, Safarov ( 2019 ) points out that it might be useful to examine in more detail the design and impact of various OGD activities, such as open data awards or specific training programs. Several other researchers also discuss the necessity and value of findings based on integrative methods and trainings regarding the implementation and usage of OGD. In this way, among other things, experimental studies can be performed to determine which training methods can be used most successfully in relation to specific content and data sets in order to ensure a lasting curiosity and interest in OGD (Gascó-Hernández et al., 2018 ).

Further long-term studies will also show how government institutions’ perceptions and usage behavior change over time as the methods are compared (Altayar, 2018 ; Wang & Lo, 2016 ). Wirtz et al. ( 2018 ) postulate the need for further research to examine the degree to which the usage behavior of citizens changes over time and which situational and socio-cultural aspects play a role in this process. In this respect, in addition to the use of longitudinal studies, comparative cross-cultural or cross-country studies can also be used to identify relevant differences and investigate their consequences for user behavior (Saxena, 2018 ). Considering these demands for further research, the following research questions may inspire research regarding the role of the digital economy in creating digital OGD literacy: Do G2B partnerships in OGD increase the digital literacy of public employees? Do OGD training programs and educational measures have a greater effect on the trainees if education involves cooperation with IT firms?

(4) OGD outcomes: Success/performance/value

Since current research on OGD outcomes is concerned with the question of how OGD offers socioeconomic added value to society, there are also potential spin-offs for the digital economy in this context. In the preceding literature synthesis, it became clear that the establishment of OGD programs could generate spillover effects on the competitiveness and innovative strength of the digital economy. Accordingly, the empirical investigation of these effects by means of case studies and time series analyses appears to be a promising goal for further research. Specifically, the following research questions suggest themselves in this context: How does the successful implementation of OGD initiatives affect the competitiveness of IT firms? Is there evidence for a causal relationship between the implementation of OGD programs and economic growth in the digital economy?

However, answering these specific research questions depends largely on the ability to record and evaluate the performance and resultant success of OGD activities. Since the success of OGD activities to be determined or measured extends to many areas among public institutions and external stakeholders, it is generally difficult to comprehensively classify and evaluate success and failure. In response to the challenges posed by the above-mentioned reasons, Marmier and Mettler ( 2020 ) postulate the need for additional research on the level of dedicated quality measurement and evaluation of OGD and its measurement instruments. Similarly, Jetzek et al. ( 2019 ) argue that the answer to the question of how data constructs and their quality are to be measured at the societal level poses another future research need.

Another relevant issue involves the potential value contribution of OGD and describes the need for further research to identify the potential contribution of OGD activities in terms of overall value creation in terms of social, economic, and public value. The origin of this value creation lies in the fact that data from public institutions are first made available in an appropriate quality, wherefrom Luna-Reyes et al. ( 2019 ) derive the need for further research to identify suitable governance and leadership approaches and to examine their influence on the quality of the data to be emitted. Mergel et al. ( 2018 ) further emphasize the large amount of valuable innovations that can be triggered by OGD and point out the need for further research in this regard to broaden and strengthen existing knowledge. Magalhaes and Roseira ( 2020 ) present similar points and show that, albeit the increasing recognition of the potential value for the private business sector, the reasons for or against integrating OGD into business processes, and thus also the potential economic value that can be achieved, still often remain unexploited or even unclear. They emphasize the need for further in-depth analysis at the firm level in order to move from a general top view to explicit insights into the behavior of and consequences for firms in their interactions with OGD. A research question of central importance might consequently be: How can dedicated products and processes be explored and exploited in order to generate sustainable economic and public value in different OGD contexts?

(5) OGD impacts: Acceptance/satisfaction/trust in government

The synthesis of the existing literature on the topic of the consequences and impacts of OGD programs on the general acceptance of OGD, the satisfaction of citizens with its use, as well as the resulting trust in government policy suggests that these impacts are all the stronger in case of a well-developed digital economy. As argued above, this is due to (1) widespread usage of ICT devices among the population, (2) IT-related customer preferences and usage perceptions, and (3) technical support from private IT firms. Taking this implication as a starting point for further research raises the following questions: Does the maturity of the digital infrastructure moderate OGD acceptance and user satisfaction? Do joint ventures of government and private IT firms providing OGD services to the public increase trust in open government?

Against this background, the investigation of external stakeholders’ perceptions and preferences is of central importance and determines the need for further research to explore and scrutinize the differing perceptions and preferences of various stakeholders in terms of OGD activities and outcomes by international comparison. Further research efforts should therefore be undertaken to examine and compare preferences and perceived satisfaction at both the citizen (Saxena & Janssen, 2017 ) and corporate levels (Afful-Dadzie & Afful-Dadzie, 2017 ). Due to the small number of studies dedicated to OGD impacts, it is of interest to broaden the focus from the external stakeholders to an in-depth investigation of the acceptance and satisfaction of governmental agencies’ internal forces, as these act as a starting point or barrier to subsequent external perception and satisfaction (Barry & Bannister, 2014 ). Consequently, scientific progress within the field of OGD antecedents might also spark research efforts in OGD impacts.

(6) OGD governance: Policies/regulation/law

Following the research implications regarding the strategic alignment of OGD programs and the corresponding OGD policy intensity, two research areas become apparent within which further research efforts can contribute to a better understanding of the specific context: the normative composition and implementation of OGD and the potential impacts of norms and policies. For the research area of normative composition and implementation of OGD it is stated that the far-reaching innovations for the state and the economy emerging from the implementation and use of open data in general and OGD in particular, require dedicated and appropriate policies from state authorities. Thus, Khurshid et al. ( 2019 ) state that in the future it will be important to understand the reasons for slow diffusion and a consequently weak adoption of general data policies at the organizational and individual levels. Furthermore, procedural metadata standards and general data quality standards should be preceded by further research (Máchová & Lněnička, 2017 ; Shepherd et al., 2019 ).

In addition to general overview studies, further in-depth analyses of applied standards and directives should be conducted in the future, which in turn will help to provide stronger guidelines for the development of data policies. Regarding the potential impacts of norms and OGD policies, further research is needed to determine how the formulation and implementation of data policies and normative guidelines affect other core aspects, such as subsequent use or the general contribution to success (Kurtz et al., 2019 ). Moreover, it is necessary to investigate, how specific policies that focus on the commercial value of OGD contain the risk of conflict with other open data values (Zuiderwijk et al., 2016 ). In order to identify and classify corresponding dependencies and consequences in this context, comparative and qualitative exploratory approaches are promising to derive conclusions from related policies and directives.

In summary, a number of starting points for IS and digital business research emerge from the findings and insights of previous studies on the various OGD research areas. In the context of the consideration within the ADO framework, various parallels between the identified research questions also become apparent. To provide a general overview of these research implications, Fig.  5 reflects the relevant research questions and depicts their integration into the theoretical review framework in terms of a research agenda for IS and digital business research.

Theory-informed research agenda for IS and digital business research

Discussion and conclusion

Data have become an inherent part and essential driver of the digital economy. The field of OGD has been largely neglected by IS and digital business research, despite its great value potential for firms and the digital economy as a whole. As governments, public organizations, and firms worldwide are struggling in exploiting the full potential of OGD for the digital economy, it is essential to gain a comprehensive understanding of OGD and to frame the concept more broadly in the context of the digital economy in order to advance the field of research accordingly. On the one hand, this particularly requires greater involvement of the IS community in the very interdisciplinary field of OGD research, which is currently dominated by the public administration and public management perspective. On the other hand, it is necessary to theoretically integrate and synthesize the vast body of knowledge to identify research gaps and provide valid research directions.

An important requirement to achieve this is first and foremost conceptual clarity of OGD, which sometimes has been confounded with the related concepts of open government and open data. Our study goes beyond prior research (e.g., Hossain et al., 2016 ; Tai, 2021 ; Wirtz & Birkmeyer, 2015 ) by demonstrating and taking account of the – widely implicitly and tacitly assumed – conceptual autonomy of OGD and acknowledging it as an independent research stream closely related but still distinct from open government and open data research. This is a vital prerequisite for drawing differentiated and valid conclusions for the field and for gaining a clear understanding of the phenomenon. In this connection, we further build on and extend the general conceptual development of OGD and respective studies (e.g., Dawes et al., 2016 ; Kassen, 2013 ) by consolidating different OGD perspectives from the literature and by outlining the role of the digital economy in the OGD ecosystem and the digital economy’s relation to OGD-related government activity.

While previous research has made valuable contributions in structuring the OGD research landscape (e.g., Saxena, 2018 ; Zuiderwijk et al., 2014 ) and analysing certain OGD issues (e.g., Attard et al., 2015 ; Purwanto et al., 2020 ; Safarov et al., 2017 ), it fails to theoretically integrate the OGD concept and its key issues, and neglects the increasingly relevant relationship between OGD and the digital economy.

This study seeks to fill in this gap by conducting a systematic literature review of empirical OGD studies, which synthesizes the body of knowledge into a theoretical framework of OGD antecedents, decisions, and outcomes with special reference to the digital economy, and which further proposes a theory-informed research agenda for IS and digital business research.

Against this background, this study generally stands in line with and extends the findings of earlier comprehensive review approaches towards OGD literature, in particular those of Zuiderwijk et al. ( 2014 ) and Saxena ( 2018 ). However, these studies lack in the coherent linkage and the display of causal relationships between the different research areas as these studies mostly follow a descriptive approach attempting to present a common denominator of the characteristics of the individual studies. This study goes beyond their purely descriptive perspective by developing an overarching theoretical review framework that models the theoretical relationships of the thematic clusters identified in the literature analysis. In addition, this study also captures the more recent developments and novel empirical insights in the field of OGD. This is especially true for the area of OGD outcomes, for which research is based on a mature implementation of OGD systems in administrative practice, but also when it comes to issues such as organizational readiness and OGD skill development in the area of OGD decisions. Moreover, by examining the OGD literature with special reference to the digital economy, our study conceptually intersects with relevant IS and digital business research, demonstrating an interdisciplinary research approach that has been missing in prior OGD literature reviews.

Taken together, the theoretical attempt in conjunction with the focus on the digital economy and the associated inclusion of an IS perspective constitutes a new approach towards OGD literature that yielded novel insights into the field by integrating and explaining scientific progress in emergent topics such as in the areas of OGD decisions and OGD outcomes. Thus, the theoretical contribution of our study to the literature in terms of originality results from the theoretical review framework that theoretically integrates previously separated thematic clusters of OGD and their points of connection to IS and digital business research, thus improving our theoretical knowledge of the field of OGD and its relation to the digital economy. Overall, the synthesis of OGD literature into this theoretical framework represents the main response to our first research question of what we know about the antecedents, decisions, and outcomes of OGD and their relations in the context of the digital economy.

In this context, bridging the gap to digital business is of particular importance as this study represents the first attempt to transfer findings and insights from the mainly public administration- and public management-driven OGD studies to the IS and digital business research domains which might spark further progression in OGD research. The research agenda derived in accordance with the theoretical framework reveals how OGD research may relate to adjacent fields of IS and digital business research, such as interface design, IT and data governance, data security, big data analytics, open data, etc., and provides concrete opportunities and research questions in each thematic cluster.

Although the review provides valuable insights into each of the six key topics, the OGD outcomes appear to be of particular importance. This is not only indicated by the fact that this cluster already comprises the largest number of studies in relation to the other clusters, but also in view of very fundamental unresolved issues pertaining to the digital economy. We know today that the use of OGD opens up far-reaching opportunities for developing innovations and improving operational and business processes, for both the public and the private sector. Notwithstanding the awareness of those opportunities and increasing research on the potential benefits, the level of knowledge regarding how best to exploit and leverage economic value remains in many respects at incomplete (Magalhaes & Roseira, 2020 ; Ruijer & Meijer, 2020 ; Zuiderwijk et al., 2014 ). In particular in this context, but also in any of the other key topics, the research avenues identified indicate that OGD research may greatly benefit from the so far underrepresented IS and digital business perspective. As such it may serve as an important tool to build the bridge from OGD to IS and digital business research.

Overall, the research agenda synthesizes the answers to our second research question of how IS and digital business research can inform OGD research, in particular with regard to its role in the digital economy. The theoretical contribution of our study in terms of utility stems especially from the systematization of the complex and heterogeneous research landscape of OGD, as well as the theory-informed research agenda. The latter makes the field more accessible and tangible for IS and digital business research by showing what issues may be studied and how they are related.

However, our study is not without limitations. Merging information obtained from research databases bears a certain risk associated with information technology limitations and time delays that may prevent the full scope of relevant studies from being represented. In addition, our final sample is limited to studies in English language, which means that we may have missed potentially relevant studies in other languages. Bearing in mind that a complete selection is hardly feasible in terms of practicality and that the literature work on which this study is based was generated with respect to well-established methodological guidelines (Rowe, 2014 ; Webster & Watson, 2002 ), we are nevertheless convinced of the sufficient coverage and informative value provided by our relevant set. In addition, our analysis is limited to empirical studies and does not take account of conceptual approaches. Future research could examine whether the review framework also hold true in this connection and how empirical and conceptual OGD research differ in their distribution across the different key topics .

While our systematization and analyses enhance the level of lucidity and understanding with regard to the overall context of OGD, it should be noted that the six identified key topics require further dedicated attention in order to thoroughly interpret and understand the insights of the respective subareas. In this connection, it should further be noted that some of these topics have also been discussed in related research areas, in particular the more general field of open data, which have not been part of our literature review. Future research could synthesize these research streams and examine how they complement our findings. Finally, our comprehensive approach inherently goes at the expense of a detailed examination and discussion of each key topic. Although the majority of literature reviews on OGD have focused on a special key topic, it remains an important task for future studies to scrutinize recent, widely unexplored subtopics in OGD research, such as innovation and value creation.

In conclusion, although OGD has accumulated a substantial body of knowledge over the last decade, the field is still in an emerging stage and calls for further research to provide answers to a variety of important unresolved issues from an IS perspective. This systematic literature review contributes to a comprehensive understanding of OGD and may serve as a suitable reference point and impetus in bridging the gap between OGD and IS research and exploiting the potential of OGD for the digital economy.

Afful-Dadzie, E., & Afful-Dadzie, A. (2017). Open government data in Africa: A preference elicitation analysis of media practitioners. Government Information Quarterly, 34 (2), 244–255. https://doi.org/10.1016/j.giq.2017.02.005

Article   Google Scholar  

Alexopoulos, C., Loukis, E., Mouzakitis, S., Petychakis, M., & Charalabidis, Y. (2018). Analysing the characteristics of open government data sources in Greece. Journal of the Knowldege Economy, 9 (3), 721–753 https://search.ebscohost.com/login.aspx?direct=true&db=eoh&AN=1726821&site=ehost-live

Altayar, M. S. (2018). Motivations for open data adoption: An institutional theory perspective. Government Information Quarterly, 35 (4), 633–643. https://doi.org/10.1016/j.giq.2018.09.006

Attard, J., Orlandi, F., Scerri, S., & Auer, S. (2015). A systematic review of open government data initiatives. Government Information Quarterly, 32 (4), 399–418. https://doi.org/10.1016/j.giq.2015.07.006

Bachtiar, A., Suhardi, & Muhamad, M. (2020). Literature review of open government data. 2020 International Conference on Information Technology Systems and Innovation (ICITSI) (pp. 329–334).

Barry, E., & Bannister, F. (2014). Barriers to open data release: A view from the top. Information Polity: The International Journal of Government & Democracy in the Information Age, 19 (1/2), 129–152. https://doi.org/10.3233/IP-140327

Berrone, P., Ricart, J., & Carrasco-Farré, C. (2017). The open kimono: Toward a general framework for open data initiatives in cities. California Management Review 59 (1), 39–70. https://doi.org/10.1177/0008125616683703

Bogdanović-Dinić, S., Veljković, N., & Stoimenov, L. (2014). How open are public government data? An assessment of seven open data portals. In M. P. Rodríguez-Bolívar (Ed.), Public administration and information technology. Measuring E-government efficiency (pp. 25–44). Springer New York.

Chapter   Google Scholar  

Bonina, C., & Eaton, B. (2020). Cultivating open government data platform ecosystems through governance: Lessons from Buenos Aires, Mexico City and Montevideo. Government Information Quarterly, 37 (3). https://doi.org/10.1016/j.giq.2020.101479

Borgesius, F. Z., Gray, J., & van Eechoud, M. (2015). Open data, privacy, and fair information principles: Towards a balancing framework. Berkeley Technology Law Journal, 30 (3), 2073–2131. https://doi.org/10.15779/Z389S18

Callaway, E., Ledford, H., Viglione, G., Watson, T., & Witze, A. (2020). COVID and 2020: An extraordinary year for science. Nature, 588 (7839), 550–552. https://doi.org/10.1038/d41586-020-03437-4

Chatfield, A., & Reddick, C. (2017). A longitudinal cross-sector analysis of open data portal service capability: The case of Australian local governments. Government Information Quarterly, 34 (2), 231–243.

Chorley, K. M. (2017). The challenges presented to records management by open government data in the public sector in england. Records Management Journal, 27 (2), 149–158. https://doi.org/10.1108/RMJ-09-2016-0034

Criado, J. I., Ruvalcaba-Gómez, E. A., & Valenzuela-Mendoza, R. (2018). Revisiting the open government phenomenon. A Meta-analysis of the international literature. JeDEM - eJournal of eDemocracy and Open Government, 10 (1), 50–81. https://doi.org/10.29379/jedem.v10i1.454

Crusoe, J., & Melin, U. (2018). Investigating open government data barriers. In P. Parycek, O. Glassey, M. Janssen, H. J. Scholl, E. Tambouris, E. Kalampokis, & S. Virkar (Eds.), Lecture notes in computer science. Electronic government (pp. 169–183). Springer International Publishing.

Google Scholar  

Dawes, S. S., Vidiasova, L., & Parkhimovich, O. (2016). Planning and designing open government data programs: An ecosystem approach. Government Information Quarterly, 33 (1), 15–27. https://doi.org/10.1016/j.giq.2016.01.003

De Kool, D. & Bekkers, V. J. J. M. (2014). The perceived impact of open inspection data on the quality of education in Dutch primary schools: A parent perspective. Social Science Computer Review . https://doi.org/10.1177/0894439314560853

Fan, B., & Zhao, Y. (2017). The moderating effect of external pressure on the relationship between internal organizational factors and the quality of open government data. Government Information Quarterly, 34 (3), 396–405. https://doi.org/10.1016/j.giq.2017.08.006

Francey, A., & Mettler, T. (2021). The effects of open government data: Some stylised facts. Information Polity: The International Journal of Government & Democracy in the Information Age, 1–16. https://doi.org/10.3233/ip-200281 .

Gascó-Hernández, M., Martin, E. G., Reggi, L., Pyo, S., & Luna-Reyes, L. F. (2018). Promoting the use of open government data: Cases of training and engagement. Government Information Quarterly, 35 (2), 233–242. https://doi.org/10.1016/j.giq.2018.01.003

González-Zapata, F. & Heeks, R. (2015). The multiple meanings of open government data: Understanding different stakeholders and their perspectives. Government Information Quarterly, 32 (4), 441–452.  https://doi.org/10.1016/j.giq.2015.09.001

Gonzálvez-Gallego, N., Nieto-Torrejón, L., & Pérez-Cárceles, M. (2020). Is open data an enabler for trust? Exploring the link and the mediating role of citizen satisfaction. International Journal of Public Administration, 43 (14), 1218–1227. https://doi.org/10.1080/01900692.2019.1668412

Gusenbauer, M. (2019). Google scholar to overshadow them all? Comparing the sizes of 12 academic search engines and bibliographic databases. Scientometrics, 118 (1), 177–214. https://doi.org/10.1007/s11192-018-2958-5

Haini, S. I., AB, R., Mohd, N. Z., Zainuddin, N. M., & Ibrahim, R. (2020). Factors influencing the adoption of open government data in the public sector: A systematic literature review. International Journal on Advanced Science, Engineering and Information Technology, 10 (2), 611. https://doi.org/10.18517/ijaseit.10.2.9488

Hassan, M. I. A., & Twinomurinzi, H. (2018). A systematic literature review of open government data research: Challenges, opportunities and gaps. 2018 Open Innovations Conference (OI) (pp. 299–304).

Hermanto, A., Solimun, S., Rinaldo Fernandes, A. A., Wahyono, W., & Zulkarnain, Z. (2018). The importance of open government data for the private sector and ngos in indonesia. Digital Policy, Regulation and Governance, 20 (4), 293–309. https://search.ebscohost.com/login.aspx?direct=true&db=eoh&AN=1724283&site=ehost-live

Hossain, M. A., Dwivedi, Y. K., & Rana, N. P. (2016). State-of-the-art in open data research: Insights from existing literature and a research agenda. Journal of Organizational Computing & Electronic Commerce, 26 (1/2), 14–40. https://doi.org/10.1080/10919392.2015.1124007

Janssen, M., Charalabidis, Y., & Zuiderwijk, A. (2012). Benefits, adoption barriers and myths of open data and open government. Information Systems Management, 29 (4), 258–268. https://doi.org/10.1080/10580530.2012.716740

Jetzek, T., Avital, M., & Bjørn-Andersen, N. (2013). Generating value from open government data. In R. Baskerville, & M. Chau (Eds.), Proceedings of the 34th International Conference on Information Systems (ICIS 2013) . http://aisel.aisnet.org/cgi/viewcontent.cgi?article=1181&context=icis2013

Jetzek, T., Avital, M., & Bjørn-Andersen, N. (2014). Data-driven innovation through open government data. Journal of Theoretical and Applied Electronic Commerce Research, 9 (2), 100–120. https://doi.org/10.4067/S0718-18762014000200008

Jetzek, T., Avital, M., & Bjørn-Andersen, N. (2019). The sustainable value of open government data. Journal of the Association for Information Systems, 20 (6), 702–734. https://doi.org/10.17705/1jais.00549

Karkin, N., & Yavuz, N. (2017). An inquiry for local open government data policy: Is a proactive model of open government data portals possible in Turkey? Current Politics and Economics of the Middle East, 8 (2), 129–150 https://www.proquest.com/scholarly-journals/inquiry-local-open-government-data-policy-is/docview/2213786284/se-2?accountid=10218

Kassen, M. (2013). A promising phenomenon of open data: A case study of the Chicago open data project. Government Information Quarterly, 30 (4), 508–513. https://doi.org/10.1016/j.giq.2013.05.012

Kessler, E. H., & Chakrabarti, A. K. (1996). Innovation speed: A conceptual model of context, antecedents, and outcomes. Academy of Management Review, 21 (4), 1143–1191. https://doi.org/10.5465/amr.1996.9704071866

Khurshid, M. M., Zakaria, N. H., Rashid, A., Kazmi, R., Shafique, M. N., & Nazir Ahmad, M. (2019). Analyzing diffusion patterns of big open data as policy innovation in public sector. Computers & Electrical Engineering, 78 , 148–161. https://doi.org/10.1016/j.compeleceng.2019.07.010

Kim, H. (2018). Interlinking open government data in Korea using Administrative District knowledge graph. Journal of Information Science Theory & Practice (JIStaP), 6 (1), 18–30. https://doi.org/10.1633/JISTaP.2018.6.1.2

Kleiman, F., Meijer, S., & Janssen, M. (2020). Behavioral factors influencing the opening of government data by civil servants.  Proceedings of the 13th International Conference on Theory and Practice of Electronic Governance (ICEGOV), (pp. 529–534) . Association for Computing Machinery.

Book   Google Scholar  

Kurtz, L. P., Santos, P. M., Rover, A. J., Zuiderwijk, A., & Hinnant, C. C. (2019). Open data via websites of Brazilian superior courts of justice: Changes between 2013 and 2017. Information Polity: The International Journal of Government & Democracy in the Information Age, 24 (2), 147–162. https://doi.org/10.3233/IP-180104

Lane, P. J., Koka, B. R., & Pathak, S. (2006). The reification of absorptive capacity: A critical review and rejuvenation of the construct. Academy of Management Review, 31 (4), 833–863. https://doi.org/10.5465/amr.2006.22527456

Lee, S. Y., Díaz-Puente, J. M., & Martin, S. (2019). The contribution of open government to prosperity of society. International Journal of Public Administration, 42 (2), 144–157. https://doi.org/10.1080/01900692.2017.1405446

Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke,M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. PLoS Medicine, 6 (7). https://doi.org/10.1371/journal.pmed.1000100

Lim, T. C. (2021). Patterns in environmental priorities revealed through government open data portals. Telematics and Informatics, 101678 . https://doi.org/10.1016/j.tele.2021.101678

Lim, W. M., Yap, S.-F., & Makkar, M. (2021). Home sharing in marketing and tourism at a tipping point: What do we know, how do we know, and where should we be heading? Journal of Business Research, 122 , 534–566. https://doi.org/10.1016/j.jbusres.2020.08.051

Luna-Reyes, L. F., Najafabadi, M. M., Zuiderwijk, A., & Hinnant, C. C. (2019). The US open data initiative: The road ahead. Information Polity: The International Journal of Government & Democracy in the Information Age, 24 (2), 163–182. https://doi.org/10.3233/IP-180106

Máchová, R., & Lněnička, M. (2017). Evaluating the quality of open data portals on the National Level. Journal of Theoretical & Applied Electronic Commerce Research, 12 (1), 21–41. https://doi.org/10.4067/S0718-18762017000100003

Magalhaes, G., & Roseira, C. (2020). Open government data and the private sector: An empirical view on business models and value creation. Government Information Quarterly, 37 (3). https://doi.org/10.1016/j.giq.2017.08.004

Marjanovic, O. & Cecez-Kecmanovic, D. (2017). Exploring the tension between transparency and datification effects of open government IS through the lens of Complex Adaptive Systems. The Journal of Strategic Information Systems, 26 (3), 210–232.  https://doi.org/10.1016/j.jsis.2017.07.001

Marmier, A., & Mettler, T. (2020). Developing an index for measuring OGD publisher compliance to good practice standards: Insights from opendata.swiss. Information Polity: The International Journal of Government & Democracy in the Information Age, 25 (1), 91–110. https://doi.org/10.3233/IP-180120

Martín-Martín, A., Orduna-Malea, E., Harzing, A.-W., & Delgado López-Cózar, E. (2017). Can we use Google scholar to identify highly-cited documents? Journal of Informetrics, 11 (1), 152–163. https://doi.org/10.1016/j.joi.2016.11.008

Martín-Martín, A., Thelwall, M., Orduna-Malea, E., & Delgado López-Cózar, E. (2020). Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations' COCI: A multidisciplinary comparison of coverage via citations. Scientometrics, 1–36. https://doi.org/10.1007/s11192-020-03690-4 .

McBride, K., Aavik, G., Toots, M., Kalvet, T., & Krimmer, R. (2019). How does open government data driven co-creation occur? Six factors and a 'perfect storm'; insights from Chicago's food inspection forecasting model. Government Information Quarterly, 36 (1), 88–97. https://doi.org/10.1016/j.giq.2018.11.006

McNutt, J., Justice, J., Melitski, J., Ahn, M., Siddiqui, S., Carter, D., & Kline, A. (2016). The diffusion of civic technology and open government in the United States. Information Polity, 21 (2), 153–170. https://doi.org/10.3233/IP-160385

Meijer, A. (2015). Government transparency in historical perspective: From the ancient regime to open data in The Netherlands. International Journal of Public Administration, 38 (3), 189–199. https://doi.org/10.1080/01900692.2014.934837

Meijer, A., & Potjer, S. (2018). Citizen-generated open data: An explorative analysis of 25 cases. Government Information Quarterly, 35 (4), 613–621. https://doi.org/10.1016/j.giq.2018.10.004

Mergel, I., Kleibrink, A., & Sörvik, J. (2018). Open data outcomes: U.S. cities between product and process innovation. Government Information Quarterly, 35 (4), 622–632. https://doi.org/10.1016/j.giq.2018.09.004

Neto, A. J. A., Neves, D. F., Santos, L. C., Junior, M. C. R., & do Nascimento, Rogério P. C. (2018). Open government data usage overview: A systematic literature mapping.  Proceedings of the Euro American Conference on Telematics and Information Systems (EATIS) . Association for Computing Machinery.

Novais, T., Albuquerque, J. P. de, & Craveiro, G. S. (2013). An account of research on open government data (2007–2012): A systematic literature review. In Scholl H., Wimmer M.A., Tambouris E., Janssen M., & Macintosh A. (Eds.) (pp. 76–83). Gesellschaft fur Informatik (GI) . Retrieved from https://www.scopus.com/inward/record.uri?eid=2-s2.0-84918551984&partnerID=40&md5=8b722dd21af1f1c3743202eaeaae389d

Open Knowledge Foundation. (2021). Open definition: Defining open in open data, open content and open knowledge . Retrieved from http://opendefinition.org/

Parycek, P., Höchtl, J., & Ginner, M. (2014). Open Government Data Implementation Evaluation. Journal of theoretical and applied electronic commerce research, 9 (2), 80–99. https://doi.org/10.4067/S0718-18762014000200007

Paul, J., & Benito, G. R. G. (2018). A review of research on outward foreign direct investment from emerging countries, including China: What do we know, how do we know and where should we be heading? Asia Pacific Business Review, 24 (1), 90–115. https://doi.org/10.1080/13602381.2017.1357316

Paul, J., & Criado, A. R. (2020). The art of writing literature review: What do we know and what do we need to know? International Business Review, 29 (4), 101717. https://doi.org/10.1016/j.ibusrev.2020.101717

Piotrowski, S. J. (2017). The “open government reform” movement. American Review of Public Administration, 47 (2), 155–171. https://doi.org/10.1177/0275074016676575

Purwanto, A., Zuiderwijk, A., & Janssen, M. (2020). Citizen engagement with open government data. International Journal of Electronic Government Research, 16 (3), 1–25. https://doi.org/10.4018/IJEGR.2020070101

Raisch, S., & Birkinshaw, J. (2008). Organizational ambidexterity: Antecedents, outcomes, and moderators. Journal of Management, 34 (3), 375–409. https://doi.org/10.1177/0149206308316058

Roa, H. N., Loza-Aguirre, E., & Flores, P. (2019). A survey on the problems affecting the development of open government data initiatives. In Teran L., Meier A., & Pincay J. (Eds.) (pp. 157–163). Institute of Electrical and Electronics Engineers Inc. Retrieved from https://www.scopus.com/inward/record.uri?eid=2-s2.0-85068367066&doi=10.1109%2fICEDEG.2019.8734452&partnerID=40&md5=edfd3858943a1342f976f8bc20cf71e9

Rowe, F. (2014). What literature review is not: Diversity, boundaries and recommendations. European Journal of Information Systems, 23 (3), 241–255. https://doi.org/10.1057/ejis.2014.7

Ruijer, E., Grimmelikhuijsen, S., & Meijer, A. (2017). Open data for democracy: Developing a theoretical framework for open data use. Government Information Quarterly, 34 (1), 45–52. https://doi.org/10.1016/j.giq.2017.01.001

Ruijer, E., & Martinius, E. (2017). Researching the democratic impact of open government data: A systematic literature review. Information Polity: The International Journal of Government & Democracy in the Information Age, 22 (4), 233–250. https://doi.org/10.3233/IP-170413

Ruijer, E., & Meijer, A. (2020). Open government data as an innovation process: Lessons from a living lab experiment. Public Performance & Management Review, 43 (3), 613–635. https://doi.org/10.1080/15309576.2019.1568884

Safarov, I. (2019). Institutional dimensions of open government data implementation: Evidence from the Netherlands, Sweden, and the UK. Public Performance & Management Review, 42 (2), 305–328. https://doi.org/10.1080/15309576.2018.1438296

Safarov, I., Meijer, A., & Grimmelikhuijsen, S. (2017). Utilization of open government data: A systematic literature review of types, conditions, effects and users. Information Polity: The International Journal of Government & Democracy in the Information Age, 22 (1), 1–24. https://doi.org/10.3233/IP-160012

Saldaña, J. (2013). The coding manual for qualitative researchers (2. ed.) . SAGE Publ.

Saxena, S. (2018). Summarizing the decadal literature in open government data (OGD) research: A systematic review. Foresight, 20 (6), 648–664 https://search.ebscohost.com/login.aspx?direct=true&db=eoh&AN=1754758&site=ehost-live

Saxena, S., & Janssen, M. (2017). Examining open government data (OGD) usage in India through UTAUT framework. Foresight, 19 (4), 421–436 https://search.ebscohost.com/login.aspx?direct=true&db=eoh&AN=1674894&site=ehost-live

Sayogo, D. S., Pardo, T. A., & Cook, M. (2014). A framework for benchmarking open government data efforts. 47th Hawaii International Conference on System Sciences (pp. 1896–1905).  https://doi.org/10.1109/HICSS.2014.240

Shao, D. D., & Saxena, S. (2019). Barriers to open government data (OGD) initiative in Tanzania: Stakeholders' perspectives. Growth and Change, 50 (1), 470–485. https://doi.org/10.1111/grow.12282

Shepherd, E., Bunn, J., Flinn, A., Lomas, E., Sexton, A., Brimble, S., Chorley, K., Harrison, E., Lowry, J. & Page, J. (2019). Open government data: Critical information management perspectives. Records Management Journal, 29 (1), 152–167. https://doi.org/10.1108/RMJ-08-2018-0023

Smith, G., & Sandberg, J. (2018). Barriers to innovating with open government data: Exploring experiences across service phases and user types. Information Polity: The International Journal of Government & Democracy in the Information Age, 23 (3), 249–265. https://doi.org/10.3233/IP-170045

Susha, I., Grönlund, Å., & Janssen, M. (2015). Driving factors of service innovation using open government data: An exploratory study of entrepreneurs in two countries. Information Polity: The International Journal of Government & Democracy in the Information Age, 20 (1), 19–34. https://doi.org/10.3233/IP-150353

Tai, K.-T. (2021). Open government research over a decade: A systematic review. Government Information Quarterly, 38 (2), 101566. https://doi.org/10.1016/j.giq.2021.101566

Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management, 14 (3), 207–222. https://doi.org/10.1111/1467-8551.00375

Ubaldi, B. (2013). Open government data: towards empirical analysis of open government data initiatives. OECD working papers on public governance (No. 22) . Organisation for Economic Cooperation and Development (OECD) https://www.proquest.com/working-papers/open-government-data-towards-empirical-analysis/docview/1394263730/se-2?accountid=10218

Urquhart, C. (2013). Grounded theory for qualitative research: A practical guide . SAGE http://www.sagepub.com/upm-data/51468_Urquhart_Chapter_1.pdf

Veljković, N., Bogdanović-Dinić, S., & Stoimenov, L. (2014). Benchmarking open government: An open data perspective. Government Information Quarterly, 31 (2), 278–290. https://doi.org/10.1016/j.giq.2013.10.011

Wang, H.‑J., & Lo, J. (2016). Adoption of open government data among government agencies. Government Information Quarterly, 33 (1), 80–88. https://doi.org/10.1016/j.giq.2015.11.004

Wang, D., Chen, C., & Richards, D. (2018). A prioritization-based analysis of local open government data portals: A case study of Chinese province-level governments. Government Information Quarterly, 35 (4), 644–656. https://doi.org/10.1016/j.giq.2018.10.006

Wang, Y., Han, J. H., & Beynon-Davies, P. (2019). Understanding blockchain technology for future supply chains: A systematic literature review and research agenda. Supply Chain Management: An International Journal, 24 (1), 62–84. https://doi.org/10.1108/SCM-03-2018-0148

Wang, V., & Shepherd, D. (2020). Exploring the extent of openness of open government data – a critique of open government datasets in the UK. Government Information Quarterly, 37 (1). https://doi.org/10.1016/j.giq.2019.101405

Watson, R. T., & Webster, J. (2020). Analysing the past to prepare for the future: Writing a literature review a roadmap for release 2.0. Journal of Decision Systems, 29 (3), 129–147. https://doi.org/10.1080/12460125.2020.1798591

Webster, J., & Watson, R. T. (2002). Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly, 26 (2), xiii–xxiii. https://doi.org/10.2307/4132319

Welch, E., Feeney, M., & Park, C. H. (2016). Determinants of data sharing in U.S. city governments. Government Information Quarterly, 33 (3), 393–403. https://doi.org/10.1016/j.giq.2016.07.002

Wirtz, B. W., & Birkmeyer, S. (2015). Open government: Origin, development, and conceptual perspectives. International Journal of Public Administration, 38 (5), 381–396. https://doi.org/10.1080/01900692.2014.942735

Wirtz, B. W., Weyerer, J. C., & Rösch, M. (2018). Citizen and open government: An empirical analysis of antecedents of open government data. International Journal of Public Administration, 41 (4), 308–320. https://doi.org/10.1080/01900692.2016.1263659

Wirtz, B. W., Weyerer, J. C., & Rösch, M. (2019). Open government and citizen participation: An empirical analysis of citizen expectancy towards open government data. International Review of Administrative Sciences, 85 (3), 566–586. https://doi.org/10.1177/0020852317719996

World Wide Web Foundation. (2017). Open data barometer: Global report . Retrieved from https://opendatabarometer.org/doc/4thEdition/ODB-4thEdition-GlobalReport.pdf

Worthy, B. (2015). The impact of open data in the UK: Complex, unpredictable, and political. Public Administration, 93 (3), 788–805. https://doi.org/10.1111/padm.12166

Yang, T.-M., Lo, J., & Shiang, J. (2015). To open or not to open? Determinants of open government data. Journal of Information Science, 41 (5), 596–612. https://doi.org/10.1177/0165551515586715

Yang, T. M., & Wu, Y. J. (2016). Examining the socio-technical determinants influencing government agencies’ open data publication: A study in Taiwan. Government Information Quarterly, 33 (3):378–392. https://doi.org/10.1016/j.giq.2016.05.003

Young, M. M. (2020). Implementation of digital-era governance: The case of open data in U.S Cities. Public Administration Review, 80 (2), 305–315 https://search.ebscohost.com/login.aspx?direct=true&db=eoh&AN=1833568&site=ehost-live

Zhao, Y., & Fan, B. (2018). Exploring open government data capacity of government agency: Based on the resource-based theory. Government Information Quarterly, 35 (1), 1–12. https://doi.org/10.1016/j.giq.2018.01.002

Zhenbin, Y., Kankanhalli, A., Ha, S., & Tayi, G. K. (2020). What drives public agencies to participate in open government data initiatives? An innovation resource perspective. Information & Management, 57 (3), N.PAG-N.PAG. https://doi.org/10.1016/j.im.2019.103179

Zientek, L. R., Werner, J. M., Campuzano, M. V., & Nimon, K. (2018). The use of Google scholar for research and research dissemination. New Horizons in Adult Education and Human Resource Development, 30 (1), 39–46. https://doi.org/10.1002/nha3.20209

Zuiderwijk, A., Helbig, N., Gil-Garcia, J., & Janssen, M. (2014). Innovation through open data: A review of the state-of-the-art and an emerging research agenda. Journal of Theoretical & Applied Electronic Commerce Research, 9 (2), I–XIII. https://doi.org/10.4067/S0718-18762014000200001

Zuiderwijk, A., Janssen, M., & Dwivedi, Y. K. (2015). Acceptance and use predictors of open data technologies: Drawing upon the unified theory of acceptance and use of technology. Government Information Quarterly, 32 (4), 429–440. https://doi.org/10.1016/j.giq.2015.09.005

Zuiderwijk, A., Janssen, M., van de Kaa, G., & Poulis, K. (2016). The wicked problem of commercial value creation in open data ecosystems: Policy guidelines for governments. Information Polity: The International Journal of Government & Democracy in the Information Age, 21 (3), 223–236. https://doi.org/10.3233/IP-160391

Download references

Open Access funding enabled and organized by Projekt DEAL.

Author information

Authors and affiliations.

German University of Administrative Science Speyer, Chair for Information & Communication Management, Postbox 1409, 67324, Speyer, Germany

Bernd W. Wirtz, Jan C. Weyerer, Marcel Becker & Wilhelm M. Müller

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Bernd W. Wirtz .

Additional information

Responsible Editor: Thomas Hess

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

(DOCX 39 kb)

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Wirtz, B.W., Weyerer, J.C., Becker, M. et al. Open government data: A systematic literature review of empirical research. Electron Markets 32 , 2381–2404 (2022). https://doi.org/10.1007/s12525-022-00582-8

Download citation

Received : 25 March 2021

Accepted : 27 July 2022

Published : 20 September 2022

Issue Date : December 2022

DOI : https://doi.org/10.1007/s12525-022-00582-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Open government data
• Digital economy
• Digital business
• Literature review

JEL Classification

• Find a journal
• Publish with us
• Track your research

KG-EmpiRE: A Community-Maintainable Knowledge Graph for a Sustainable Literature Review on the State and Evolution of Empirical Research in Requirements Engineering

In the last two decades, several researchers provided snapshots of the “current” state and evolution of empirical research in requirements engineering (RE) through literature reviews. However, these literature reviews were not sustainable, as none built on or updated previous works due to the unavailability of the extracted and analyzed data. KG-EmpiRE is a Knowledge Graph (KG) of empirical research in RE based on scientific data extracted from currently 680 papers published in the IEEE International Requirements Engineering Conference (1994-2022). KG-EmpiRE is maintained in the Open Research Knowledge Graph (ORKG), making all data openly and long-term available according to the FAIR data principles. Our long-term goal is to constantly maintain KG-EmpiRE with the research community to synthesize a comprehensive, up-to-date, and long-term available overview of the state and evolution of empirical research in RE. Besides KG-EmpiRE, we provide its analysis with all supplementary materials in a repository. This repository contains all files with instructions for replicating and (re-)using the analysis locally or via executable environments and for repeating the research approach. Since its first release based on 199 papers (2014-2022), KG-EmpiRE and its analysis have been updated twice, currently covering over 650 papers. KG-EmpiRE and its analysis demonstrate how innovative infrastructures, such as the ORKG, can be leveraged to make data from literature reviews FAIR, openly available, and maintainable for the research community in the long term. In this way, we can enable replicable, (re-)usable , and thus sustainable literature reviews to ensure the quality, reliability, and timeliness of their research results.

Index Terms:

I introduction.

For 20 years, various researchers conducted literature reviews to examine the state and evolution of empirical research in requirements engineering (RE) with the shared goal of providing a comprehensive, up-to-date, and long-term available overview  [ 1 , 2 ] . However, these literature reviews were not sustainable, as none built on or updated previous ones, which are known challenges of literature reviews  [ 3 ] . While recent research addresses these challenges by providing social and economic decision support and guidance  [ 3 ] , the underlying problem is the unavailability of the extracted and analyzed data. Researchers need technical support, i.e., infrastructures, to conduct sustainable literature reviews so that all data is openly and long-term available according to the FAIR data principles  [ 3 ] and corresponding to open science in SE  [ 4 ] .

In their joint work, Wernlein  [ 5 ] and Karras et al.  [ 1 , 6 ] examined the use of the Open Research Knowledge Graph (ORKG)  [ 7 ] , as such technical support by building, publishing, and analyzing a Knowledge Graph (KG) of empirical research in RE (KG-EmpiRE) based on currently 680 research track papers of the IEEE International Requirements Engineering Conference (1994-2022).

In this paper, we present the KG-EmpiRE, available in the ORKG 1 1 1 https://orkg.org/observatory/Empirical_Software_Engineering , and its analysis, available on GitHub  [ 8 ] , Zenodo  [ 9 ] , and on Binder 2 2 2 https://tinyurl.com/empire-analysis for interactive replication and (re-)use.

KG-EmpiRE contains scientific data on the six themes  research paradigm , research design , research method , data collection , data analysis , and bibliographic metadata . We plan to expand these themes in the long term. For more details on the themes, refer to the supplementary materials  [ 9 , 8 ] . Since its first release based on 199 papers (2014-2022)  [ 5 ] , KG-EmpiRE and its analysis have been updated twice. Karras et al.   [ 1 ] published the first update with 570 papers (2000-2022) at the 17th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement 2023, where they received the best paper award. The second update is ongoing and covers 680 papers (1994-2022) so far. The goal for the second update is to cover all 748 research track papers from the IEEE International Requirements Engineering Conference  (1993-2023) .

The analysis provides answers to 16 out of 77 competency questions (cf. supplementary materials  [ 9 , 8 ] ) regarding empirical research in RE that we derived from the vision of Sjøberg et al.  [ 10 ] on the role of empirical methods in SE, including RE, for 2020-2025. While the number of competency questions answered reflects the coverage of the curated topic in KG-EmpiRE, the answers to competency questions provide insights into the state and evolution of empirical research in RE. For each competency question answered, we provide all details of the analysis with its data, visualizations, explanations, and answers in a repository  [ 9 , 8 ] that is also hosted on Binder for interactive replication and (re-)use.

Overall, this repository contains all files with detailed explanations and instructions for replication and (re-)use of KG-EmpiRE and its analysis locally or via executable environments (Binder and GitHub Codespaces), as well as for repeating the research approach for sustainable literature reviews with the ORKG. The repository also contains all generated visualizations with their data, exported as PNG and CSV files, as well as supplementary materials on the themes, their structuring in the ORKG, and all 77 competency questions.

II Structure of KG-EmpiRE and the Repository

Ii-a kg-empire.

We developed an ORKG template 3 3 3 https://orkg.org/template/R186491 to organize the scientific data extracted from the papers in the ORKG. ORKG templates implement a subset of the Shapes Constraint Language (SHACL) and allow specifying the underlying (graph) structure to organize the data in a structured manner  [ 11 ] . In this way, we determined which data to extract and standardized their description to ensure they are FAIR, consistent, and comparable across all papers. The developed ORKG template covers the six themes investigated. For more details on the ORKG template, refer to the supplementary materials  [ 9 , 8 ] .

By applying the ORKG template to the papers, KG-EmpiRE currently consists of almost 35,000 triples, which are made up of over 51,000 resources and almost 19,000 literals (see Table I ). While these statistics reflect the efforts to provide a solid structured description of the extracted data, they also show that KG-EmpiRE is relatively small compared to the entire ORKG and other well-known knowledge graphs, e.g., Wikidata or DBpedia, which include millions of entities.

II-B Repository

In the repository, there are three folders and six files, with the Jupyter Notebook empire-analysis.ipynb as the main file. The Jupyter Notebook encapsulates the entire analysis of KG-EmpiRE and provides visualizations, explanations, and answers for each of the 16 competency questions. The visualizations are exported as PNG files per competency question to the Figures folder. The data retrieved by KG-EmpiRE for analysis is stored as CSV files for each competency question in the SPARQL-Data folder by date. In this folder, we also provide CSV files of the latest release to replicate the results of the related publication  [ 1 ] . The last folder Supplementary materials provides additional materials for detailed overviews of the content for data extraction regarding the themes, the developed ORKG template, all 77 competency questions derived, and the research approach. The second most important file is README.md , which contains detailed explanations and instructions about the project, the repository, its installation (locally and via executable environments), the replication of the analysis, and the (re)use of KG-EmpiRE with its most recent data. The remaining four files support the installation ( requirements.txt , runtime.txt ), clarify the copyright ( LICENSE ), and ensure the citability of the repository ( CITATION.cff ) 4 4 4 https://citation-file-format.github.io/ .

III Conclusion

Overall, KG-EmpiRE and its analysis lay the foundation for a sustainable literature review on the state and evolution of empirical research in requirements engineering. They can be used to replicate the results from the related publication  [ 1 ] , (re-)use the data for further studies, and repeat the research approach for sustainable literature reviews on other topics. KG-EmpiRE and its analysis demonstrate how innovative infrastructures, such as the ORKG, can be leveraged to make data from literature reviews FAIR and openly available in the long term. In this way, researchers can build on and update the data ideally collaboratively, enabling sustainable literature reviews for comprehensive, up-to-date, and long-term available overviews, true to the principle: Divide et Impera .

In summary, the special feature of KG-EmpiRE lies in the proof that data from literature reviews can already be prepared during data extraction in such a way that they are understandable and processable by humans and machines to update, replicate, and (re-)use them sustainably. KG-EmpiRE and the underlying research approach using technical infrastructures, such as the ORKG, have the potential to be used on a large scale to establish sustainable literature reviews and thus ensure the quality, reliability, and timeliness of their research results.

Acknowledgment

The authors thank the Federal Government, the Heads of Government of the Länder, as well as the Joint Science Conference (GWK), for their funding and support within the NFDI4Ing and NFDI4DataScience consortia. This work was funded by the German Research Foundation (DFG) project numbers 44214671 and 460234259 and by the European Research Council for the project ScienceGRAPH (Grant agreement ID: 819536).

• [1] O. Karras, F. Wernlein, J. Klünder, and S. Auer, “Divide and Conquer the EmpiRE: A Community-Maintainable Knowledge Graph of Empirical Research in Requirements Engineering,” in International Symposium on Empirical Software Engineering and Measurement , 2023.
• [2] O. Karras, F. Wernlein, J. Klünder, and S. Auer, “A Comparison of Scientific Publications on the State of Empirical Research in Requirements Engineering and Software Engineering,” 2023. [Online]. Available: https://orkg.org/comparison/R650023/
• [3] V. dos Santos et al. , “Towards Sustainability of Systematic Literature Reviews,” in 15th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement , 2021.
• [4] D. Mendez et al. , “Open Science in Software Engineering,” in Contemporary Empirical Methods in Software Engineering .   Springer, 2020.
• [5] F. Wernlein, “Acquisition and Analysis of Research Practices Using Semantic Structures,” Bachelor Thesis, Gottfried Wilhelm Leibniz Universität, 2022. [Online]. Available: https://doi.org/10.15488/12796
• [6] O. Karras, F. Wernlein, J. Klünder, and S. Auer, “KG-EmpiRE: A Community-Maintainable Knowledge Graph of Empirical Research in Requirements Engineering,” in Software Engineering 2024 , 2024.
• [7] M. Stocker et al. , “FAIR Scientific Information with the Open Research Knowledge Graph,” FAIR Connect , vol. 1, no. 1, 2023.
• [8] O. Karras, “Divide and Conquer the EmpiRE: A Community-Maintainable Knowledge Graph of Empirical Research in Requirements Engineering - A Sustainable Literature Review for Analyzing the State and Evolution of Empirical Research in Requirements Engineering,” 2023. [Online]. Available: https://github.com/okarras/EmpiRE-Analysis
• [9] ——, “Divide and Conquer the EmpiRE: A Community-Maintainable Knowledge Graph of Empirical Research in Requirements Engineering - A Sustainable Literature Review for Analyzing the State and Evolution of Empirical Research in Requirements Engineering,” 2024. [Online]. Available: https://doi.org/10.5281/zenodo.11092471
• [10] D. I. Sjoberg et al. , “The Future of Empirical Methods in Software Engineering Research,” in Future of Software Engineering , 2007.
• [11] H. Hussein et al. , “Increasing Reproducibility in Science by Interlinking Semantic Artifact Descriptions in a Knowledge Graph,” in Leveraging Generative Intelligence in Digital Libraries: Towards Human-Machine Collaboration .   Springer, 2023.

SYSTEMATIC REVIEW article

The role of counseling for non-traditional students in formal higher education: a scoping review provisionally accepted.

• 1 Department of Pedagogy and Primary Education, National and Kapodistrian University of Athens, Greece

The final, formatted version of the article will be published soon.

Since the mid-20th century, the number of adult students enrolled in formal higher education (HE) programs has significantly increased. The profile of non-traditional students differs significantly from that of traditional students in terms of their characteristics, learning methods, obstacles and challenges, motivations for learning, and conditions for effective learning. Unlike traditional students, adult students often balance family, work, and educational responsibilities, necessitating a more nuanced approach to support and guidance. However, most HE institutions primarily serve the needs of traditional student populations, which results in limited support available to adult students. This scoping review aimed to explore and map the existing literature on the role of adult (or non-traditional) students counseling in the context of formal HE. We focused on literature related to academic advising for non-traditional students in formal HE, restricting our search to both empirical and non-empirical articles published in peer-reviewed journals between 2010 and 2022. Employing Arksey and O'Malley's scoping review method and the PRISMA-ScR Checklist, we searched four databases (EBSCOhost, Crossref, Semantic Scholar, and ERIC), supplemented by a manual search. Of the 1,330 articles identified and screened, 25 studies met the eligibility criteria. Our review included 17 empirical and eight non-empirical studies, with the majority conducted in the USA (21 of 25). Thematic analysis revealed five key research areas (or themes): academic advising practices, perceptions of advising, technology, and advising, advising models, and academic success. The most common research theme, advising practices for adult (undergraduate and doctoral) students, constituted 52% of the studies (n=13). Drawing from our analysis, we discuss current trends and future development in advising non-traditional students within formal HE settings. The added value of academic advising for adult students is explored, and any potential gaps in research literature knowledge are identified.

Keywords: Academic advising, Counseling, non-traditional students, adult students, higher education

Received: 25 Dec 2023; Accepted: 16 May 2024.

Copyright: © 2024 Stamou, Tsoli and Babalis. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

* Correspondence: Ms. Panagiota Stamou, National and Kapodistrian University of Athens, Department of Pedagogy and Primary Education, Athens, Greece

People also looked at