what is the meaning of critical in research

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base
• Working with sources
• What Is Critical Thinking? | Definition & Examples

What Is Critical Thinking? | Definition & Examples

Published on May 30, 2022 by Eoghan Ryan . Revised on May 31, 2023.

Critical thinking is the ability to effectively analyze information and form a judgment .

To think critically, you must be aware of your own biases and assumptions when encountering information, and apply consistent standards when evaluating sources .

Critical thinking skills help you to:

• Identify credible sources
• Evaluate and respond to arguments
• Assess alternative viewpoints
• Test hypotheses against relevant criteria

Table of contents

Why is critical thinking important, critical thinking examples, how to think critically, other interesting articles, frequently asked questions about critical thinking.

Critical thinking is important for making judgments about sources of information and forming your own arguments. It emphasizes a rational, objective, and self-aware approach that can help you to identify credible sources and strengthen your conclusions.

Critical thinking is important in all disciplines and throughout all stages of the research process . The types of evidence used in the sciences and in the humanities may differ, but critical thinking skills are relevant to both.

In academic writing , critical thinking can help you to determine whether a source:

• Is free from research bias
• Provides evidence to support its research findings
• Considers alternative viewpoints

Outside of academia, critical thinking goes hand in hand with information literacy to help you form opinions rationally and engage independently and critically with popular media.

Prevent plagiarism. Run a free check.

Critical thinking can help you to identify reliable sources of information that you can cite in your research paper . It can also guide your own research methods and inform your own arguments.

Outside of academia, critical thinking can help you to be aware of both your own and others’ biases and assumptions.

Academic examples

However, when you compare the findings of the study with other current research, you determine that the results seem improbable. You analyze the paper again, consulting the sources it cites.

You notice that the research was funded by the pharmaceutical company that created the treatment. Because of this, you view its results skeptically and determine that more independent research is necessary to confirm or refute them. Example: Poor critical thinking in an academic context You’re researching a paper on the impact wireless technology has had on developing countries that previously did not have large-scale communications infrastructure. You read an article that seems to confirm your hypothesis: the impact is mainly positive. Rather than evaluating the research methodology, you accept the findings uncritically.

Nonacademic examples

However, you decide to compare this review article with consumer reviews on a different site. You find that these reviews are not as positive. Some customers have had problems installing the alarm, and some have noted that it activates for no apparent reason.

You revisit the original review article. You notice that the words “sponsored content” appear in small print under the article title. Based on this, you conclude that the review is advertising and is therefore not an unbiased source. Example: Poor critical thinking in a nonacademic context You support a candidate in an upcoming election. You visit an online news site affiliated with their political party and read an article that criticizes their opponent. The article claims that the opponent is inexperienced in politics. You accept this without evidence, because it fits your preconceptions about the opponent.

There is no single way to think critically. How you engage with information will depend on the type of source you’re using and the information you need.

However, you can engage with sources in a systematic and critical way by asking certain questions when you encounter information. Like the CRAAP test , these questions focus on the currency , relevance , authority , accuracy , and purpose of a source of information.

When encountering information, ask:

• Who is the author? Are they an expert in their field?
• What do they say? Is their argument clear? Can you summarize it?
• When did they say this? Is the source current?
• Where is the information published? Is it an academic article? Is it peer-reviewed ?
• Why did the author publish it? What is their motivation?
• How do they make their argument? Is it backed up by evidence? Does it rely on opinion, speculation, or appeals to emotion ? Do they address alternative arguments?

Critical thinking also involves being aware of your own biases, not only those of others. When you make an argument or draw your own conclusions, you can ask similar questions about your own writing:

• Am I only considering evidence that supports my preconceptions?
• Is my argument expressed clearly and backed up with credible sources?
• Would I be convinced by this argument coming from someone else?

If you want to know more about ChatGPT, AI tools , citation , and plagiarism , make sure to check out some of our other articles with explanations and examples.

• ChatGPT vs human editor
• ChatGPT citations
• Is ChatGPT trustworthy?
• Using ChatGPT for your studies
• What is ChatGPT?
• Chicago style
• Paraphrasing

 Plagiarism

• Types of plagiarism
• Self-plagiarism
• Avoiding plagiarism
• Academic integrity
• Consequences of plagiarism
• Common knowledge

The only proofreading tool specialized in correcting academic writing - try for free!

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Try for free

Critical thinking refers to the ability to evaluate information and to be aware of biases or assumptions, including your own.

Like information literacy , it involves evaluating arguments, identifying and solving problems in an objective and systematic way, and clearly communicating your ideas.

Critical thinking skills include the ability to:

You can assess information and arguments critically by asking certain questions about the source. You can use the CRAAP test , focusing on the currency , relevance , authority , accuracy , and purpose of a source of information.

Ask questions such as:

• Who is the author? Are they an expert?
• How do they make their argument? Is it backed up by evidence?

A credible source should pass the CRAAP test  and follow these guidelines:

• The information should be up to date and current.
• The author and publication should be a trusted authority on the subject you are researching.
• The sources the author cited should be easy to find, clear, and unbiased.
• For a web source, the URL and layout should signify that it is trustworthy.

Information literacy refers to a broad range of skills, including the ability to find, evaluate, and use sources of information effectively.

Being information literate means that you:

• Know how to find credible sources
• Use relevant sources to inform your research
• Understand what constitutes plagiarism
• Know how to cite your sources correctly

Confirmation bias is the tendency to search, interpret, and recall information in a way that aligns with our pre-existing values, opinions, or beliefs. It refers to the ability to recollect information best when it amplifies what we already believe. Relatedly, we tend to forget information that contradicts our opinions.

Although selective recall is a component of confirmation bias, it should not be confused with recall bias.

On the other hand, recall bias refers to the differences in the ability between study participants to recall past events when self-reporting is used. This difference in accuracy or completeness of recollection is not related to beliefs or opinions. Rather, recall bias relates to other factors, such as the length of the recall period, age, and the characteristics of the disease under investigation.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Ryan, E. (2023, May 31). What Is Critical Thinking? | Definition & Examples. Scribbr. Retrieved April 12, 2024, from https://www.scribbr.com/working-with-sources/critical-thinking/

Is this article helpful?

Eoghan Ryan

Other students also liked, student guide: information literacy | meaning & examples, what are credible sources & how to spot them | examples, applying the craap test & evaluating sources, unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

• Privacy Policy

Buy Me a Coffee

Home » Critical Analysis – Types, Examples and Writing Guide

Critical Analysis – Types, Examples and Writing Guide

Table of Contents

Critical Analysis

Definition:

Critical analysis is a process of examining a piece of work or an idea in a systematic, objective, and analytical way. It involves breaking down complex ideas, concepts, or arguments into smaller, more manageable parts to understand them better.

Types of Critical Analysis

Types of Critical Analysis are as follows:

Literary Analysis

This type of analysis focuses on analyzing and interpreting works of literature , such as novels, poetry, plays, etc. The analysis involves examining the literary devices used in the work, such as symbolism, imagery, and metaphor, and how they contribute to the overall meaning of the work.

Film Analysis

This type of analysis involves examining and interpreting films, including their themes, cinematography, editing, and sound. Film analysis can also include evaluating the director’s style and how it contributes to the overall message of the film.

Art Analysis

This type of analysis involves examining and interpreting works of art , such as paintings, sculptures, and installations. The analysis involves examining the elements of the artwork, such as color, composition, and technique, and how they contribute to the overall meaning of the work.

Cultural Analysis

This type of analysis involves examining and interpreting cultural artifacts , such as advertisements, popular music, and social media posts. The analysis involves examining the cultural context of the artifact and how it reflects and shapes cultural values, beliefs, and norms.

Historical Analysis

This type of analysis involves examining and interpreting historical documents , such as diaries, letters, and government records. The analysis involves examining the historical context of the document and how it reflects the social, political, and cultural attitudes of the time.

Philosophical Analysis

This type of analysis involves examining and interpreting philosophical texts and ideas, such as the works of philosophers and their arguments. The analysis involves evaluating the logical consistency of the arguments and assessing the validity and soundness of the conclusions.

Scientific Analysis

This type of analysis involves examining and interpreting scientific research studies and their findings. The analysis involves evaluating the methods used in the study, the data collected, and the conclusions drawn, and assessing their reliability and validity.

Critical Discourse Analysis

This type of analysis involves examining and interpreting language use in social and political contexts. The analysis involves evaluating the power dynamics and social relationships conveyed through language use and how they shape discourse and social reality.

Comparative Analysis

This type of analysis involves examining and interpreting multiple texts or works of art and comparing them to each other. The analysis involves evaluating the similarities and differences between the texts and how they contribute to understanding the themes and meanings conveyed.

Critical Analysis Format

Critical Analysis Format is as follows:

I. Introduction

• Provide a brief overview of the text, object, or event being analyzed
• Explain the purpose of the analysis and its significance
• Provide background information on the context and relevant historical or cultural factors

II. Description

• Provide a detailed description of the text, object, or event being analyzed
• Identify key themes, ideas, and arguments presented
• Describe the author or creator’s style, tone, and use of language or visual elements

III. Analysis

• Analyze the text, object, or event using critical thinking skills
• Identify the main strengths and weaknesses of the argument or presentation
• Evaluate the reliability and validity of the evidence presented
• Assess any assumptions or biases that may be present in the text, object, or event
• Consider the implications of the argument or presentation for different audiences and contexts

IV. Evaluation

• Provide an overall evaluation of the text, object, or event based on the analysis
• Assess the effectiveness of the argument or presentation in achieving its intended purpose
• Identify any limitations or gaps in the argument or presentation
• Consider any alternative viewpoints or interpretations that could be presented
• Summarize the main points of the analysis and evaluation
• Reiterate the significance of the text, object, or event and its relevance to broader issues or debates
• Provide any recommendations for further research or future developments in the field.

VI. Example

• Provide an example or two to support your analysis and evaluation
• Use quotes or specific details from the text, object, or event to support your claims
• Analyze the example(s) using critical thinking skills and explain how they relate to your overall argument

VII. Conclusion

• Reiterate your thesis statement and summarize your main points
• Provide a final evaluation of the text, object, or event based on your analysis
• Offer recommendations for future research or further developments in the field
• End with a thought-provoking statement or question that encourages the reader to think more deeply about the topic

How to Write Critical Analysis

Writing a critical analysis involves evaluating and interpreting a text, such as a book, article, or film, and expressing your opinion about its quality and significance. Here are some steps you can follow to write a critical analysis:

• Read and re-read the text: Before you begin writing, make sure you have a good understanding of the text. Read it several times and take notes on the key points, themes, and arguments.
• Identify the author’s purpose and audience: Consider why the author wrote the text and who the intended audience is. This can help you evaluate whether the author achieved their goals and whether the text is effective in reaching its audience.
• Analyze the structure and style: Look at the organization of the text and the author’s writing style. Consider how these elements contribute to the overall meaning of the text.
• Evaluate the content : Analyze the author’s arguments, evidence, and conclusions. Consider whether they are logical, convincing, and supported by the evidence presented in the text.
• Consider the context: Think about the historical, cultural, and social context in which the text was written. This can help you understand the author’s perspective and the significance of the text.
• Develop your thesis statement : Based on your analysis, develop a clear and concise thesis statement that summarizes your overall evaluation of the text.
• Support your thesis: Use evidence from the text to support your thesis statement. This can include direct quotes, paraphrases, and examples from the text.
• Write the introduction, body, and conclusion : Organize your analysis into an introduction that provides context and presents your thesis, a body that presents your evidence and analysis, and a conclusion that summarizes your main points and restates your thesis.
• Revise and edit: After you have written your analysis, revise and edit it to ensure that your writing is clear, concise, and well-organized. Check for spelling and grammar errors, and make sure that your analysis is logically sound and supported by evidence.

When to Write Critical Analysis

You may want to write a critical analysis in the following situations:

• Academic Assignments: If you are a student, you may be assigned to write a critical analysis as a part of your coursework. This could include analyzing a piece of literature, a historical event, or a scientific paper.
• Journalism and Media: As a journalist or media person, you may need to write a critical analysis of current events, political speeches, or media coverage.
• Personal Interest: If you are interested in a particular topic, you may want to write a critical analysis to gain a deeper understanding of it. For example, you may want to analyze the themes and motifs in a novel or film that you enjoyed.
• Professional Development : Professionals such as writers, scholars, and researchers often write critical analyses to gain insights into their field of study or work.

Critical Analysis Example

An Example of Critical Analysis Could be as follow:

Research Topic:

The Impact of Online Learning on Student Performance

Introduction:

The introduction of the research topic is clear and provides an overview of the issue. However, it could benefit from providing more background information on the prevalence of online learning and its potential impact on student performance.

Literature Review:

The literature review is comprehensive and well-structured. It covers a broad range of studies that have examined the relationship between online learning and student performance. However, it could benefit from including more recent studies and providing a more critical analysis of the existing literature.

Research Methods:

The research methods are clearly described and appropriate for the research question. The study uses a quasi-experimental design to compare the performance of students who took an online course with those who took the same course in a traditional classroom setting. However, the study may benefit from using a randomized controlled trial design to reduce potential confounding factors.

The results are presented in a clear and concise manner. The study finds that students who took the online course performed similarly to those who took the traditional course. However, the study only measures performance on one course and may not be generalizable to other courses or contexts.

Discussion :

The discussion section provides a thorough analysis of the study’s findings. The authors acknowledge the limitations of the study and provide suggestions for future research. However, they could benefit from discussing potential mechanisms underlying the relationship between online learning and student performance.

Conclusion :

The conclusion summarizes the main findings of the study and provides some implications for future research and practice. However, it could benefit from providing more specific recommendations for implementing online learning programs in educational settings.

Purpose of Critical Analysis

There are several purposes of critical analysis, including:

• To identify and evaluate arguments : Critical analysis helps to identify the main arguments in a piece of writing or speech and evaluate their strengths and weaknesses. This enables the reader to form their own opinion and make informed decisions.
• To assess evidence : Critical analysis involves examining the evidence presented in a text or speech and evaluating its quality and relevance to the argument. This helps to determine the credibility of the claims being made.
• To recognize biases and assumptions : Critical analysis helps to identify any biases or assumptions that may be present in the argument, and evaluate how these affect the credibility of the argument.
• To develop critical thinking skills: Critical analysis helps to develop the ability to think critically, evaluate information objectively, and make reasoned judgments based on evidence.
• To improve communication skills: Critical analysis involves carefully reading and listening to information, evaluating it, and expressing one’s own opinion in a clear and concise manner. This helps to improve communication skills and the ability to express ideas effectively.

Importance of Critical Analysis

Here are some specific reasons why critical analysis is important:

• Helps to identify biases: Critical analysis helps individuals to recognize their own biases and assumptions, as well as the biases of others. By being aware of biases, individuals can better evaluate the credibility and reliability of information.
• Enhances problem-solving skills : Critical analysis encourages individuals to question assumptions and consider multiple perspectives, which can lead to creative problem-solving and innovation.
• Promotes better decision-making: By carefully evaluating evidence and arguments, critical analysis can help individuals make more informed and effective decisions.
• Facilitates understanding: Critical analysis helps individuals to understand complex issues and ideas by breaking them down into smaller parts and evaluating them separately.
• Fosters intellectual growth : Engaging in critical analysis challenges individuals to think deeply and critically, which can lead to intellectual growth and development.

Advantages of Critical Analysis

Some advantages of critical analysis include:

• Improved decision-making: Critical analysis helps individuals make informed decisions by evaluating all available information and considering various perspectives.
• Enhanced problem-solving skills : Critical analysis requires individuals to identify and analyze the root cause of a problem, which can help develop effective solutions.
• Increased creativity : Critical analysis encourages individuals to think outside the box and consider alternative solutions to problems, which can lead to more creative and innovative ideas.
• Improved communication : Critical analysis helps individuals communicate their ideas and opinions more effectively by providing logical and coherent arguments.
• Reduced bias: Critical analysis requires individuals to evaluate information objectively, which can help reduce personal biases and subjective opinions.
• Better understanding of complex issues : Critical analysis helps individuals to understand complex issues by breaking them down into smaller parts, examining each part and understanding how they fit together.
• Greater self-awareness: Critical analysis helps individuals to recognize their own biases, assumptions, and limitations, which can lead to personal growth and development.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Cluster Analysis – Types, Methods and Examples

Data Collection – Methods Types and Examples

Delimitations in Research – Types, Examples and...

Discriminant Analysis – Methods, Types and...

Research Process – Steps, Examples and Tips

Research Design – Types, Methods and Examples

• Archives & Special Collections home
• Art Library home
• Ekstrom Library home
• Kornhauser Health Sciences Library home
• Law Library home
• Music Library home
• University of Louisville Hospital home
• Interlibrary Loan
• Off-Campus Login
• Renew Books
• Cardinal Card
• My Print Center
• Business Ops
• Cards Career Connection

Search Site

Search catalog, critical thinking and academic research: intro.

• Information
• Point of View
• Assumptions
• Implications

Critical Thinking and Academic Research

Academic research focuses on the creation of new ideas, perspectives, and arguments. The researcher seeks relevant information in articles, books, and other sources, then develops an informed point of view within this ongoing "conversation" among researchers.

The research process is not simply collecting data, evidence, or "facts," then piecing together this preexisting information into a paper. Instead, the research process is about inquiry—asking questions and developing answers through serious critical thinking and thoughtful reflection.

As a result, the research process is recursive, meaning that the researcher regularly revisits ideas, seeks new information when necessary, and reconsiders and refines the research question, topic, or approach. In other words, research almost always involves constant reflection and revision.

This guide is designed to help you think through various aspects of the research process. The steps are not sequential, nor are they prescriptive about what steps you should take at particular points in the research process. Instead, the guide should help you consider the larger, interrelated elements of thinking involved in research.

Research Anxiety?

Research is not often easy or straightforward, so it's completely normal to feel anxious, frustrated, or confused. In fact, if you feel anxious, it can be a good sign that you're engaging in the type of critical thinking necessary to research and write a high-quality paper.

Think of the research process not as one giant, impossibly complicated task, but as a series of smaller, interconnected steps. These steps can be messy, and there is not one correct sequence of steps that will work for every researcher. However, thinking about research in small steps can help you be more productive and alleviate anxiety.

Paul-Elder Framework

This guide is based on the "Elements of Reasoning" from the Paul-Elder framework for critical thinking. For more information about the Paul-Elder framework, click the link below.

Some of the content in this guide has been adapted from The Aspiring Thinker's Guide to Critical Thinking (2009) by Linda Elder and Richard Paul.

• Next: Purpose >>
• Last Updated: Jul 10, 2023 11:50 AM
• Librarian Login

Critical Analysis in Composition

Glossary of Grammatical and Rhetorical Terms

• An Introduction to Punctuation
• Ph.D., Rhetoric and English, University of Georgia
• M.A., Modern English and American Literature, University of Leicester
• B.A., English, State University of New York

In composition , critical analysis is a careful examination and evaluation of a text , image, or other work or performance.

Performing a critical analysis does not necessarily involve finding fault with a work. On the contrary, a thoughtful critical analysis may help us understand the interaction of the particular elements that contribute to a work's power and effectiveness. For this reason, critical analysis is a central component of academic training; the skill of critical analysis is most often thought of in the context of analyzing a work of art or literature, but the same techniques are useful to build an understanding of texts and resources in any discipline.

In this context, the word "critical" carries a different connotation than in vernacular, everyday speech. "Critical" here does not simply mean pointing out a work's flaws or arguing why it is objectionable by some standard. Instead, it points towards a close reading of that work to gather meaning, as well as to evaluate its merits. The evaluation is not the sole point of critical analysis, which is where it differs from the colloquial meaning of "criticize."

Examples of Critical Essays

• "Jack and Gill: A Mock Criticism" by Joseph Dennie
• "Miss Brill's Fragile Fantasy": A Critical Essay About Katherine Mansfield's Short Story "Miss Brill" and "Poor, Pitiful Miss Brill"
• "On the Knocking at the Gate in Macbeth " by Thomas De Quincey
• A Rhetorical Analysis of Claude McKay's "Africa"
• A Rhetorical Analysis of E B. White's Essay "The Ring of Time"
• A Rhetorical Analysis of U2's "Sunday Bloody Sunday"
• "Saloonio: A Study in Shakespearean Criticism" by Stephen Leacock
• Writing About Fiction: A Critical Essay on Hemingway's Novel The Sun Also Rises

Quotes About Critical Analysis

• " [C]ritical analysis involves breaking down an idea or a statement, such as a claim , and subjecting it to critical thinking in order to test its validity." (Eric Henderson, The Active Reader: Strategies for Academic Reading and Writing . Oxford University Press, 2007)​
• "To write an effective critical analysis, you need to understand the difference between analysis and summary . . . . [A] critical analysis looks beyond the surface of a text—it does far more than summarize a work. A critical analysis isn't simply dashing off a few words about the work in general." ( Why Write?: A Guide to BYU Honors Intensive Writing . Brigham Young University, 2006)
• "Although the main purpose of a critical analysis is not to persuade , you do have the responsibility of organizing a discussion that convinces readers that your analysis is astute." (Robert Frew et al., Survival: A Sequential Program for College Writing . Peek, 1985)

Critical Thinking and Research

"[I]n response to the challenge that a lack of time precludes good, critical analysis , we say that good, critical analysis saves time. How? By helping you be more efficient in terms of the information you gather. Starting from the premise that no practitioner can claim to collect all the available information, there must always be a degree of selection that takes place. By thinking analytically from the outset, you will be in a better position to 'know' which information to collect, which information is likely to be more or less significant and to be clearer about what questions you are seeking to answer." (David Wilkins and Godfred Boahen, Critical Analysis Skills For Social Workers . McGraw-Hill, 2013)

How to Read Text Critically

 "Being critical in academic enquiry means: - adopting an attitude of skepticism or reasoned doubt towards your own and others' knowledge in the field of enquiry . . . - habitually questioning the quality of your own and others' specific claims to knowledge about the field and the means by which these claims were generated; - scrutinizing claims to see how far they are convincing . . .; - respecting others as people at all times. Challenging others' work is acceptable, but challenging their worth as people is not; - being open-minded , willing to be convinced if scrutiny removes your doubts, or to remain unconvinced if it does not; - being constructive by putting your attitude of skepticism and your open-mindedness to work in attempting to achieve a worthwhile goal." (Mike Wallace and Louise Poulson, "Becoming a Critical Consumer of the Literature." Learning to Read Critically in Teaching and Learning , ed. by Louise Poulson and Mike Wallace. SAGE, 2004)

Critically Analyzing Persuasive Ads

"[I]n my first-year composition class, I teach a four-week advertisement analysis project as a way to not only heighten students' awareness of the advertisements they encounter and create on a daily basis but also to encourage students to actively engage in a discussion about critical analysis by examining rhetorical appeals in persuasive contexts. In other words, I ask students to pay closer attention to a part of the pop culture in which they live. " . . . Taken as a whole, my ad analysis project calls for several writing opportunities in which students write essays , responses, reflections, and peer assessments . In the four weeks, we spend a great deal of time discussing the images and texts that make up advertisements, and through writing about them, students are able to heighten their awareness of the cultural 'norms' and stereotypes which are represented and reproduced in this type of communication ." (Allison Smith, Trixie Smith, and Rebecca Bobbitt, Teaching in the Pop Culture Zone: Using Popular Culture in the Composition Classroom . Wadsworth Cengage, 2009)

Critically Analyzing Video Games

 "When dealing with a game's significance, one could analyze the themes of the game be they social, cultural, or even political messages. Most current reviews seem to focus on a game's success: why it is successful, how successful it will be, etc. Although this is an important aspect of what defines the game, it is not critical analysis . Furthermore, the reviewer should dedicate some to time to speaking about what the game has to contribute to its genre (Is it doing something new? Does it present the player with unusual choices? Can it set a new standard for what games of this type should include?)." (Mark Mullen, "On Second Thought . . ." Rhetoric/Composition/Play Through Video Games: Reshaping Theory and Practice , ed. by Richard Colby, Matthew S.S. Johnson, and Rebekah Shultz Colby. Palgrave Macmillan, 2013)

Critical Thinking and Visuals

 "The current critical turn in rhetoric and composition studies underscores the role of the visual, especially the image artifact, in agency. For instance, in Just Advocacy? a collection of essays focusing on the representation of women and children in international advocacy efforts, coeditors Wendy S. Hesford and Wendy Kozol open their introduction with a critical analysis of a documentary based on a picture: the photograph of an unknown Afghan girl taken by Steve McCurry and gracing the cover of National Geographic in 1985. Through an examination of the ideology of the photo's appeal as well as the 'politics of pity' circulating through the documentary, Hesford and Kozol emphasize the power of individual images to shape perceptions, beliefs, actions, and agency." (Kristie S. Fleckenstein, Vision, Rhetoric, and Social Action in the Composition Classroom . Southern Illinois University Press, 2010)

Related Concepts

• Analysis  and  Critical Essay
• Book Report
• Close Reading
• Critical Thinking
• Discourse Analysis
• Evaluation Essay
• Explication
• Problem-Solution
• Rhetorical Analysis
• Definition and Examples of Explication (Analysis)
• Rhetorical Analysis Definition and Examples
• Quotes About Close Reading
• How to Write a Critical Essay
• What Is a Critique in Composition?
• Definition and Examples of Analysis in Composition
• Definition and Examples of Evaluation Essays
• Understanding the Use of Language Through Discourse Analysis
• The Definition of a Review in Composition
• Audience Analysis in Speech and Composition
• Argument (Rhetoric and Composition)
• Holistic Grading (Composition)
• Persuasion and Rhetorical Definition
• A Writing Portfolio Can Help You Perfect Your Writing Skills
• A Rhetorical Analysis of U2's 'Sunday Bloody Sunday'
• Rhetoric: Definitions and Observations

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• Applying Critical Thinking
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

Critical thinking refers to deliberately scrutinizing and evaluating theories, concepts, or ideas using reasoned reflection and analysis. The act of thinking critically implies moving beyond simply understanding information, but questioning its source, its production, and its presentation in order to expose potential bias or researcher subjectivity [i.e., being influenced by personal opinions and feelings rather than by external determinants ] . Applying critical thinking to investigating a research problem involves actively challenging assumptions and questioning the choices and potential motives underpinning how the author designed the study, conducted the research, and arrived at particular conclusions or recommended courses of action.

Mintz, Steven. "How the Word "Critical" Came to Signify the Leading Edge of Cultural Analysis." Higher Ed Gamma Blog , Inside Higher Ed, February 13, 2024; Van Merriënboer, Jeroen JG and Paul A. Kirschner. Ten Steps to Complex Learning: A Systematic Approach to Four-component Instructional Design . New York: Routledge, 2017.

Thinking Critically

Applying Critical Thinking to Research and Writing

Professors like to use the term critical thinking; in fact, the idea of being critical permeates much of higher education writ large. In the classroom, the idea of thinking critically is often mentioned by professors when students ask how they should approach a research and writing assignment [other approaches your professor might mention include interdisciplinarity, comparative, gendered, global, etc.]. However, critical thinking is more than just an approach to research and writing. It is an acquired skill used in becoming a complex learner capable of discerning important relationships among the elements of, as well as integrating multiple ways of understanding applied to, the research problem. Critical thinking is a lens through which you holistically interrogate a topic.

Given this, thinking critically encompasses a variety of inter-related connotations applied to college-level research and writing * :

• Integrated and Multi-Dimensional . Critical thinking is not focused on any one element of research, but rather, is applied holistically throughout the process of identifying the research problem, reviewing of literature, applying methods of analysis, describing the results, discussing their implications, and, if appropriate, offering recommendations for further research. The act of thinking critically is also non-linear [i.e., applies to going back and changing prior thoughts when new evidence emerges]; it permeates the entire research endeavor from contemplating what to write to proofreading the final product.
• Humanize Research . Thinking critically can help humanize the research problem by extending the scope of your analysis beyond the boundaries of traditional approaches to studying the topic. Traditional approaches can include, for example, sampling homogeneous populations, considering only certain factors related to investigating a phenomenon, or limiting the way you frame or represent the context of your study. Critical thinking can help reveal opportunities to incorporate the experiences of others into the research, creating a more representative examination of the research problem.
• Normative . This refers to the idea that critical thinking can be used to challenge prior assumptions in ways that advocate for social justice, equity, and inclusion and which can lead to research having a more transformative and expansive impact. In this respect, critical thinking can be a method for breaking away from dominant culture norms so as to produce research outcomes that illuminate previously hidden aspects of exploitation and injustice.
• Power Dynamics . Research in the social and behavioral sciences often includes examining aspects of power and influence that shape social relations, organizations, institutions, and the production and maintenance of knowledge. This approach encompasses studying how power operates, how it can be acquired, and how power and influence can be maintained. Critical thinking can reveal how societal structures perpetuate power and influence in ways that marginalizes and oppresses certain groups or communities within the contexts of history , politics, economics, culture, and other factors.
• Reflection . A key aspect of critical thinking is practicing reflexivity; the act of turning ideas and concepts back onto yourself in order to reveal and clarify your own beliefs, assumptions, and perspectives. Being critically reflexive is important because it can reveal hidden biases you may have that could unintentionally influence how you interpret and validate information. The more reflexive you are, the better able and more comfortable you are about opening yourself up to new modes of understanding.
• Rigorous Questioning . Thinking critically is guided by asking questions that lead to addressing complex concepts, principles, theories, or problems more effectively and to help distinguish what is known from from what is not known [or that may be hidden]. In this way, critical thinking involves deliberately framing inquiries not just as research questions, but as a way to focus on systematic, disciplined,  in-depth questioning concerning the research problem and your positionality as a researcher.
• Social Change . An overarching goal of critical thinking applied to research and writing is to seek to identify and challenge sources of inequality, exploitation, oppression, and marinalization that contributes to maintaining the status quo within institutions of society. This can include entities, such as, schools, courts, businesses, government agencies, religious centers, that have been created and maintained through certain ways of thinking within the dominant culture.

In writing a research paper, the act of critical thinking applies most directly to the literature review and discussion sections of your paper . In reviewing the literature, it is important to reflect upon specific aspects of a study, such as, determining if the research design effectively establishes cause and effect relationships or provides insight into explaining why certain phenomena do or do not occur, assessing whether the method of gathering data or information supports the objectives of the study, and evaluating if the assumptions used t o arrive at a specific conclusion are evidence-based and relevant to addressing the research problem. An assessment of whether a source is helpful to investigating the research problem also involves critically analyzing how the research challenges conventional approaches to investigations that perpetuate inequalities or hides the voices of others.

Critical thinking also applies to the discussion section of your paper because this is where you interpret the findings of your study and explain its significance. This involves more than summarizing findings and describing outcomes. It includes reflecting on their importance and providing reasoned explanations why the research study is important in filling a gap in the literature or expanding knowledge and understanding about the topic in ways that inform practice. Critical reflection helps you think introspectively about your own beliefs concerning the significance of the findings but in ways that avoid biased judgment and decision making.

* Mintz, Steven. "How the Word "Critical" Came to Signify the Leading Edge of Cultural Analysis." Higher Ed Gamma Blog , Inside Higher Ed, February 13, 2024; Suter, W. Newton. Introduction to Educational Research: A Critical Thinking Approach. 2nd edition. Thousand Oaks, CA: SAGE Publications, 2012

Behar-Horenstein, Linda S., and Lian Niu. “Teaching Critical Thinking Skills in Higher Education: A Review of the Literature.” Journal of College Teaching and Learning 8 (February 2011): 25-41; Bayou, Yemeserach and Tamene Kitila. "Exploring Instructors’ Beliefs about and Practices in Promoting Students’ Critical Thinking Skills in Writing Classes." GIST–Education and Learning Research Journal 26 (2023): 123-154; Butcher, Charity. "Using In-class Writing to Promote Critical Thinking and Application of Course Concepts." Journal of Political Science Education 18 (2022): 3-21; Loseke, Donileen R. Methodological Thinking: Basic Principles of Social Research Design. Thousand Oaks, CA: Sage, 2012; Hart, Claire et al. “Exploring Higher Education Students’ Critical Thinking Skills through Content Analysis.” Thinking Skills and Creativity 41 (September 2021): 100877; Sabrina, R., Emilda Sulasmi, and Mandra Saragih. "Student Critical Thinking Skills and Student Writing Ability: The Role of Teachers' Intellectual Skills and Student Learning." Cypriot Journal of Educational Sciences 17 (2022): 2493-2510.Van Merriënboer, Jeroen JG and Paul A. Kirschner. Ten Steps to Complex Learning: A Systematic Approach to Four-component Instructional Design. New York: Routledge, 2017; Yeh, Hui-Chin, Shih-hsien Yang, Jo Shan Fu, and Yen-Chen Shih. "Developing College Students’ Critical Thinking through Reflective Writing." Higher Education Research & Development 42 (2023): 244-259.

• << Previous: Academic Writing Style
• Next: Choosing a Title >>
• Last Updated: Apr 11, 2024 1:27 PM
• URL: https://libguides.usc.edu/writingguide

Understanding Critical Theory

Charlotte Nickerson

Research Assistant at Harvard University

Undergraduate at Harvard University

Charlotte Nickerson is a student at Harvard University obsessed with the intersection of mental health, productivity, and design.

Learn about our Editorial Process

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

On This Page:

• Critical theories aim to change and critique society as a whole by finding the underlying assumptions in social life that prevent people from participating in a “true democracy.”
• Critical Theory developed in the Frankfurt school from scholars such as Horkheimer and Adorno with an emphasis on examining and deconstructing fascism and mass media.
• Habermas continued the tradition of Critical Theory through his notion of the lifeworld and the public sphere. He theorized that political and economic institutions had invaded public life, leading to a lack of nuance in discourse and preventing people from participating in a “real democracy.”
• Critical Theory morphed into critical legal theory in the latter 20th century, which eventually gave rise to branches such as critical race and critical gender theory.

Critical Theory is a social theory that aims to critique and change society as a whole. Critical theories attempt to find the underlying assumptions in social life that keep people from fully and truly understanding how the world works.

These underlying assumptions, in the view of critical theories, create a “False consciousness” that actively undermines people’s progress toward a true democracy.

Critical Theory, first emerging from Horkheimer at the Frankfurt School, bridges its reach to ethics, political philosophy, and the philosophy of history.

Critical theorists claim that the social sciences must integrate philosophy into their methods to make their findings practical to advance the moral cause of freeing humans from circumstances such as domination and oppression (Horkheimer, 1993).

While Critical Theory is most associated with the Frankfurt School, beginning with Horkheimer and Adorno and ending with Marcuse and Habermas.

Critical Theory has extended to many other disciplines, such as feminism, critical race theory, and critiques of colonialism.

Critical Theories of Gender

Critical theories of gender are concerned with the ways in which literature and other cultural media reinforce or undermine the economic, political, social, and psychological oppression of people of various genders.

Critical feminist theories, in particular, focus on issues of power and seek to explain the origins and consequences of gender relations, particularly those that privilege men.

They study the ways that assumptions and ideologies around gender are produced, reproduced, resisted, and changed in and through the everyday experiences of men and women (Coakley and Pike, 2014).

Like critical theories of race, critical theories of gender see their origins in critical legal studies.

Critical Theories of Race

Racism is prevalent in everyday life, and Critical Theory scholars agree that the ideology and assumptions of racism are so ingrained in the political and legal structures of society as to be nearly unrecognizable (Parker and Roberts, 2005).

The critical study of race and ethnicity is centered on examining the experiences of racial oppression in the context of an attempt to challenge existing assumptions about the construction of race.

Critical theories of race can also trace their roots to philosophical, historical, and sociological critiques of oppression, such as Marxism , feminist theory, and postcolonialism (Parker and Roberts, 2005).

Critical Race Theory emerged as an outgrowth of the critical legal studies movement originating at Harvard Law School in the early 1980s.

Law professors and students criticized how the law served to privilege the wealthy and powerful in US society while impeding the poor from using the courts as a means of writing their own wrongs (Parker and Roberts, 2005).

One of the main tenets of critical race theory is that, while classical racism has subsided, everyday racism remains alive, characterized by mundane practices and events infused with varying degrees of racism, such as “microaggressions” and other subtle, automatic, non-verbal exchanges.

For example, an educational institution can commit a microaggression by creating hostile environmental encounters for African Americans, such as seeing black males engaged in black youth culture as predatory (Parker and Roberts, 2005).

Globalization

One criticism of the Frankfurt school is that it lacked a solid grounding in social reality (Kozlarek, 2001).

Kozlarek (2001) argues that Horkheimer and Adorno take an overly euro-centric stance on the world and that Eurocentrism is a crucial impediment to Critical Theory, and suggests alternatives to the Eurocentric worldview in modern Critical Theory research.

Rather than philosophically constructing ideas of what should be normal and an ideal society, Kozlarek claims, one must ask where the underlying assumptions of Critical Theory come from and what their sociocultural functions were and are.

Critical Theory and the Frankfurt School

Critical Theory has many distinct historical phases spanning several generations; however, it was born in the Frankfurt School.

The Frankfurt School opened as the Institute for Social Research in the 1920s in the social context of rising fascism in Germany and Italy. The theorists of the Frankfurt school went into exile in Switzerland and the United States before returning to Frankfurt in 1953.

According to the theorists in the Frankfurt School, a so-called “Critical Theory” could be distinguished from a “traditional” theory in that critical theories have a specific practical purpose, such as promoting an understanding of the world that leads to human “emancipation from slavery” (Horkheimer, 1973).

In order for a Critical Theory to be a Critical Theory in Horkheimer’s view, it must be explanatory, practical, and normative.

By these, Horkheimer means that the theory must explain what is wrong with the current social reality, identify the people and actors that can change it, and provide both achievable, practical objectives for social transformation and ways of criticizing those objectives (Horkheimer, 1972).

Research that furthers Critical Theory must, in this view, combine psychological, cultural, and social dimensions, as well as an examination of institutional forms of domination.

Horkheimer’s Critical Theory, heavily influenced by Marxism, aimed to transform contemporary capitalist society into a more consensual one.

By this, Horkheimer meant that a capitalist society could only be transformed by becoming more democratic in order to make sure that all of the conditions of social life that are controllable by people can be determined by the consensus of the people living in that society (Horkheimer, 1972).

The Frankfurt School theorists extended the work of Marx, Weber, and Freud, as well as considering the pull of authoritarian regimes, the relationship between art, technology, mass society, and social psychology.

Initial Concerns of the Frankfurt School

In its initial phases, Critical Theory attempted to differentiate the idea of a “real democracy” from the forms of government then present in Western societies.

According to critical theorists, real democracy is rational because it allows individuals to gain control over the social processes that affect themselves and their life choices.

The next phases of Critical Theory were concerned with anti-democratic trends, such as the emergence of fascism in the 1930s. These studies focused on phenomena such as fascist states and authoritarian personalities.

Horkheimer saw these anti-democratic trends and a process called reification as undermining people’s ability to determine their own social circumstances.

Reification is a complex idea where something that is immaterial — like happiness, fear, or evil — is treated as a material thing. In the context of Critical Theory’s early musings about authoritarianism, this meant that the spread of increasingly abstract but fascist social principles led to societies that were more fascist on a concrete level.

To critical theorists in the 1940s, reification happened at two different levels. Firstly, reification happened at a small scale, and theorists could examine the psychological conditions that lead to people supporting democracy or authoritarianism.

Secondly, reification also happened at a larger scale and over a longer time period, where people explained enduring societal trends by projecting their democratic or authoritarian principles onto retellings of history (Stanford Encyclopedia of Philosophy).

Habermas sought to develop a level of analysis between that of the individual and the entire society. He called this intermediate level the “public sphere” or Offentlichkeit.

Ideology Critique

One of the main concerns of the scholars of the Frankfurt School was the rise of “mass culture” — the technological developments that allow cultural products, such as music, movies, and art, to be distributed on a massive scale.

Horkheimer, Adorno, Benjamin, and Marcuse speculated that technology allowed audiences to consume content passively rather than actively engaging with one another, making people intellectually inactive and politically passive.

Contemporary Critical Theory: Habermas

Habermas was a member of the second generation of Critical Theory. Habermas’s Critical Theory went beyond the theoretical roots of the Frankfurt school and became more life-American pragmatism, which holds that both the meaning and the truth of any idea are a function of its practical outcome.

Haabermas’ work in Critical Theory was concerned with two main issues: developing a justification for the normative dimension of critical social theory and the problem of establishing a connection between the theory and political practice.

These problems were carried over from the Frankfurt school (Roderick, 1986).

The first of these problems dealt with what counts as a rational criticism of society, while the second is directed at how these criticisms can be used to construct a society that is more rational.

Habermas dealt with three kinds of knowledge: empirical knowledge, which is technical or scientific; Hermeneutic or interpretive knowledge, which is interested in human understanding and cooperation; and critical knowledge, which is focused on freeing humans from societal assumptions.

Habermas acknowledged that science cannot be value-free and that those who study society are part of its subject matter. Critical Theory, to Habermas, unmasks the distortions, representations, and politics embedded in our knowledge and speech.

The Lifeworld

Habermas focused on the idea of the lifeworld, which is a person’s everyday life and experiences. The lifeworld encompasses culture, social relations, and everyday communication.

Habermas’s theory is that the lifeworld is increasingly being taken over by political and economic systems. As politics is about power, attempts at becoming more powerful by politicians and the interests of political parties affect everyday lives.

People also, according to Habermas, are heavily influenced by the system of capitalism – they talk about companies, work at them, and consume constantly. Habermas believed that the lifeworld could not be reduced to what he calls “media ” — such as the power of the state and money.

However, the modern state and economic systems have imposed their media on the lifeworld, and as money and politics seek to dominate the local lifeworld of people, they impede them from achieving a “true democracy.”

To Habermas, value rationality — attempts to achieve value-based goals that make life meaningful, such as being a good father — is tied to the lifeworld. Meanwhile, instrumental rationality — such as calculations of the means necessary to achieve a particular end — is tied to the state and economy.

The everyday relationships of people in society in modernity have been overtaken by a social structure that promotes money and power as keys to success and what is seen as morally right.

Habermas emphasized that money and power, and the instrumental relationships that people form in trying to achieve them, cannot be the sole foundation for consensus and communication.

The Colonization of the Public Sphere

Another idea that Habermas considers is the colonization of the public sphere. Habermas believed that news sources motivated by profit, promoting entertainment, and oversimplification dominated public life.

Rather than focusing on the nuances of issues through lengthy public debate, these colonizers encouraged arguments between people with simplified perspectives.

Habermas sets an ideal for public discourse, which he calls ideal speech communities. Habermaass differentiated communicative action — any action that someone takes with the intent to communicate — from strategic speech — which is instrumentally based and permeates the lifeworld under capitalist-based societies.

Rather than trying to simply communicate with people, those who use strategic speech are trying to manipulate people into achieving an end.

Habermas believed that the promise and hope of enlightenment and modernity is a society where people can talk in order to reach a consensus and make reasoned decisions. To have this, everyone needs an equal chance to speak without coercion, where any topic can be discussed, and where everyone can keep their speech free from ideology.

Such a situation constitutes the ideal speech community, which is the basis for and gives rise to civil society. This civil society must develop in the context of a liberal political culture that promotes equality and draws strong boundaries between large institutions and the lifeworld.

In the public sphere of real democracy, the state’s power is limited, and people can use persuasion but cannot obtain political power in the public sphere (Roderick, 1986).

Critical Evaluation

As a broad-ranging philosophical project, Critical Theory has experienced many tensions between theorists both in the same generation and across different generations of the tradition.

Critical Theory has also drawn criticism from outside.

Perhaps the most major criticism of Critical Theory is that it fails to provide rational standards by which it can show that it is superior to other theories of knowledge, science, or practice.

Gibson (1986), for example, says that critical theories suffer from cliquishness, conformity, elitism, immodesty, anti-individualism, contradictoriness, criticalness, and naivety.

As Hughes and Hughes say of Habermas’s theory of ideal public discourse, it “says much about rational talkers talking, but very little about actors acting: Felt, perceptive, imaginative, bodily experience does not fit these theories” (1990).

Critical Theory has also been criticized from a feminist perspective. This feminist criticism of Critical Theory contends that critical theories can be as narrow and oppressive as the rationalization, bureaucratization, and cultures they seek to unmask and change.

Ellsworth (1989), for example, acknowledges that critical theories are often so tied to their vision of the truth that they fail to see themselves as one of many voices and that the enlightening of the false consciousness of others may be a form of domination rather than liberation.

Coakley, J., & Pike, E. (2014). EBOOK: Sports in Society. McGraw Hill.

Critical Theory. (2005). In Stanford Encyclopedia of Philosophy. Retrieved from https://plato.stanford.edu/entries/critical-theory/

Ellsworth, E. (1989). Why doesn’t this feel empowering? Working through the repressive myths of critical pedagogy. Harvard educational review, 59(3), 297-325.

Gibson, R. (1986). Critical Theory and education. Hodder and Stoughton.

Habermas, Jürgen (1990a): Moral consciousness and communicative action. Christian Lenhardt and Shierry Weber Nicholsen (trans.). Maldon, MA: Polity Press.

Horkheimer, M. (1972). Critical Theory: Selected essays (Vol. 1). A&C Black.

Horkheimer, M. (1993). The present situation of social philosophy and the tasks of an institute for social research. Between philosophy and social science: Selected early writings, 11.

Horkheimer, M., & Adorno, T. (1973). The Dialectic of Enlightenment (London, Allen Lane).

Kozlarek, O. (2001). Critical Theory and the challenge of globalization. International Sociology, 16(4), 607-622.

Parker, L., & Roberts, L. (2005). Critical theories of race. RESEARCH METHODS, 74.

Roderick, R. (1986). Habermas and the foundations of Critical Theory. Macmillan International Higher Education.

• About the LSE Impact Blog
• Comments Policy
• Popular Posts
• Recent Posts
• Subscribe to the Impact Blog
• Write for us
• LSE comment

September 4th, 2018

Impact from critical research: what might it look like and what support is required.

1 comment | 13 shares

Estimated reading time: 5 minutes

Critical research is often impassioned by a desire for social change. Yet as research that challenges the status quo – by unpacking the socio-historical contingency of meanings and exposing the reproduction of structural inequalities of power – critical research often faces a more challenging pathway to impact. As demands for demonstrating impact are increasingly woven throughout the funding and institutional architectures of higher education, Smith and Stewart are not alone in raising concerns that the impact agenda could adversely affect critical and blue-skies research, favouring instead research that lends itself more easily to societal uptake.

With the draft guidelines for REF2021 open for consultation , now, perhaps more than ever, is a good moment to think about what impact from critical research could look like. And how assessment frameworks could support, rather than squeeze out, space for critical research. To this end, this post outlines four modes of critical research impact: challenging policy; empowering resistances; platforming voices; and nurturing new critical publics.

Research impact: to engage or not?

Anxiety around the impact agenda arises from the increasing instrumentalisation of knowledge, the corporatisation of UK higher education, and the relationship between assessment metrics and neoliberalism ( Pain et al. 2011 , Pain 2014 , Gregson et al. 2012 , Olssen 2015 ). As well as fears that impact will prioritise certain kinds of knowledge, there are also concerns it rewards particular types of researcher; academic elites with established reputations and influential networks rather than early-career or international researchers ( Smith and Stewart 2016 ). These are vitally important concerns. Yet some scholars also identify opportunities for “doing impact differently”. Participatory Action Research (PAR) had delivered social benefits through collaboration with non-academic partners long before research impact became instrumentalised within academic assessments (Pain et al. 2011 ). Pain in particular seeks to reclaim impact as “walking together”, rather than “striking a blow” ( 2014 ; see also Evans 2016 ). Likewise, Reed and Chubb suggest that impact is a provocation to reconsider our intrinsic motivations for research and epistemic responsibilities. Re-engaging with these, they argue, incentivises impact without research becoming driven by external incentives ( Reed and Chubb 2018) . Perhaps there is merit in heeding Back’s argument that, as soon as someone suggests “this would make a good impact case study”, we should be alert to how our attention is being directed. However, does pursuing impact necessarily put us, as Back ( 2015) suggests, “on the side of the powerful”?

Exempting Laing et al’s recent work in education (2018), there is a strange silence around what impact in critical research might look like beyond PAR. Suggestions that not all research, and not all researchers, need to realise impact open space for critical research only through exception, negation, or omission. Instead, building from Pain et al.’s emphasis on the “political imperative to restate the kind of academy in which we want to work” ( 2011:187 ), I focus on how impact might be pursued in ways that support and enrich critical agendas.

What might critical research impact look like?

A review of the REF2014 impact case studies yields the following simple typology, which might provide a useful starting point:

  Figure 1: four possible modes of critical research impact

Mode 1: challenging policy.

The UK Government Magenta Book – the UK Government’s guidance for policy evaluation – in principle endorses the need for critical approaches that unpack assumptions underlying policy and analyses. Confronting mainstream policy head-on could involve policy amendments by highlighting the implications of existing policies on particular underrepresented groups, geographies, or concerns. More transformative policy change is likely to involve election manifesto writers and/or targeted social pressure rather than consultations within existing policy cycles.

• Example REF2014 Impact Case Study: “ The struggle for material democratisation: contributing to the defence of essential water and sanitation services in Latin America ” (Newcastle University). Castro’s research into the privatisation and commodification of water and sanitation services in Brazil achieved legal and policy change.

Mode 2: Empowering resistance

Greater traction around critical research findings is sometimes found amongst activist organisations with a degree of policy standing. For these organisations, research, or the connections it articulates, may help to strengthen their discursive position or alternative vision.

• Example REF 2014 Impact Case Study: “ Questioning the ‘financialisation of nature’: influencing international policy thinking on biodiversity conservation ” (Birkbeck College). Sullivan’s demonstration of how contemporary environmental policy trends intensify socio-economic inequality and increase biodiversity loss was used by advocacy groups to challenge neoliberal approaches to biodiversity conservation.

Mode 3: Platforming voices

This mode is typified by PAR, where working with marginalised communities often co-produces research questions around non-academic challenges, foregrounds and empowers underrepresented voices, and sometimes challenges participant narratives through deliberation (see Roberts and Escobar’s work on citizen juries ).

• Example REF2014 Impact Case Study: “ ‘The Cambridge Project’ empowering gypsy/traveller communities through collaborative participation action research ” (Buckinghamshire New University). Greenfields’ work with gypsy and traveller communities first empowered participants (as interviewers) to identify their own needs, and then mobilised these “ hitherto invisible” needs within UK policy circles.

Mode 4: Nurturing new critical publics

Critical research can inspire new critically engaged citizens. Gregson et al. (2012) argue that engaging with schools can “ reclaim critical praxis and constitute new critical subjects ”. With rapidly developing digital technologies and the growing role of social media in generating critical publics, there are opportunities to think about new forms of media through which critical publics become fashioned, politically engaged, and/or mobilised.

• Example REF2014 Impact Case Study: “ ‘Follow The Things’: developing critical pedagogies to promote geographically-informed and ethically-aware consumption in school geography curriculum ” (University of Exeter). Cook’s pedagogical approach to examining the social relations and ethics of international trade has promoted critical perspectives on consumption within schools and universities.

Recommendations for supporting critical research impact

To support and encourage critical forms of research in the pursuit of societal change, assessments of research impact should bear in mind the following:

• Direct policy citation of critical research is rare. Change is more likely through political ownership of ideas, and what Pain et al. have called a “more diverse and porous series of smaller transformative actions that arise through changed understanding among all of those involved” ( 2011: 187 ). Retaining a strong focus on narratives in impact assessment and recognising the role played by relationships are both important.
• Changing the terms of debate is difficult and slow, with quick wins unlikely. Critical research impact may require longer timeframes to develop, materialising outside or cross-cutting assessment periods.
• Marginal/alternative organisations may be smaller and/or more local in reach. Assessing significance and reach together helps to prevent reach from dominating.

The typology presented here is basic, provisional, and by no means exhaustive. Its goal is to prompt debate and expand possibilities for thinking about critical research impact. With similar conversations reportedly held at Open (2013), Bristol (2013), and Glasgow (2015) universities, it would be great to hear more about these discussion findings – especially in thinking through forms of impact beyond policy.

Note: This article gives the views of the author, and not the position of the LSE Impact Blog, nor of the London School of Economics. Please review our  comments policy  if you have any concerns on posting a comment below.

Featured image credit: Paul Skorupskas , licensed under a  CC0 1.0  license.

About the author

Ruth Machen is a Research Fellow at Newcastle University. Her research on science-policy interaction focuses on environmental knowledge where her recent work takes a critical look at science-policy translation – Towards a Critical Politics of Translation .

• Pingback: 2018 in review: round-up of our top posts on research evaluation and impact | Impact of Social Sciences

Leave a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Notify me of follow-up comments by email.

Related Posts

The great potential of citizen science: restoring the role of tacit knowledge and amateur discovery.

November 5th, 2014.

Aiming to create impacts on societies in 14 countries, the collaborative MYPLACE project partners traditional journal publishing with a continuous online presence.

February 2nd, 2012.

Collaborate or die? Interdisciplinary work holds great promise but goal-oriented assumptions must be challenged.

April 2nd, 2015.

Book Review: Who’s Asking? Native Science, Western Science and Science Education by Douglas L. Medin and Megan Bang

June 15th, 2014.

Visit our sister blog LSE Review of Books

What Does It Mean to be ‘Critical’ in IS Research?

Cite this chapter.

• Carole Brooke 4  

907 Accesses

The main aim of this paper is to explore what it means to conduct ‘critical’ research in IS. In order to begin this, it is necessary to look beyond the scope of IS inquiry itself to other disciplines, especially organizational analysis. A preliminary review is made of the state of critical thinking in the fields of information systems and organization. In addressing the question ‘what is critical research?’ the paper shows how definitions have changed and broadened over time.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Unable to display preview.  Download preview PDF.

Ackroyd, S. and Thompson, P. (1999) Organizational Misbehaviour (Sage, London).

Google Scholar  

Alvesson, M. and Deetz, S. (2000) Doing Critical Management Research (Sage, London).

Alvesson, M. and Willmott, H. (eds.) (1992) Critical Management Studies (Sage, London).

Alvesson, M. and Willmott, H. (1996) Making Sense of Management: a critical introduction (Sage, London).

Ashenden, S. and Owen, D. (eds.) (1999) Foucault contra Habermas (Sage, London).

Beirne, M., Ramsay, H. and Panteli, A. (1998) Developments in Computing Work: control and contradiction in the software labour process. In Thompson, P. and Warhurst, C. (eds.) Workplaces of the Future . (Macmillan Business, Basingstoke), 142–62.

Chapter   Google Scholar  

Boje, D. (2001) What is Critical Postmodern Theory? Available online at http://cbae.nmsu.edu /~dboje/

Clegg, S.R. (2001) Changing Concepts of Power, Changing Concepts of Politics . Available online at: aom.pace.edu /cms/Washington/clegg.htm

Conway, D.W. (1999) Pas de Deux: Habermas and Foucault in Genealogical Communication. In Ashenden, S. and Owen, D. (eds.) Foucault contra Habermas , (Sage, London), 60–89.

Dean, M. (1999) Normalising Democracy: Foucault and Habermas on democracy, liberalism and law. In Ashenden, S. and Owen, D. (eds.) Foucault contra Habermas , (Sage, London), 166–94.

Fleming, P. (2001) Beyond the Panopticon? Book Review of Organizational Misbehaviour by Ackroyd and Thompson (1999). Ephemera , 1 , 2, 190–4.

Foucault, M. (1972) The Archaeology of Knowledge (Routledge, London).

Habermas, J. (1984) The Theory of Communicative Action: Volume 1: Reason and the Rationalization of Society; Volume 2: Lifeworld and System: A Critique of Functionalist Reason . Translated by Thomas McCarthy. (Heinemann, London).

Heidegger, M. (1962) Being and Time (Blackwell, Oxford).

Jackson, M.C. (2000) Systems Approaches to Management (Kluwer, New York).

Klein, H.K. and Hirschheim, R. (1991) Rationality Concepts in Information System Development Methodologies. Accounting, Management and Information Technologies , 1 , 2, 157–87.

Article   Google Scholar  

Lyytinen, K. (1992) Information Systems and Critical Theory. In Alvesson, M., and Willmott, H. (eds.) Critical Management Studies . (Sage, London), 159–80.

Lyytinen, K. and Klein, H.K. (1985) The Critical Theory of Jürgen Habermas as a Basis for a Theory of Information Systems, in Mumford, E., et al . (eds.) (Elsevier Science, North-Holland), 207–25.

Mingers, J. (1992) Technical, Practical and Critical OR: past, present and future? In M. Alvesson and H. Willmott (eds.) Critical Management Studies . (Sage, London), 90–112.

Reed, M. (1993) Organizations and Modernity: continuity and discontinuity in organization theory. In Hassard, J. and Parker, M. (eds.) Postmodernism and Organisations . (Sage, London), 163–82.

Scarbrough, H. and Corbett, J. M. (1992) Technology and Organization: power, meaning and design . (Routledge, London).

Tully, J. (1999) To Think and Act Differently. In Ashenden, S. and Owen, D. (eds.) Foucault contra Habermas . (Sage, London), 90–142.

Valero-Silva, N. (1996) A Foucauldian Reflection on Critical Systems Thinking. In Flood, R. and Romm, N. (eds.) Critical Systems Thinking: current research and practice . (Plenum Press, New York), 63–79.

Valero-Silva, N. (2001) Demystifying the Theory and Practice of Critical Systems Thinking. In Wilby, J. and Allen, J.K. (eds.) Proceedings of the 45th Annual Conference of the International Society for the Systems Sciences . Asilomar, California, July 8–13.

Walsham, G. (1993) Interpreting Information Systems in Organizations . (John Wiley & Sons, Chichester).

Willmott, H. (1993) Breaking the Paradigm Mentality. Information and Organization , 14 , 5, 681–719.

Download references

Author information

Authors and affiliations.

Faculty of Business and Management, University of Lincoln, Brayford Pool, UK

Carole Brooke

You can also search for this author in PubMed   Google Scholar

Editor information

Editors and affiliations.

London School of Economics and Political Science, UK

Leslie P. Willcocks

Oxford University, UK

Chris Sauer

University of Missouri-St Louis, USA

Mary C. Lacity

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Association for Information Technology Trust

About this chapter

Brooke, C. (2016). What Does It Mean to be ‘Critical’ in IS Research?. In: Willcocks, L.P., Sauer, C., Lacity, M.C. (eds) Enacting Research Methods in Information Systems: Volume 1. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-319-29266-3_3

Download citation

DOI : https://doi.org/10.1007/978-3-319-29266-3_3

Publisher Name : Palgrave Macmillan, Cham

Print ISBN : 978-3-319-29265-6

Online ISBN : 978-3-319-29266-3

eBook Packages : Business and Management Business and Management (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• CBE Life Sci Educ
• v.17(1); Spring 2018

Understanding the Complex Relationship between Critical Thinking and Science Reasoning among Undergraduate Thesis Writers

Jason e. dowd.

† Department of Biology, Duke University, Durham, NC 27708

Robert J. Thompson, Jr.

‡ Department of Psychology and Neuroscience, Duke University, Durham, NC 27708

Leslie A. Schiff

§ Department of Microbiology and Immunology, University of Minnesota, Minneapolis, MN 55455

Julie A. Reynolds

Associated data.

This study empirically examines the relationship between students’ critical-thinking skills and scientific reasoning as reflected in undergraduate thesis writing in biology. Writing offers a unique window into studying this relationship, and the findings raise potential implications for instruction.

Developing critical-thinking and scientific reasoning skills are core learning objectives of science education, but little empirical evidence exists regarding the interrelationships between these constructs. Writing effectively fosters students’ development of these constructs, and it offers a unique window into studying how they relate. In this study of undergraduate thesis writing in biology at two universities, we examine how scientific reasoning exhibited in writing (assessed using the Biology Thesis Assessment Protocol) relates to general and specific critical-thinking skills (assessed using the California Critical Thinking Skills Test), and we consider implications for instruction. We find that scientific reasoning in writing is strongly related to inference , while other aspects of science reasoning that emerge in writing (epistemological considerations, writing conventions, etc.) are not significantly related to critical-thinking skills. Science reasoning in writing is not merely a proxy for critical thinking. In linking features of students’ writing to their critical-thinking skills, this study 1) provides a bridge to prior work suggesting that engagement in science writing enhances critical thinking and 2) serves as a foundational step for subsequently determining whether instruction focused explicitly on developing critical-thinking skills (particularly inference ) can actually improve students’ scientific reasoning in their writing.

INTRODUCTION

Critical-thinking and scientific reasoning skills are core learning objectives of science education for all students, regardless of whether or not they intend to pursue a career in science or engineering. Consistent with the view of learning as construction of understanding and meaning ( National Research Council, 2000 ), the pedagogical practice of writing has been found to be effective not only in fostering the development of students’ conceptual and procedural knowledge ( Gerdeman et al. , 2007 ) and communication skills ( Clase et al. , 2010 ), but also scientific reasoning ( Reynolds et al. , 2012 ) and critical-thinking skills ( Quitadamo and Kurtz, 2007 ).

Critical thinking and scientific reasoning are similar but different constructs that include various types of higher-order cognitive processes, metacognitive strategies, and dispositions involved in making meaning of information. Critical thinking is generally understood as the broader construct ( Holyoak and Morrison, 2005 ), comprising an array of cognitive processes and dispostions that are drawn upon differentially in everyday life and across domains of inquiry such as the natural sciences, social sciences, and humanities. Scientific reasoning, then, may be interpreted as the subset of critical-thinking skills (cognitive and metacognitive processes and dispositions) that 1) are involved in making meaning of information in scientific domains and 2) support the epistemological commitment to scientific methodology and paradigm(s).

Although there has been an enduring focus in higher education on promoting critical thinking and reasoning as general or “transferable” skills, research evidence provides increasing support for the view that reasoning and critical thinking are also situational or domain specific ( Beyer et al. , 2013 ). Some researchers, such as Lawson (2010) , present frameworks in which science reasoning is characterized explicitly in terms of critical-thinking skills. There are, however, limited coherent frameworks and empirical evidence regarding either the general or domain-specific interrelationships of scientific reasoning, as it is most broadly defined, and critical-thinking skills.

The Vision and Change in Undergraduate Biology Education Initiative provides a framework for thinking about these constructs and their interrelationship in the context of the core competencies and disciplinary practice they describe ( American Association for the Advancement of Science, 2011 ). These learning objectives aim for undergraduates to “understand the process of science, the interdisciplinary nature of the new biology and how science is closely integrated within society; be competent in communication and collaboration; have quantitative competency and a basic ability to interpret data; and have some experience with modeling, simulation and computational and systems level approaches as well as with using large databases” ( Woodin et al. , 2010 , pp. 71–72). This framework makes clear that science reasoning and critical-thinking skills play key roles in major learning outcomes; for example, “understanding the process of science” requires students to engage in (and be metacognitive about) scientific reasoning, and having the “ability to interpret data” requires critical-thinking skills. To help students better achieve these core competencies, we must better understand the interrelationships of their composite parts. Thus, the next step is to determine which specific critical-thinking skills are drawn upon when students engage in science reasoning in general and with regard to the particular scientific domain being studied. Such a determination could be applied to improve science education for both majors and nonmajors through pedagogical approaches that foster critical-thinking skills that are most relevant to science reasoning.

Writing affords one of the most effective means for making thinking visible ( Reynolds et al. , 2012 ) and learning how to “think like” and “write like” disciplinary experts ( Meizlish et al. , 2013 ). As a result, student writing affords the opportunities to both foster and examine the interrelationship of scientific reasoning and critical-thinking skills within and across disciplinary contexts. The purpose of this study was to better understand the relationship between students’ critical-thinking skills and scientific reasoning skills as reflected in the genre of undergraduate thesis writing in biology departments at two research universities, the University of Minnesota and Duke University.

In the following subsections, we discuss in greater detail the constructs of scientific reasoning and critical thinking, as well as the assessment of scientific reasoning in students’ thesis writing. In subsequent sections, we discuss our study design, findings, and the implications for enhancing educational practices.

Critical Thinking

The advances in cognitive science in the 21st century have increased our understanding of the mental processes involved in thinking and reasoning, as well as memory, learning, and problem solving. Critical thinking is understood to include both a cognitive dimension and a disposition dimension (e.g., reflective thinking) and is defined as “purposeful, self-regulatory judgment which results in interpretation, analysis, evaluation, and inference, as well as explanation of the evidential, conceptual, methodological, criteriological, or contextual considera­tions upon which that judgment is based” ( Facione, 1990, p. 3 ). Although various other definitions of critical thinking have been proposed, researchers have generally coalesced on this consensus: expert view ( Blattner and Frazier, 2002 ; Condon and Kelly-Riley, 2004 ; Bissell and Lemons, 2006 ; Quitadamo and Kurtz, 2007 ) and the corresponding measures of critical-­thinking skills ( August, 2016 ; Stephenson and Sadler-McKnight, 2016 ).

Both the cognitive skills and dispositional components of critical thinking have been recognized as important to science education ( Quitadamo and Kurtz, 2007 ). Empirical research demonstrates that specific pedagogical practices in science courses are effective in fostering students’ critical-thinking skills. Quitadamo and Kurtz (2007) found that students who engaged in a laboratory writing component in the context of a general education biology course significantly improved their overall critical-thinking skills (and their analytical and inference skills, in particular), whereas students engaged in a traditional quiz-based laboratory did not improve their critical-thinking skills. In related work, Quitadamo et al. (2008) found that a community-based inquiry experience, involving inquiry, writing, research, and analysis, was associated with improved critical thinking in a biology course for nonmajors, compared with traditionally taught sections. In both studies, students who exhibited stronger presemester critical-thinking skills exhibited stronger gains, suggesting that “students who have not been explicitly taught how to think critically may not reach the same potential as peers who have been taught these skills” ( Quitadamo and Kurtz, 2007 , p. 151).

Recently, Stephenson and Sadler-McKnight (2016) found that first-year general chemistry students who engaged in a science writing heuristic laboratory, which is an inquiry-based, writing-to-learn approach to instruction ( Hand and Keys, 1999 ), had significantly greater gains in total critical-thinking scores than students who received traditional laboratory instruction. Each of the four components—inquiry, writing, collaboration, and reflection—have been linked to critical thinking ( Stephenson and Sadler-McKnight, 2016 ). Like the other studies, this work highlights the value of targeting critical-thinking skills and the effectiveness of an inquiry-based, writing-to-learn approach to enhance critical thinking. Across studies, authors advocate adopting critical thinking as the course framework ( Pukkila, 2004 ) and developing explicit examples of how critical thinking relates to the scientific method ( Miri et al. , 2007 ).

In these examples, the important connection between writing and critical thinking is highlighted by the fact that each intervention involves the incorporation of writing into science, technology, engineering, and mathematics education (either alone or in combination with other pedagogical practices). However, critical-thinking skills are not always the primary learning outcome; in some contexts, scientific reasoning is the primary outcome that is assessed.

Scientific Reasoning

Scientific reasoning is a complex process that is broadly defined as “the skills involved in inquiry, experimentation, evidence evaluation, and inference that are done in the service of conceptual change or scientific understanding” ( Zimmerman, 2007 , p. 172). Scientific reasoning is understood to include both conceptual knowledge and the cognitive processes involved with generation of hypotheses (i.e., inductive processes involved in the generation of hypotheses and the deductive processes used in the testing of hypotheses), experimentation strategies, and evidence evaluation strategies. These dimensions are interrelated, in that “experimentation and inference strategies are selected based on prior conceptual knowledge of the domain” ( Zimmerman, 2000 , p. 139). Furthermore, conceptual and procedural knowledge and cognitive process dimensions can be general and domain specific (or discipline specific).

With regard to conceptual knowledge, attention has been focused on the acquisition of core methodological concepts fundamental to scientists’ causal reasoning and metacognitive distancing (or decontextualized thinking), which is the ability to reason independently of prior knowledge or beliefs ( Greenhoot et al. , 2004 ). The latter involves what Kuhn and Dean (2004) refer to as the coordination of theory and evidence, which requires that one question existing theories (i.e., prior knowledge and beliefs), seek contradictory evidence, eliminate alternative explanations, and revise one’s prior beliefs in the face of contradictory evidence. Kuhn and colleagues (2008) further elaborate that scientific thinking requires “a mature understanding of the epistemological foundations of science, recognizing scientific knowledge as constructed by humans rather than simply discovered in the world,” and “the ability to engage in skilled argumentation in the scientific domain, with an appreciation of argumentation as entailing the coordination of theory and evidence” ( Kuhn et al. , 2008 , p. 435). “This approach to scientific reasoning not only highlights the skills of generating and evaluating evidence-based inferences, but also encompasses epistemological appreciation of the functions of evidence and theory” ( Ding et al. , 2016 , p. 616). Evaluating evidence-based inferences involves epistemic cognition, which Moshman (2015) defines as the subset of metacognition that is concerned with justification, truth, and associated forms of reasoning. Epistemic cognition is both general and domain specific (or discipline specific; Moshman, 2015 ).

There is empirical support for the contributions of both prior knowledge and an understanding of the epistemological foundations of science to scientific reasoning. In a study of undergraduate science students, advanced scientific reasoning was most often accompanied by accurate prior knowledge as well as sophisticated epistemological commitments; additionally, for students who had comparable levels of prior knowledge, skillful reasoning was associated with a strong epistemological commitment to the consistency of theory with evidence ( Zeineddin and Abd-El-Khalick, 2010 ). These findings highlight the importance of the need for instructional activities that intentionally help learners develop sophisticated epistemological commitments focused on the nature of knowledge and the role of evidence in supporting knowledge claims ( Zeineddin and Abd-El-Khalick, 2010 ).

Scientific Reasoning in Students’ Thesis Writing

Pedagogical approaches that incorporate writing have also focused on enhancing scientific reasoning. Many rubrics have been developed to assess aspects of scientific reasoning in written artifacts. For example, Timmerman and colleagues (2011) , in the course of describing their own rubric for assessing scientific reasoning, highlight several examples of scientific reasoning assessment criteria ( Haaga, 1993 ; Tariq et al. , 1998 ; Topping et al. , 2000 ; Kelly and Takao, 2002 ; Halonen et al. , 2003 ; Willison and O’Regan, 2007 ).

At both the University of Minnesota and Duke University, we have focused on the genre of the undergraduate honors thesis as the rhetorical context in which to study and improve students’ scientific reasoning and writing. We view the process of writing an undergraduate honors thesis as a form of professional development in the sciences (i.e., a way of engaging students in the practices of a community of discourse). We have found that structured courses designed to scaffold the thesis-­writing process and promote metacognition can improve writing and reasoning skills in biology, chemistry, and economics ( Reynolds and Thompson, 2011 ; Dowd et al. , 2015a , b ). In the context of this prior work, we have defined scientific reasoning in writing as the emergent, underlying construct measured across distinct aspects of students’ written discussion of independent research in their undergraduate theses.

The Biology Thesis Assessment Protocol (BioTAP) was developed at Duke University as a tool for systematically guiding students and faculty through a “draft–feedback–revision” writing process, modeled after professional scientific peer-review processes ( Reynolds et al. , 2009 ). BioTAP includes activities and worksheets that allow students to engage in critical peer review and provides detailed descriptions, presented as rubrics, of the questions (i.e., dimensions, shown in Table 1 ) upon which such review should focus. Nine rubric dimensions focus on communication to the broader scientific community, and four rubric dimensions focus on the accuracy and appropriateness of the research. These rubric dimensions provide criteria by which the thesis is assessed, and therefore allow BioTAP to be used as an assessment tool as well as a teaching resource ( Reynolds et al. , 2009 ). Full details are available at www.science-writing.org/biotap.html .

Theses assessment protocol dimensions

In previous work, we have used BioTAP to quantitatively assess students’ undergraduate honors theses and explore the relationship between thesis-writing courses (or specific interventions within the courses) and the strength of students’ science reasoning in writing across different science disciplines: biology ( Reynolds and Thompson, 2011 ); chemistry ( Dowd et al. , 2015b ); and economics ( Dowd et al. , 2015a ). We have focused exclusively on the nine dimensions related to reasoning and writing (questions 1–9), as the other four dimensions (questions 10–13) require topic-specific expertise and are intended to be used by the student’s thesis supervisor.

Beyond considering individual dimensions, we have investigated whether meaningful constructs underlie students’ thesis scores. We conducted exploratory factor analysis of students’ theses in biology, economics, and chemistry and found one dominant underlying factor in each discipline; we termed the factor “scientific reasoning in writing” ( Dowd et al. , 2015a , b , 2016 ). That is, each of the nine dimensions could be understood as reflecting, in different ways and to different degrees, the construct of scientific reasoning in writing. The findings indicated evidence of both general and discipline-specific components to scientific reasoning in writing that relate to epistemic beliefs and paradigms, in keeping with broader ideas about science reasoning discussed earlier. Specifically, scientific reasoning in writing is more strongly associated with formulating a compelling argument for the significance of the research in the context of current literature in biology, making meaning regarding the implications of the findings in chemistry, and providing an organizational framework for interpreting the thesis in economics. We suggested that instruction, whether occurring in writing studios or in writing courses to facilitate thesis preparation, should attend to both components.

Research Question and Study Design

The genre of thesis writing combines the pedagogies of writing and inquiry found to foster scientific reasoning ( Reynolds et al. , 2012 ) and critical thinking ( Quitadamo and Kurtz, 2007 ; Quitadamo et al. , 2008 ; Stephenson and Sadler-­McKnight, 2016 ). However, there is no empirical evidence regarding the general or domain-specific interrelationships of scientific reasoning and critical-thinking skills, particularly in the rhetorical context of the undergraduate thesis. The BioTAP studies discussed earlier indicate that the rubric-based assessment produces evidence of scientific reasoning in the undergraduate thesis, but it was not designed to foster or measure critical thinking. The current study was undertaken to address the research question: How are students’ critical-thinking skills related to scientific reasoning as reflected in the genre of undergraduate thesis writing in biology? Determining these interrelationships could guide efforts to enhance students’ scientific reasoning and writing skills through focusing instruction on specific critical-thinking skills as well as disciplinary conventions.

To address this research question, we focused on undergraduate thesis writers in biology courses at two institutions, Duke University and the University of Minnesota, and examined the extent to which students’ scientific reasoning in writing, assessed in the undergraduate thesis using BioTAP, corresponds to students’ critical-thinking skills, assessed using the California Critical Thinking Skills Test (CCTST; August, 2016 ).

Study Sample

The study sample was composed of students enrolled in courses designed to scaffold the thesis-writing process in the Department of Biology at Duke University and the College of Biological Sciences at the University of Minnesota. Both courses complement students’ individual work with research advisors. The course is required for thesis writers at the University of Minnesota and optional for writers at Duke University. Not all students are required to complete a thesis, though it is required for students to graduate with honors; at the University of Minnesota, such students are enrolled in an honors program within the college. In total, 28 students were enrolled in the course at Duke University and 44 students were enrolled in the course at the University of Minnesota. Of those students, two students did not consent to participate in the study; additionally, five students did not validly complete the CCTST (i.e., attempted fewer than 60% of items or completed the test in less than 15 minutes). Thus, our overall rate of valid participation is 90%, with 27 students from Duke University and 38 students from the University of Minnesota. We found no statistically significant differences in thesis assessment between students with valid CCTST scores and invalid CCTST scores. Therefore, we focus on the 65 students who consented to participate and for whom we have complete and valid data in most of this study. Additionally, in asking students for their consent to participate, we allowed them to choose whether to provide or decline access to academic and demographic background data. Of the 65 students who consented to participate, 52 students granted access to such data. Therefore, for additional analyses involving academic and background data, we focus on the 52 students who consented. We note that the 13 students who participated but declined to share additional data performed slightly lower on the CCTST than the 52 others (perhaps suggesting that they differ by other measures, but we cannot determine this with certainty). Among the 52 students, 60% identified as female and 10% identified as being from underrepresented ethnicities.

In both courses, students completed the CCTST online, either in class or on their own, late in the Spring 2016 semester. This is the same assessment that was used in prior studies of critical thinking ( Quitadamo and Kurtz, 2007 ; Quitadamo et al. , 2008 ; Stephenson and Sadler-McKnight, 2016 ). It is “an objective measure of the core reasoning skills needed for reflective decision making concerning what to believe or what to do” ( Insight Assessment, 2016a ). In the test, students are asked to read and consider information as they answer multiple-choice questions. The questions are intended to be appropriate for all users, so there is no expectation of prior disciplinary knowledge in biology (or any other subject). Although actual test items are protected, sample items are available on the Insight Assessment website ( Insight Assessment, 2016b ). We have included one sample item in the Supplemental Material.

The CCTST is based on a consensus definition of critical thinking, measures cognitive and metacognitive skills associated with critical thinking, and has been evaluated for validity and reliability at the college level ( August, 2016 ; Stephenson and Sadler-McKnight, 2016 ). In addition to providing overall critical-thinking score, the CCTST assesses seven dimensions of critical thinking: analysis, interpretation, inference, evaluation, explanation, induction, and deduction. Scores on each dimension are calculated based on students’ performance on items related to that dimension. Analysis focuses on identifying assumptions, reasons, and claims and examining how they interact to form arguments. Interpretation, related to analysis, focuses on determining the precise meaning and significance of information. Inference focuses on drawing conclusions from reasons and evidence. Evaluation focuses on assessing the credibility of sources of information and claims they make. Explanation, related to evaluation, focuses on describing the evidence, assumptions, or rationale for beliefs and conclusions. Induction focuses on drawing inferences about what is probably true based on evidence. Deduction focuses on drawing conclusions about what must be true when the context completely determines the outcome. These are not independent dimensions; the fact that they are related supports their collective interpretation as critical thinking. Together, the CCTST dimensions provide a basis for evaluating students’ overall strength in using reasoning to form reflective judgments about what to believe or what to do ( August, 2016 ). Each of the seven dimensions and the overall CCTST score are measured on a scale of 0–100, where higher scores indicate superior performance. Scores correspond to superior (86–100), strong (79–85), moderate (70–78), weak (63–69), or not manifested (62 and below) skills.

Scientific Reasoning in Writing

At the end of the semester, students’ final, submitted undergraduate theses were assessed using BioTAP, which consists of nine rubric dimensions that focus on communication to the broader scientific community and four additional dimensions that focus on the exhibition of topic-specific expertise ( Reynolds et al. , 2009 ). These dimensions, framed as questions, are displayed in Table 1 .

Student theses were assessed on questions 1–9 of BioTAP using the same procedures described in previous studies ( Reynolds and Thompson, 2011 ; Dowd et al. , 2015a , b ). In this study, six raters were trained in the valid, reliable use of BioTAP rubrics. Each dimension was rated on a five-point scale: 1 indicates the dimension is missing, incomplete, or below acceptable standards; 3 indicates that the dimension is adequate but not exhibiting mastery; and 5 indicates that the dimension is excellent and exhibits mastery (intermediate ratings of 2 and 4 are appropriate when different parts of the thesis make a single category challenging). After training, two raters independently assessed each thesis and then discussed their independent ratings with one another to form a consensus rating. The consensus score is not an average score, but rather an agreed-upon, discussion-based score. On a five-point scale, raters independently assessed dimensions to be within 1 point of each other 82.4% of the time before discussion and formed consensus ratings 100% of the time after discussion.

In this study, we consider both categorical (mastery/nonmastery, where a score of 5 corresponds to mastery) and numerical treatments of individual BioTAP scores to better relate the manifestation of critical thinking in BioTAP assessment to all of the prior studies. For comprehensive/cumulative measures of BioTAP, we focus on the partial sum of questions 1–5, as these questions relate to higher-order scientific reasoning (whereas questions 6–9 relate to mid- and lower-order writing mechanics [ Reynolds et al. , 2009 ]), and the factor scores (i.e., numerical representations of the extent to which each student exhibits the underlying factor), which are calculated from the factor loadings published by Dowd et al. (2016) . We do not focus on questions 6–9 individually in statistical analyses, because we do not expect critical-thinking skills to relate to mid- and lower-order writing skills.

The final, submitted thesis reflects the student’s writing, the student’s scientific reasoning, the quality of feedback provided to the student by peers and mentors, and the student’s ability to incorporate that feedback into his or her work. Therefore, our assessment is not the same as an assessment of unpolished, unrevised samples of students’ written work. While one might imagine that such an unpolished sample may be more strongly correlated with critical-thinking skills measured by the CCTST, we argue that the complete, submitted thesis, assessed using BioTAP, is ultimately a more appropriate reflection of how students exhibit science reasoning in the scientific community.

Statistical Analyses

We took several steps to analyze the collected data. First, to provide context for subsequent interpretations, we generated descriptive statistics for the CCTST scores of the participants based on the norms for undergraduate CCTST test takers. To determine the strength of relationships among CCTST dimensions (including overall score) and the BioTAP dimensions, partial-sum score (questions 1–5), and factor score, we calculated Pearson’s correlations for each pair of measures. To examine whether falling on one side of the nonmastery/mastery threshold (as opposed to a linear scale of performance) was related to critical thinking, we grouped BioTAP dimensions into categories (mastery/nonmastery) and conducted Student’s t tests to compare the means scores of the two groups on each of the seven dimensions and overall score of the CCTST. Finally, for the strongest relationship that emerged, we included additional academic and background variables as covariates in multiple linear-regression analysis to explore questions about how much observed relationships between critical-thinking skills and science reasoning in writing might be explained by variation in these other factors.

Although BioTAP scores represent discreet, ordinal bins, the five-point scale is intended to capture an underlying continuous construct (from inadequate to exhibiting mastery). It has been argued that five categories is an appropriate cutoff for treating ordinal variables as pseudo-continuous ( Rhemtulla et al. , 2012 )—and therefore using continuous-variable statistical methods (e.g., Pearson’s correlations)—as long as the underlying assumption that ordinal scores are linearly distributed is valid. Although we have no way to statistically test this assumption, we interpret adequate scores to be approximately halfway between inadequate and mastery scores, resulting in a linear scale. In part because this assumption is subject to disagreement, we also consider and interpret a categorical (mastery/nonmastery) treatment of BioTAP variables.

We corrected for multiple comparisons using the Holm-Bonferroni method ( Holm, 1979 ). At the most general level, where we consider the single, comprehensive measures for BioTAP (partial-sum and factor score) and the CCTST (overall score), there is no need to correct for multiple comparisons, because the multiple, individual dimensions are collapsed into single dimensions. When we considered individual CCTST dimensions in relation to comprehensive measures for BioTAP, we accounted for seven comparisons; similarly, when we considered individual dimensions of BioTAP in relation to overall CCTST score, we accounted for five comparisons. When all seven CCTST and five BioTAP dimensions were examined individually and without prior knowledge, we accounted for 35 comparisons; such a rigorous threshold is likely to reject weak and moderate relationships, but it is appropriate if there are no specific pre-existing hypotheses. All p values are presented in tables for complete transparency, and we carefully consider the implications of our interpretation of these data in the Discussion section.

CCTST scores for students in this sample ranged from the 39th to 99th percentile of the general population of undergraduate CCTST test takers (mean percentile = 84.3, median = 85th percentile; Table 2 ); these percentiles reflect overall scores that range from moderate to superior. Scores on individual dimensions and overall scores were sufficiently normal and far enough from the ceiling of the scale to justify subsequent statistical analyses.

Descriptive statistics of CCTST dimensions a

a Scores correspond to superior (86–100), strong (79–85), moderate (70–78), weak (63–69), or not manifested (62 and lower) skills.

The Pearson’s correlations between students’ cumulative scores on BioTAP (the factor score based on loadings published by Dowd et al. , 2016 , and the partial sum of scores on questions 1–5) and students’ overall scores on the CCTST are presented in Table 3 . We found that the partial-sum measure of BioTAP was significantly related to the overall measure of critical thinking ( r = 0.27, p = 0.03), while the BioTAP factor score was marginally related to overall CCTST ( r = 0.24, p = 0.05). When we looked at relationships between comprehensive BioTAP measures and scores for individual dimensions of the CCTST ( Table 3 ), we found significant positive correlations between the both BioTAP partial-sum and factor scores and CCTST inference ( r = 0.45, p < 0.001, and r = 0.41, p < 0.001, respectively). Although some other relationships have p values below 0.05 (e.g., the correlations between BioTAP partial-sum scores and CCTST induction and interpretation scores), they are not significant when we correct for multiple comparisons.

Correlations between dimensions of CCTST and dimensions of BioTAP a

a In each cell, the top number is the correlation, and the bottom, italicized number is the associated p value. Correlations that are statistically significant after correcting for multiple comparisons are shown in bold.

b This is the partial sum of BioTAP scores on questions 1–5.

c This is the factor score calculated from factor loadings published by Dowd et al. (2016) .

When we expanded comparisons to include all 35 potential correlations among individual BioTAP and CCTST dimensions—and, accordingly, corrected for 35 comparisons—we did not find any additional statistically significant relationships. The Pearson’s correlations between students’ scores on each dimension of BioTAP and students’ scores on each dimension of the CCTST range from −0.11 to 0.35 ( Table 3 ); although the relationship between discussion of implications (BioTAP question 5) and inference appears to be relatively large ( r = 0.35), it is not significant ( p = 0.005; the Holm-Bonferroni cutoff is 0.00143). We found no statistically significant relationships between BioTAP questions 6–9 and CCTST dimensions (unpublished data), regardless of whether we correct for multiple comparisons.

The results of Student’s t tests comparing scores on each dimension of the CCTST of students who exhibit mastery with those of students who do not exhibit mastery on each dimension of BioTAP are presented in Table 4 . Focusing first on the overall CCTST scores, we found that the difference between those who exhibit mastery and those who do not in discussing implications of results (BioTAP question 5) is statistically significant ( t = 2.73, p = 0.008, d = 0.71). When we expanded t tests to include all 35 comparisons—and, like above, corrected for 35 comparisons—we found a significant difference in inference scores between students who exhibit mastery on question 5 and students who do not ( t = 3.41, p = 0.0012, d = 0.88), as well as a marginally significant difference in these students’ induction scores ( t = 3.26, p = 0.0018, d = 0.84; the Holm-Bonferroni cutoff is p = 0.00147). Cohen’s d effect sizes, which reveal the strength of the differences for statistically significant relationships, range from 0.71 to 0.88.

The t statistics and effect sizes of differences in ­dimensions of CCTST across dimensions of BioTAP a

a In each cell, the top number is the t statistic for each comparison, and the middle, italicized number is the associated p value. The bottom number is the effect size. Correlations that are statistically significant after correcting for multiple comparisons are shown in bold.

Finally, we more closely examined the strongest relationship that we observed, which was between the CCTST dimension of inference and the BioTAP partial-sum composite score (shown in Table 3 ), using multiple regression analysis ( Table 5 ). Focusing on the 52 students for whom we have background information, we looked at the simple relationship between BioTAP and inference (model 1), a robust background model including multiple covariates that one might expect to explain some part of the variation in BioTAP (model 2), and a combined model including all variables (model 3). As model 3 shows, the covariates explain very little variation in BioTAP scores, and the relationship between inference and BioTAP persists even in the presence of all of the covariates.

Partial sum (questions 1–5) of BioTAP scores ( n = 52)

** p < 0.01.

*** p < 0.001.

The aim of this study was to examine the extent to which the various components of scientific reasoning—manifested in writing in the genre of undergraduate thesis and assessed using BioTAP—draw on general and specific critical-thinking skills (assessed using CCTST) and to consider the implications for educational practices. Although science reasoning involves critical-thinking skills, it also relates to conceptual knowledge and the epistemological foundations of science disciplines ( Kuhn et al. , 2008 ). Moreover, science reasoning in writing , captured in students’ undergraduate theses, reflects habits, conventions, and the incorporation of feedback that may alter evidence of individuals’ critical-thinking skills. Our findings, however, provide empirical evidence that cumulative measures of science reasoning in writing are nonetheless related to students’ overall critical-thinking skills ( Table 3 ). The particularly significant roles of inference skills ( Table 3 ) and the discussion of implications of results (BioTAP question 5; Table 4 ) provide a basis for more specific ideas about how these constructs relate to one another and what educational interventions may have the most success in fostering these skills.

Our results build on previous findings. The genre of thesis writing combines pedagogies of writing and inquiry found to foster scientific reasoning ( Reynolds et al. , 2012 ) and critical thinking ( Quitadamo and Kurtz, 2007 ; Quitadamo et al. , 2008 ; Stephenson and Sadler-McKnight, 2016 ). Quitadamo and Kurtz (2007) reported that students who engaged in a laboratory writing component in a general education biology course significantly improved their inference and analysis skills, and Quitadamo and colleagues (2008) found that participation in a community-based inquiry biology course (that included a writing component) was associated with significant gains in students’ inference and evaluation skills. The shared focus on inference is noteworthy, because these prior studies actually differ from the current study; the former considered critical-­thinking skills as the primary learning outcome of writing-­focused interventions, whereas the latter focused on emergent links between two learning outcomes (science reasoning in writing and critical thinking). In other words, inference skills are impacted by writing as well as manifested in writing.

Inference focuses on drawing conclusions from argument and evidence. According to the consensus definition of critical thinking, the specific skill of inference includes several processes: querying evidence, conjecturing alternatives, and drawing conclusions. All of these activities are central to the independent research at the core of writing an undergraduate thesis. Indeed, a critical part of what we call “science reasoning in writing” might be characterized as a measure of students’ ability to infer and make meaning of information and findings. Because the cumulative BioTAP measures distill underlying similarities and, to an extent, suppress unique aspects of individual dimensions, we argue that it is appropriate to relate inference to scientific reasoning in writing . Even when we control for other potentially relevant background characteristics, the relationship is strong ( Table 5 ).

In taking the complementary view and focusing on BioTAP, when we compared students who exhibit mastery with those who do not, we found that the specific dimension of “discussing the implications of results” (question 5) differentiates students’ performance on several critical-thinking skills. To achieve mastery on this dimension, students must make connections between their results and other published studies and discuss the future directions of the research; in short, they must demonstrate an understanding of the bigger picture. The specific relationship between question 5 and inference is the strongest observed among all individual comparisons. Altogether, perhaps more than any other BioTAP dimension, this aspect of students’ writing provides a clear view of the role of students’ critical-thinking skills (particularly inference and, marginally, induction) in science reasoning.

While inference and discussion of implications emerge as particularly strongly related dimensions in this work, we note that the strongest contribution to “science reasoning in writing in biology,” as determined through exploratory factor analysis, is “argument for the significance of research” (BioTAP question 2, not question 5; Dowd et al. , 2016 ). Question 2 is not clearly related to critical-thinking skills. These findings are not contradictory, but rather suggest that the epistemological and disciplinary-specific aspects of science reasoning that emerge in writing through BioTAP are not completely aligned with aspects related to critical thinking. In other words, science reasoning in writing is not simply a proxy for those critical-thinking skills that play a role in science reasoning.

In a similar vein, the content-related, epistemological aspects of science reasoning, as well as the conventions associated with writing the undergraduate thesis (including feedback from peers and revision), may explain the lack of significant relationships between some science reasoning dimensions and some critical-thinking skills that might otherwise seem counterintuitive (e.g., BioTAP question 2, which relates to making an argument, and the critical-thinking skill of argument). It is possible that an individual’s critical-thinking skills may explain some variation in a particular BioTAP dimension, but other aspects of science reasoning and practice exert much stronger influence. Although these relationships do not emerge in our analyses, the lack of significant correlation does not mean that there is definitively no correlation. Correcting for multiple comparisons suppresses type 1 error at the expense of exacerbating type 2 error, which, combined with the limited sample size, constrains statistical power and makes weak relationships more difficult to detect. Ultimately, though, the relationships that do emerge highlight places where individuals’ distinct critical-thinking skills emerge most coherently in thesis assessment, which is why we are particularly interested in unpacking those relationships.

We recognize that, because only honors students submit theses at these institutions, this study sample is composed of a selective subset of the larger population of biology majors. Although this is an inherent limitation of focusing on thesis writing, links between our findings and results of other studies (with different populations) suggest that observed relationships may occur more broadly. The goal of improved science reasoning and critical thinking is shared among all biology majors, particularly those engaged in capstone research experiences. So while the implications of this work most directly apply to honors thesis writers, we provisionally suggest that all students could benefit from further study of them.

There are several important implications of this study for science education practices. Students’ inference skills relate to the understanding and effective application of scientific content. The fact that we find no statistically significant relationships between BioTAP questions 6–9 and CCTST dimensions suggests that such mid- to lower-order elements of BioTAP ( Reynolds et al. , 2009 ), which tend to be more structural in nature, do not focus on aspects of the finished thesis that draw strongly on critical thinking. In keeping with prior analyses ( Reynolds and Thompson, 2011 ; Dowd et al. , 2016 ), these findings further reinforce the notion that disciplinary instructors, who are most capable of teaching and assessing scientific reasoning and perhaps least interested in the more mechanical aspects of writing, may nonetheless be best suited to effectively model and assess students’ writing.

The goal of the thesis writing course at both Duke University and the University of Minnesota is not merely to improve thesis scores but to move students’ writing into the category of mastery across BioTAP dimensions. Recognizing that students with differing critical-thinking skills (particularly inference) are more or less likely to achieve mastery in the undergraduate thesis (particularly in discussing implications [question 5]) is important for developing and testing targeted pedagogical interventions to improve learning outcomes for all students.

The competencies characterized by the Vision and Change in Undergraduate Biology Education Initiative provide a general framework for recognizing that science reasoning and critical-thinking skills play key roles in major learning outcomes of science education. Our findings highlight places where science reasoning–related competencies (like “understanding the process of science”) connect to critical-thinking skills and places where critical thinking–related competencies might be manifested in scientific products (such as the ability to discuss implications in scientific writing). We encourage broader efforts to build empirical connections between competencies and pedagogical practices to further improve science education.

One specific implication of this work for science education is to focus on providing opportunities for students to develop their critical-thinking skills (particularly inference). Of course, as this correlational study is not designed to test causality, we do not claim that enhancing students’ inference skills will improve science reasoning in writing. However, as prior work shows that science writing activities influence students’ inference skills ( Quitadamo and Kurtz, 2007 ; Quitadamo et al. , 2008 ), there is reason to test such a hypothesis. Nevertheless, the focus must extend beyond inference as an isolated skill; rather, it is important to relate inference to the foundations of the scientific method ( Miri et al. , 2007 ) in terms of the epistemological appreciation of the functions and coordination of evidence ( Kuhn and Dean, 2004 ; Zeineddin and Abd-El-Khalick, 2010 ; Ding et al. , 2016 ) and disciplinary paradigms of truth and justification ( Moshman, 2015 ).

Although this study is limited to the domain of biology at two institutions with a relatively small number of students, the findings represent a foundational step in the direction of achieving success with more integrated learning outcomes. Hopefully, it will spur greater interest in empirically grounding discussions of the constructs of scientific reasoning and critical-thinking skills.

This study contributes to the efforts to improve science education, for both majors and nonmajors, through an empirically driven analysis of the relationships between scientific reasoning reflected in the genre of thesis writing and critical-thinking skills. This work is rooted in the usefulness of BioTAP as a method 1) to facilitate communication and learning and 2) to assess disciplinary-specific and general dimensions of science reasoning. The findings support the important role of the critical-thinking skill of inference in scientific reasoning in writing, while also highlighting ways in which other aspects of science reasoning (epistemological considerations, writing conventions, etc.) are not significantly related to critical thinking. Future research into the impact of interventions focused on specific critical-thinking skills (i.e., inference) for improved science reasoning in writing will build on this work and its implications for science education.

Supplementary Material

Acknowledgments.

We acknowledge the contributions of Kelaine Haas and Alexander Motten to the implementation and collection of data. We also thank Mine Çetinkaya-­Rundel for her insights regarding our statistical analyses. This research was funded by National Science Foundation award DUE-1525602.

• American Association for the Advancement of Science. (2011). Vision and change in undergraduate biology education: A call to action . Washington, DC: Retrieved September 26, 2017, from https://visionandchange.org/files/2013/11/aaas-VISchange-web1113.pdf . [ Google Scholar ]
• August D. (2016). California Critical Thinking Skills Test user manual and resource guide . San Jose: Insight Assessment/California Academic Press. [ Google Scholar ]
• Beyer C. H., Taylor E., Gillmore G. M. (2013). Inside the undergraduate teaching experience: The University of Washington’s growth in faculty teaching study . Albany, NY: SUNY Press. [ Google Scholar ]
• Bissell A. N., Lemons P. P. (2006). A new method for assessing critical thinking in the classroom . BioScience , ( 1 ), 66–72. https://doi.org/10.1641/0006-3568(2006)056[0066:ANMFAC]2.0.CO;2 . [ Google Scholar ]
• Blattner N. H., Frazier C. L. (2002). Developing a performance-based assessment of students’ critical thinking skills . Assessing Writing , ( 1 ), 47–64. [ Google Scholar ]
• Clase K. L., Gundlach E., Pelaez N. J. (2010). Calibrated peer review for computer-assisted learning of biological research competencies . Biochemistry and Molecular Biology Education , ( 5 ), 290–295. [ PubMed ] [ Google Scholar ]
• Condon W., Kelly-Riley D. (2004). Assessing and teaching what we value: The relationship between college-level writing and critical thinking abilities . Assessing Writing , ( 1 ), 56–75. https://doi.org/10.1016/j.asw.2004.01.003 . [ Google Scholar ]
• Ding L., Wei X., Liu X. (2016). Variations in university students’ scientific reasoning skills across majors, years, and types of institutions . Research in Science Education , ( 5 ), 613–632. https://doi.org/10.1007/s11165-015-9473-y . [ Google Scholar ]
• Dowd J. E., Connolly M. P., Thompson R. J., Jr., Reynolds J. A. (2015a). Improved reasoning in undergraduate writing through structured workshops . Journal of Economic Education , ( 1 ), 14–27. https://doi.org/10.1080/00220485.2014.978924 . [ Google Scholar ]
• Dowd J. E., Roy C. P., Thompson R. J., Jr., Reynolds J. A. (2015b). “On course” for supporting expanded participation and improving scientific reasoning in undergraduate thesis writing . Journal of Chemical Education , ( 1 ), 39–45. https://doi.org/10.1021/ed500298r . [ Google Scholar ]
• Dowd J. E., Thompson R. J., Jr., Reynolds J. A. (2016). Quantitative genre analysis of undergraduate theses: Uncovering different ways of writing and thinking in science disciplines . WAC Journal , , 36–51. [ Google Scholar ]
• Facione P. A. (1990). Critical thinking: a statement of expert consensus for purposes of educational assessment and instruction. Research findings and recommendations . Newark, DE: American Philosophical Association; Retrieved September 26, 2017, from https://philpapers.org/archive/FACCTA.pdf . [ Google Scholar ]
• Gerdeman R. D., Russell A. A., Worden K. J., Gerdeman R. D., Russell A. A., Worden K. J. (2007). Web-based student writing and reviewing in a large biology lecture course . Journal of College Science Teaching , ( 5 ), 46–52. [ Google Scholar ]
• Greenhoot A. F., Semb G., Colombo J., Schreiber T. (2004). Prior beliefs and methodological concepts in scientific reasoning . Applied Cognitive Psychology , ( 2 ), 203–221. https://doi.org/10.1002/acp.959 . [ Google Scholar ]
• Haaga D. A. F. (1993). Peer review of term papers in graduate psychology courses . Teaching of Psychology , ( 1 ), 28–32. https://doi.org/10.1207/s15328023top2001_5 . [ Google Scholar ]
• Halonen J. S., Bosack T., Clay S., McCarthy M., Dunn D. S., Hill G. W., Whitlock K. (2003). A rubric for learning, teaching, and assessing scientific inquiry in psychology . Teaching of Psychology , ( 3 ), 196–208. https://doi.org/10.1207/S15328023TOP3003_01 . [ Google Scholar ]
• Hand B., Keys C. W. (1999). Inquiry investigation . Science Teacher , ( 4 ), 27–29. [ Google Scholar ]
• Holm S. (1979). A simple sequentially rejective multiple test procedure . Scandinavian Journal of Statistics , ( 2 ), 65–70. [ Google Scholar ]
• Holyoak K. J., Morrison R. G. (2005). The Cambridge handbook of thinking and reasoning . New York: Cambridge University Press. [ Google Scholar ]
• Insight Assessment. (2016a). California Critical Thinking Skills Test (CCTST) Retrieved September 26, 2017, from www.insightassessment.com/Products/Products-Summary/Critical-Thinking-Skills-Tests/California-Critical-Thinking-Skills-Test-CCTST .
• Insight Assessment. (2016b). Sample thinking skills questions. Retrieved September 26, 2017, from www.insightassessment.com/Resources/Teaching-Training-and-Learning-Tools/node_1487 .
• Kelly G. J., Takao A. (2002). Epistemic levels in argument: An analysis of university oceanography students’ use of evidence in writing . Science Education , ( 3 ), 314–342. https://doi.org/10.1002/sce.10024 . [ Google Scholar ]
• Kuhn D., Dean D., Jr. (2004). Connecting scientific reasoning and causal inference . Journal of Cognition and Development , ( 2 ), 261–288. https://doi.org/10.1207/s15327647jcd0502_5 . [ Google Scholar ]
• Kuhn D., Iordanou K., Pease M., Wirkala C. (2008). Beyond control of variables: What needs to develop to achieve skilled scientific thinking? . Cognitive Development , ( 4 ), 435–451. https://doi.org/10.1016/j.cogdev.2008.09.006 . [ Google Scholar ]
• Lawson A. E. (2010). Basic inferences of scientific reasoning, argumentation, and discovery . Science Education , ( 2 ), 336–364. https://doi.org/­10.1002/sce.20357 . [ Google Scholar ]
• Meizlish D., LaVaque-Manty D., Silver N., Kaplan M. (2013). Think like/write like: Metacognitive strategies to foster students’ development as disciplinary thinkers and writers . In Thompson R. J. (Ed.), Changing the conversation about higher education (pp. 53–73). Lanham, MD: Rowman & Littlefield. [ Google Scholar ]
• Miri B., David B.-C., Uri Z. (2007). Purposely teaching for the promotion of higher-order thinking skills: A case of critical thinking . Research in Science Education , ( 4 ), 353–369. https://doi.org/10.1007/s11165-006-9029-2 . [ Google Scholar ]
• Moshman D. (2015). Epistemic cognition and development: The psychology of justification and truth . New York: Psychology Press. [ Google Scholar ]
• National Research Council. (2000). How people learn: Brain, mind, experience, and school . Expanded ed. Washington, DC: National Academies Press. [ Google Scholar ]
• Pukkila P. J. (2004). Introducing student inquiry in large introductory genetics classes . Genetics , ( 1 ), 11–18. https://doi.org/10.1534/genetics.166.1.11 . [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Quitadamo I. J., Faiola C. L., Johnson J. E., Kurtz M. J. (2008). Community-based inquiry improves critical thinking in general education biology . CBE—Life Sciences Education , ( 3 ), 327–337. https://doi.org/10.1187/cbe.07-11-0097 . [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Quitadamo I. J., Kurtz M. J. (2007). Learning to improve: Using writing to increase critical thinking performance in general education biology . CBE—Life Sciences Education , ( 2 ), 140–154. https://doi.org/10.1187/cbe.06-11-0203 . [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Reynolds J. A., Smith R., Moskovitz C., Sayle A. (2009). BioTAP: A systematic approach to teaching scientific writing and evaluating undergraduate theses . BioScience , ( 10 ), 896–903. https://doi.org/10.1525/bio.2009.59.10.11 . [ Google Scholar ]
• Reynolds J. A., Thaiss C., Katkin W., Thompson R. J. (2012). Writing-to-learn in undergraduate science education: A community-based, conceptually driven approach . CBE—Life Sciences Education , ( 1 ), 17–25. https://doi.org/10.1187/cbe.11-08-0064 . [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Reynolds J. A., Thompson R. J. (2011). Want to improve undergraduate thesis writing? Engage students and their faculty readers in scientific peer review . CBE—Life Sciences Education , ( 2 ), 209–215. https://doi.org/­10.1187/cbe.10-10-0127 . [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Rhemtulla M., Brosseau-Liard P. E., Savalei V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions . Psychological Methods , ( 3 ), 354–373. https://doi.org/­10.1037/a0029315 . [ PubMed ] [ Google Scholar ]
• Stephenson N. S., Sadler-McKnight N. P. (2016). Developing critical thinking skills using the science writing heuristic in the chemistry laboratory . Chemistry Education Research and Practice , ( 1 ), 72–79. https://doi.org/­10.1039/C5RP00102A . [ Google Scholar ]
• Tariq V. N., Stefani L. A. J., Butcher A. C., Heylings D. J. A. (1998). Developing a new approach to the assessment of project work . Assessment and Evaluation in Higher Education , ( 3 ), 221–240. https://doi.org/­10.1080/0260293980230301 . [ Google Scholar ]
• Timmerman B. E. C., Strickland D. C., Johnson R. L., Payne J. R. (2011). Development of a “universal” rubric for assessing undergraduates’ scientific reasoning skills using scientific writing . Assessment and Evaluation in Higher Education , ( 5 ), 509–547. https://doi.org/10.1080/­02602930903540991 . [ Google Scholar ]
• Topping K. J., Smith E. F., Swanson I., Elliot A. (2000). Formative peer assessment of academic writing between postgraduate students . Assessment and Evaluation in Higher Education , ( 2 ), 149–169. https://doi.org/10.1080/713611428 . [ Google Scholar ]
• Willison J., O’Regan K. (2007). Commonly known, commonly not known, totally unknown: A framework for students becoming researchers . Higher Education Research and Development , ( 4 ), 393–409. https://doi.org/10.1080/07294360701658609 . [ Google Scholar ]
• Woodin T., Carter V. C., Fletcher L. (2010). Vision and Change in Biology Undergraduate Education: A Call for Action—Initial responses . CBE—Life Sciences Education , ( 2 ), 71–73. https://doi.org/10.1187/cbe.10-03-0044 . [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Zeineddin A., Abd-El-Khalick F. (2010). Scientific reasoning and epistemological commitments: Coordination of theory and evidence among college science students . Journal of Research in Science Teaching , ( 9 ), 1064–1093. https://doi.org/10.1002/tea.20368 . [ Google Scholar ]
• Zimmerman C. (2000). The development of scientific reasoning skills . Developmental Review , ( 1 ), 99–149. https://doi.org/10.1006/drev.1999.0497 . [ Google Scholar ]
• Zimmerman C. (2007). The development of scientific thinking skills in elementary and middle school . Developmental Review , ( 2 ), 172–223. https://doi.org/10.1016/j.dr.2006.12.001 . [ Google Scholar ]

Critical Qualitative Research

Critical research serves to address societal structures and institutions that oppress and exclude so that transformative actions can be generated that reduce inequitable power conditions. We invite proposals for authored and edited volumes that describe critical social science research (re)conceptualizations, practices, and methodologies that can be used by other scholars who wish to design and implement critical qualitative inquiry. Critical Qualitative Research challenges modernist orientations toward research by using social theory, designs, and research practices that emerge from critical questions like: Who/what is heard? Who/what is silenced? Who is privileged? Who is disqualified? How are forms of inclusion/exclusion being created? How are relations of power constructed and managed? How do various forms of privilege and oppression intersect to impact life possibilities for various individuals and groups? How do the arts inform research? How can multiple knowledges be engaged in research? How can research be socially just?

Walking with Strangers

Mapping the Terrains of Student Voice Pedagogies

Whiteness Is the New South Africa

Politics, Pedagogy and Power

Re-Assembly Required

Representing Youth with Disability on Television

Conducting Hermeneutic Research

Place, Being, Resonance

Critically Researching Youth

Artistic Research Methodology

Key subject areas.

• English Studies
• German Studies
• History & Political Science
• Law, Economics & Management
• Linguistics
• Media & Communication
• Romance Studies
• Science, Society and Culture
• Slavic Studies
• Theology & Philosophy
• Peter Lang Classics

• Get IGI Global News

• All Products
• Book Chapters
• Journal Articles
• Video Lessons
• Teaching Cases

Shortly You Will Be Redirected to Our Partner eContent Pro's Website

eContent Pro powers all IGI Global Author Services. From this website, you will be able to receive your 25% discount (automatically applied at checkout), receive a free quote, place an order, and retrieve your final documents .

What is Critical Research Paradigm

Related Books View All Books

Related Journals View All Journals

Research Objectives: The Compass of Your Study

Table of contents

• 1 Definition and Purpose of Setting Clear Research Objectives
• 2 How Research Objectives Fit into the Overall Research Framework
• 3 Types of Research Objectives
• 4 Aligning Objectives with Research Questions and Hypotheses
• 5 Role of Research Objectives in Various Research Phases
• 6.1 Key characteristics of well-defined research objectives
• 6.2 Step-by-Step Guide on How to Formulate Both General and Specific Research Objectives
• 6.3 How to Know When Your Objectives Need Refinement
• 7 Research Objectives Examples in Different Fields
• 8 Conclusion

Embarking on a research journey without clear objectives is like navigating the sea without a compass. This article delves into the essence of establishing precise research objectives, serving as the guiding star for your scholarly exploration.

We will unfold the layers of how the objective of study not only defines the scope of your research but also directs every phase of the research process, from formulating research questions to interpreting research findings. By bridging theory with practical examples, we aim to illuminate the path to crafting effective research objectives that are both ambitious and attainable. Let’s chart the course to a successful research voyage, exploring the significance, types, and formulation of research paper objectives.

Definition and Purpose of Setting Clear Research Objectives

Defining the research objectives includes which two tasks? Research objectives are clear and concise statements that outline what you aim to achieve through your study. They are the foundation for determining your research scope, guiding your data collection methods, and shaping your analysis. The purpose of research proposal and setting clear objectives in it is to ensure that your research efforts are focused and efficient, and to provide a roadmap that keeps your study aligned with its intended outcomes.

To define the research objective at the outset, researchers can avoid the pitfalls of scope creep, where the study’s focus gradually broadens beyond its initial boundaries, leading to wasted resources and time. Clear objectives facilitate communication with stakeholders, such as funding bodies, academic supervisors, and the broader academic community, by succinctly conveying the study’s goals and significance. Furthermore, they help in the formulation of precise research questions and hypotheses, making the research process more systematic and organized. Yet, it is not always easy. For this reason, PapersOwl is always ready to help. Lastly, clear research objectives enable the researcher to critically assess the study’s progress and outcomes against predefined benchmarks, ensuring the research stays on track and delivers meaningful results.

How Research Objectives Fit into the Overall Research Framework

Research objectives are integral to the research framework as the nexus between the research problem, questions, and hypotheses. They translate the broad goals of your study into actionable steps, ensuring every aspect of your research is purposefully aligned towards addressing the research problem. This alignment helps in structuring the research design and methodology, ensuring that each component of the study is geared towards answering the core questions derived from the objectives. Creating such a difficult piece may take a lot of time. If you need it to be accurate yet fast delivered, consider getting professional research paper writing help whenever the time comes. It also aids in the identification and justification of the research methods and tools used for data collection and analysis, aligning them with the objectives to enhance the validity and reliability of the findings.

Furthermore, by setting clear objectives, researchers can more effectively evaluate the impact and significance of their work in contributing to existing knowledge. Additionally, research objectives guide literature review, enabling researchers to focus their examination on relevant studies and theoretical frameworks that directly inform their research goals.

Types of Research Objectives

In the landscape of research, setting objectives is akin to laying down the tracks for a train’s journey, guiding it towards its destination. Constructing these tracks involves defining two main types of objectives: general and specific. Each serves a unique purpose in guiding the research towards its ultimate goals, with general objectives providing the broad vision and specific objectives outlining the concrete steps needed to fulfill that vision. Together, they form a cohesive blueprint that directs the focus of the study, ensuring that every effort contributes meaningfully to the overarching research aims.

• General objectives articulate the overarching goals of your study. They are broad, setting the direction for your research without delving into specifics. These objectives capture what you wish to explore or contribute to existing knowledge.
• Specific objectives break down the general objectives into measurable outcomes. They are precise, detailing the steps needed to achieve the broader goals of your study. They often correspond to different aspects of your research question , ensuring a comprehensive approach to your study.

To illustrate, consider a research project on the impact of digital marketing on consumer behavior. A general objective might be “to explore the influence of digital marketing on consumer purchasing decisions.” Specific objectives could include “to assess the effectiveness of social media advertising in enhancing brand awareness” and “to evaluate the impact of email marketing on customer loyalty.”

Aligning Objectives with Research Questions and Hypotheses

The harmony between what research objectives should be, questions, and hypotheses is critical. Objectives define what you aim to achieve; research questions specify what you seek to understand, and hypotheses predict the expected outcomes.

This alignment ensures a coherent and focused research endeavor. Achieving it necessitates a thoughtful consideration of how each component interrelates, ensuring that the objectives are not only ambitious but also directly answerable through the research questions and testable via the hypotheses. This interconnectedness facilitates a streamlined approach to the research process, enabling researchers to systematically address each aspect of their study in a logical sequence. Moreover, it enhances the clarity and precision of the research, making it easier for peers and stakeholders to grasp the study’s direction and potential contributions.

Role of Research Objectives in Various Research Phases

Throughout the research process, objectives guide your choices and strategies – from selecting the appropriate research design and methods to analyzing data and interpreting results. They are the criteria against which you measure the success of your study. In the initial stages, research objectives inform the selection of a topic, helping to narrow down a broad area of interest into a focused question that can be explored in depth. During the methodology phase, they dictate the type of data needed and the best methods for obtaining that data, ensuring that every step taken is purposeful and aligned with the study’s goals. As the research progresses, objectives provide a framework for analyzing the collected data, guiding the researcher in identifying patterns, drawing conclusions, and making informed decisions.

Crafting Effective Research Objectives

The effective objective of research is pivotal in laying the groundwork for a successful investigation. These objectives clarify the focus of your study and determine its direction and scope. Ensuring that your objectives are well-defined and aligned with the SMART criteria is crucial for setting a strong foundation for your research.

Key characteristics of well-defined research objectives

Well-defined research objectives are characterized by the SMART criteria – Specific, Measurable, Achievable, Relevant, and Time-bound. Specific objectives clearly define what you plan to achieve, eliminating any ambiguity. Measurable objectives allow you to track progress and assess the outcome. Achievable objectives are realistic, considering the research sources and time available. Relevant objectives align with the broader goals of your field or research question. Finally, Time-bound objectives have a clear timeline for completion, adding urgency and a schedule to your work.

Step-by-Step Guide on How to Formulate Both General and Specific Research Objectives

So lets get to the part, how to write research objectives properly?

• Understand the issue or gap in existing knowledge your study aims to address.
• Gain insights into how similar challenges have been approached to refine your objectives.
• Articulate the broad goal of research based on your understanding of the problem.
• Detail the specific aspects of your research, ensuring they are actionable and measurable.

How to Know When Your Objectives Need Refinement

Your objectives of research may require refinement if they lack clarity, feasibility, or alignment with the research problem. If you find yourself struggling to design experiments or methods that directly address your objectives, or if the objectives seem too broad or not directly related to your research question, it’s likely time for refinement. Additionally, objectives in research proposal that do not facilitate a clear measurement of success indicate a need for a more precise definition. Refinement involves ensuring that each objective is specific, measurable, achievable, relevant, and time-bound, enhancing your research’s overall focus and impact.

Research Objectives Examples in Different Fields

The application of research objectives spans various academic disciplines, each with its unique focus and methodologies. To illustrate how the objectives of the study guide a research paper across different fields, here are some research objective examples:

• In Health Sciences , a research aim may be to “determine the efficacy of a new vaccine in reducing the incidence of a specific disease among a target population within one year.” This objective is specific (efficacy of a new vaccine), measurable (reduction in disease incidence), achievable (with the right study design and sample size), relevant (to public health), and time-bound (within one year).
• In Environmental Studies , the study objectives could be “to assess the impact of air pollution on urban biodiversity over a decade.” This reflects a commitment to understanding the long-term effects of human activities on urban ecosystems, emphasizing the need for sustainable urban planning.
• In Economics , an example objective of a study might be “to analyze the relationship between fiscal policies and unemployment rates in developing countries over the past twenty years.” This seeks to explore macroeconomic trends and inform policymaking, highlighting the role of economic research study in societal development.

These examples of research objectives describe the versatility and significance of research objectives in guiding scholarly inquiry across different domains. By setting clear, well-defined objectives, researchers can ensure their studies are focused and impactful and contribute valuable knowledge to their respective fields.

Defining research studies objectives and problem statement is not just a preliminary step, but a continuous guiding force throughout the research journey. These goals of research illuminate the path forward and ensure that every stride taken is meaningful and aligned with the ultimate goals of the inquiry. Whether through the meticulous application of the SMART criteria or the strategic alignment with research questions and hypotheses, the rigor in crafting and refining these objectives underscores the integrity and relevance of the research. As scholars venture into the vast terrains of knowledge, the clarity, and precision of their objectives serve as beacons of light, steering their explorations toward discoveries that advance academic discourse and resonate with the broader societal needs.

Readers also enjoyed

WHY WAIT? PLACE AN ORDER RIGHT NOW!

Just fill out the form, press the button, and have no worries!

We use cookies to give you the best experience possible. By continuing we’ll assume you board with our cookie policy.

• Open access
• Published: 09 April 2024

Active involvement in scientific research of persons living with dementia and long-term care users: a systematic review of existing methods with a specific focus on good practices, facilitators and barriers of involvement

• Janneke M. Groothuijse 1 , 2 ,
• Lisa S. van Tol 1 , 2 ,
• C. C. M. (Toos) Hoeksel-van Leeuwen 1 , 2 ,
• Johannes J. M. van Delden 3 ,
• Monique A. A. Caljouw 1 , 2 &
• Wilco P. Achterberg 1 , 2  

BMC Geriatrics volume  24 , Article number:  324 ( 2024 ) Cite this article

119 Accesses

Metrics details

Active involvement of persons living with dementia (PLWD) and long-term care (LTC) users in research is essential but less developed compared to other patient groups. However, their involvement in research is not only important but also feasible. This study aims to provide an overview of methods, facilitators, and barriers for involving PLWD and LTC users in scientific research.

A systematic literature search across 12 databases in December 2020 identified studies involving PLWD, LTC users, or their carers beyond research subjects and describing methods or models for involvement. Qualitative descriptions of involvement methods underwent a risk of bias assessment using the Critical Appraisal Skills Programme (CASP) Qualitative Checklist 2018. A data collection sheet in Microsoft Excel and thematic analysis were used to synthesize the results.

The eighteen included studies delineated five core involvement methods spanning all research phases: advisory groups, formal and informal research team meetings, action groups, workshops, and co-conducting interviews. Additionally, two co-research models with PLWD and carers were found, while only two studies detailed LTC user involvement methods. Four distinct involvement roles were identified: consulting and advisory roles, co-analysts, co-researchers, and partners. The review also addressed barriers, facilitators, and good practices in the preparation, execution, and translation phases of research, emphasizing the importance of diversity, bias reduction, and resource allocation. Trust-building, clear roles, ongoing training, and inclusive support were highlighted.

Conclusions

Planning enough time for active involvement is important to ensure that researchers have time to build a trusting relationship and meet personal needs and preferences of PLWD, LTC users and carers. Researchers are advised not to presume the meaning of burden and to avoid a deficit perspective. A flexible or emergent design could aid involved persons’ ownership of the research process.

Trial registration

Prospero 2021: CRD42021253736.

Peer Review reports

In research characterized by active involvement, the target group plays a pivotal role in shaping research decisions and outcomes, directly impacting them. Involving patients in health research offers significant benefits [ 1 , 2 ]: it enhances participant recruitment [ 2 ], refines research questions [ 2 ], aligns study results with the target population [ 1 , 2 ], and promotes effective implementation of findings [ 1 ]. Active involvement of patients has also benefits for themselves, namely an enhanced understanding of research, building relationships, personal development, improved health and wellbeing, and enjoyment and satisfaction [ 3 , 4 ]. It gives them a sense of purpose and satisfaction through their tangible impact.

However, for long-term care (LTC) users and persons living with dementia (PLWD) active involvement in research is less developed than for other patient groups [ 5 , 6 ]. PLWD and LTC users share similar care needs, encompassing assistance with activities of daily living (ADLs), medication management, medical condition monitoring, and emotional support. Furthermore, a substantial portion of LTC users comprises individuals living with dementia [ 7 ]. Additionally, statistical data from the United States reveals that one in four older individuals is likely to reside in long-term care (LTC) facilities [ 8 ], and approximately forty to eighty percent of LTC residents in the United States, Japan, Australia, and England experience dementia or severe memory problems [ 7 , 9 ].

Due to these considerations, we have chosen to combine the target audiences of PLWD and LTC users in our systematic review. However, it's important to note that while there are potential advantages to combining these target groups, there may also be challenges. PLWD and LTC users may have varying needs, preferences, and experiences, including differences in care requirements driven by individual factors like the stage of dementia, coexisting conditions, and personal preferences. Therefore, it's imperative to conduct comprehensive research and involve these communities to ensure that involvement approaches are not only inclusive but also tailored to meet their specific requirements.

Given our ageing population and the intricate health challenges faced by PLWD and LTC users, including their vulnerability and shorter life expectancy in old age, it's crucial to establish effective research involvement methods. These individuals have unique needs and preferences that require attention. They possess a voice, and as researchers, it is our responsibility to not only listen to them but also actively involve them in the research process. Consequently, it is essential to identify means through which the voices of PLWD and LTC users can be effectively heard and ensure that their input is incorporated into research.

Fortunately, publication of studies on involvement of PLWD and LTC users in scientific research is slowly increasing [ 5 , 9 , 10 , 11 ]. A few reviews have described how PLWD and LTC users were involved [ 5 , 9 , 10 ]. However, with the increasing attention for involvement, the understanding of when involvement is meaningful grows and stricter requirements can be imposed to increase the quality of active involvement [ 12 , 13 ]. To our knowledge there is no up to date overview of involvement methods used with either or both PLWD and LTC users. Such an overview of involvement methods for PWLD and LTC users would provide a valuable, comprehensive resource encompassing various stages of the research cycle and different aspects of involvement. It would equip researchers with the necessary guidance to navigate the complexities of involving PLWD and LTC users in their research projects.

Recognizing the need to enhance the involvement of PLWD and LTC users in scientific research, this systematic review aims to construct a comprehensive overview of the multiple methodologies employed in previous studies, along with an examination of the facilitators and barriers of involvement. Our overarching goal is to promote inclusive and effective involvement practices within the research community. To achieve this objective, this review will address the following questions: (1) What kind of methods are used and how are these methods implemented to facilitate involvement of PLWD and LTC users in scientific research? (2) What are the facilitators and barriers encountered in previous research projects involving PLWD and LTC users?

Protocol and registration

The search and analysis methods were specified in advance in a protocol. The protocol is registered and published in the PROSPERO database with registration number CRD42021253736. The search and analysis methods are also described below more briefly.

Information sources, search strategy, and eligibility criteria

In preparation of the systematic literature search, key articles and reviews about involvement of PLWD and LTC users in research were screened to identify search terms. In addition, Thesaurus and MeSH terms were used to broaden the search. The search was conducted on December 10, 2020, across multiple databases: PubMed, Medline, Embase, Emcare, Web of Science, Cochrane Library, PsycINFO, Academic Search Premier, JSTOR, Social Services Abstracts, Sociological Abstracts, Psychology and Behavioral Sciences Collection. The search terms were entered in "phrases". The search strategy included synonymous and related terms for dementia, LTC user, involvement, research, method, and long-term care. The full search strategy is provided in supplement 1 .

After conducting the search, records underwent initial screening based on titles and abstracts. Selected reports were retrieved for full-text assessment, and studies were evaluated for eligibility based on several criteria. However, no restriction was made regarding publication date. First, to be included studies had to be written in English, German, French, or Dutch. Second, we only included original research studies. Third, studies were excluded when the target group or their representatives were not involved in research, but only participated as research subjects. Fourth, studies were excluded when not describing involvement in research. Therefore, studies concerning involvement in care, policy, or self-help groups were excluded. Fifth, the focus of this systematic review is on methods. Therefore, studies with a main focus on the results, evaluation, ethical issues, and impact of involvement in research were excluded. Additionally, we have not set specific inclusion or exclusion criteria based on study design since our primary focus is on involvement methodologies, regardless of the chosen research design. Sixth, the included studies had to concern the involvement in research of PLWD or adult LTC users, whether living in the community or in institutional settings, as well as informal caregivers or other representatives of these groups who may represent PLWD and LTC users facing limitations. Studies that involved LTC users that were children or ‘young adults’, or their representatives, were excluded. Studies were also excluded if they involved mental healthcare users if it remained unclear if the care that they received entailed more than only treatment from mental healthcare providers, but for example also assistance with ADL.

Terminology

For readability purposes, we use the abbreviation PLWD to refer to persons diagnosed with dementia, and we use the abbreviation LTC users to refer to persons receiving long-term care, at home or as residents living in nursing homes or other residential facilities. We use the term carers to refer to informal caregivers and other representatives of either PLWD or LTC users. As clear and consistent definitions regarding participatory research remains elusive [ 14 , 15 ], we formulated a broad working definition of involvement in research so as not to exclude any approach to participatory research. We defined involvement in research as “research carried out ‘with’ or ‘by’ the target group” [ 16 ], where the target group or their representatives take part in the governance or conduct of research and have some degree of ownership of the research [ 12 ]. It concerns involvement in research in which lived experienced experts work alongside research teams. We use the terms participation and participants, to refer to people being part of the research as study subjects.

Selection process, data-collection process, and data items

Titles and abstracts were independently screened by the first and second author (JG and LT). Only the studies that both reviewers agreed and met the inclusion criteria were included in the full-text screening process. Any uncertainty about whether the studies truly described a model or approach for involvement, was resolved by a quick screening of the full-text paper. The full-text screening process was then conducted according to the same procedure by JG and LT. Any disagreement was resolved by discussion until consensus was reached. If no agreement could be reached, a third researcher (MC) was consulted. References of the included studies were screened for any missing papers.

The following information was collected on a data collection sheet in Microsoft Excel: year and country of publication, topic, research aim, study design, living situation of involved persons (at home or institutionalized), description of involved persons, study participants (study subjects), theories and methods used, type/role(s) of involvement, research phase(s), recruitment, consent approach, study setting, structure of participatory activities, training, resources, facilitators, barriers, ethics, benefits, impact, and definition of involvement used.

JG independently extracted data from all included studies, the involved co-researcher (THL) independently extracted data from two studies, the second author (LT) from five. Differences in the analysis were discussed with the co-researcher (THL) and second author (LT) until consensus was reached. As only minor differences emerged, limited to the facilitator and barrier categories, data from the remaining studies was extracted by JG.

Risk of bias assessment

Every research article identified through the systematic review exclusively comprised qualitative descriptions of the involvement method(s) employed. Consequently, all articles underwent evaluation using the Critical Appraisal Skills Programme (CASP) Qualitative Checklist 2018 [ 17 ], as opposed to the checklists intended for quantitative or mixed methods research. All included studies were independently assessed on quality by two reviewers (JG,LT) and any disagreement was resolved by discussion until consensus was reached. The CASP Qualitative Checklist consists of ten questions. The checklist does not provide suggestions on scoring, the first author designed a scoring system: zero points if no description was provided (‘no’), one point if a minimal description was provided (‘can’t tell’) and two points when the question was answered sufficiently (‘yes’). The second question of the checklist, “is a qualitative methodology appropriate”, was not applicable to the aims (i.e., to describe involvement) of the included studies and was therefore excluded. The tenth question was translated into a ‘yes’, ‘can’t tell’, or ‘no’ score to fit the scoring system. A maximum of eighteen points could be assigned.

Synthesis methods

Tables were used to summarize the findings and to acquire an overview of (1) the kinds of methods used to enable involvement of PLWD, LTC users, or carers in scientific research, and (2) the facilitators and barriers for involving this target group in scientific research. As to the first research aim, the headings of the first two tables are based on the Guidance for Reporting Involvement of Patients and the Public, long form version 2 (GRIPP2-LF) [ 18 ]. Because our systematic review focusses on methods, only the topics belonging to sections two, three, and four were included. Following Shippee et al., three main research phases were distinguished: preparation, execution, and translation [ 19 ]. Furthermore, the following fields were added to the GRIPP2-LF: First author, year of publication, country of study, setting of involvement, frequency of meetings, and a summary description of activities.

Concerning the second research aim, the extracted facilitators, barriers, and good practices were imported per study in ATLAS.ti for qualitative data analysis. Following the method for thematic synthesis of qualitative studies in systematic reviews [ 20 ], all imported barriers, facilitators and good practices were inductively coded staying 'close' to the results of the original studies, which resulted in 50 initial codes. After multiple rounds of pile sorting [ 21 ], based on similarities and differences and discussions in the research team, this long code list was grouped into a total of 27 categories, which were thereafter subsequently organized into 14 descriptive themes within the three research phases (preparation, execution, translation).

Study selection and characteristics

The Prisma Flow Diagram was used to summarize the study selection process [ 22 ]. In the full text screening, 72 of the 93 remaining studies were excluded because they were not original research articles (n = 5), not about involvement (n = 8), not about involvement in a research project (n = 1), they did not describe a model or method for involvement (n = 34), or they were not about PLWD or LTC users (n = 24). The search resulted in 18 publications eligible for analysis (Fig.  1 ).

Preferred Reporting items for Systematic Reviews and Meta-analyses (PRISMA) flow diagram

Table 1 presents the general study characteristics. Two studies explicitly aimed to develop a model for involvement or good practice, and both focus on co-research either with PLWD [ 23 ] or their carers [ 13 ]. The other sixteen provide a description of the involvement of PLWD [ 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 ] or LTC users in their research projects [ 35 , 36 , 37 , 38 , 39 ].

Quality assessment

Table 1 presents the CASP-score per study [ 17 ]. Five scored 16 to 18 points [ 13 , 28 , 29 , 32 , 35 ], indicating high quality with robust methods, clear aims, and strong data analysis. Eleven scored 12 to 15 [ 23 , 24 , 26 , 30 , 32 , 33 , 34 , 36 , 37 , 38 , 39 ], showing generally strong methodologies but with some limitations. Two scored 9 or lower [ 25 , 27 ], signifying significant methodological and analytical shortcomings. Notably, these low-scoring studies were short articles lacking clear recommendations for involvement in research.

Design and implementation of involvement

Phases and methods of involvement.

Table 2 describes the involvement methods used for and the implementation of involvement in research. The included studies jointly presented methods for involvement in the three main research phases [ 19 ]. Regarding the preparation phase, which involves the preparatory work for the study, only three studies provided detailed descriptions of the methods employed [ 26 , 30 , 32 ]. The execution phase, encompassing the actual conduct of the research, was most frequently discussed [ 23 , 24 , 25 , 26 , 27 , 28 , 29 ]. Five studies addressed the translation phase [ 13 , 25 , 31 , 36 , 37 ], where the focus shifts to translating research findings into actionable outcomes.

The eighteen studies introduced a variety of involvement methods, categorizable into five groups: 1) advisory groups, 2) research team meetings (both formal and informal), 3) action groups, 4) workshops, and 5) co-research in interviews. In five studies, individuals including PLWD, LTCF residents, carers, and health professionals participated in advisory/reference groups [ 25 , 26 , 27 , 32 ], working groups [ 27 ], and panels [ 28 ]. These groups offered valuable feedback on research aspects, spanning protocols, design, questionnaires, and implementation of research. Meetings occurred at varying frequencies - monthly, quarterly, or biannually.

Two studies exemplify diverse research collaboration settings. One involving older individuals within an academic research team of five [ 37 ], and another featuring a doctoral student and a co-researcher conducting informal monthly discussions at a local coffee shop [ 31 ]. Brown et al. sought to minimize power differentials and enhance inclusivity [ 37 ], while Mann and Hung focused on benefiting people with dementia and challenging negative discourse on dementia [ 31 ].

An additional five studies employed methods involving frequent meetings, including action [ 35 , 39 ], inquiry [ 23 ], and discussion groups [ 29 , 36 ] In these groups, involved persons with lived experience contributed to developing a shared vision and community improvements, such as enhancing the mealtime experience in care facilities [ 35 ].

Seven studies involved individuals through workshops, often conducted over one or two sessions. These workshops contributed to generating recommendations [ 37 ], informing future e-health designs [ 29 , 30 ], and ensuring diverse perspectives and lived experiences were included in data analysis [ 13 , 24 , 32 , 33 ]. In three studies, representatives worked as co-researchers in interviews, drawing on personal experiences to enhance the interview process, making it more dementia-appropriate and enriching data collection [ 13 , 32 , 34 ]. Finally, one study involved representatives in the recruitment and conduct of interviews [ 38 ].

People involved

The number of persons involved varied from a single co-researcher [ 31 ] to 34 panel individuals providing feedback on their experiences in a clinical trial [ 28 ]. Thirteen studies focussed on PLWD: eleven involved PLWD themselves [ 23 , 24 , 25 , 26 , 27 , 29 , 30 , 31 , 32 , 33 , 34 ], one exclusively focused on caregivers [ 13 ], and another one involved people without or with mild cognitive impairment, who participated in a study examining the risks of developing Alzheimer's disease [ 28 ]. Although not all articles provided descriptions of the dementia stage, available information indicated that individuals involved typically fell within the early to mid-stages of dementia [ 29 , 30 , 32 , 33 , 34 ]. Next to PLWD and carers, two studies additionally involved organizational or advocacy representatives [ 25 , 27 ]. The other five studies concerned older adults living in a LTC facility. Two of them involved older residents themselves [ 35 , 39 ], the other three carers, older community/client representatives or health care practitioners [ 36 , 37 , 38 ].

Roles and level of involvement

Four general roles could be identified. First, consultation and advisory roles were held by PLWD and carers [ 25 , 26 , 27 , 28 , 29 , 30 , 32 ], where involved persons share knowledge and experiences to make suggestions [ 32 ], but the research team retained formal decision-making power [ 25 ]. Second, PLWD were involved as co-analysts in data analysis [ 24 , 32 , 33 ]. Co-analysts influence data analysis, but the decision-making power remained with academic researchers [ 24 ]. Third, in six studies the co-researcher role was part of the research design in which involved persons and researchers steer and conduct research together [ 13 , 23 , 31 , 32 , 34 , 36 ]. Finally, two studies partnered with LTC residents [ 35 , 39 ], with residents at the core of the group, and positioned as experts by experience [ 39 ]. Residents had the decision-making authority regarding how to improve life in LTC facilities [ 35 ].

Models for involvement in research

Only two studies designed a model for co-research with PLWD [ 23 ] or their carers [ 13 ] across all research phases. These models underscored the importance of iterative training for co-researchers [ 13 , 23 ] and academic researchers [ 23 ]. Furthermore, these studies advocate involving co-researchers early on in the research process [ 13 ] and in steering committees [ 23 ]. Co-researchers can be involved in designing research materials [ 23 ], conducting interviews [ 13 , 23 ], analysing data [ 13 ], and co-disseminating findings [ 13 , 23 ]. Additionally, one study stressed involving PLWD in identifying (future) research priorities [ 23 ].

Barriers, facilitators, and good practices in research phases

Preparation phase.

Table 3 describes the barriers, facilitators, and good practices per main research phase. Lack of diversity in ethnicity and stages of dementia in the recruitment of involved persons is mentioned as a recurring barrier [ 26 , 28 , 32 , 33 ]. The exclusion of people with cognitive impairments is partly due to gatekeepers’ and recruiters’ bias towards cognitively healthy people [ 28 , 32 ]. It is stressed that researchers should refrain from making assumptions about the abilities of PLWD and ask the person what he/she is willing to do [ 31 ]. It is considered good practice to involve people regardless of cognitive abilities [ 23 ], based on skills, various personal characteristics [ 13 ] and, if possible, relevant prior experience [ 38 ].

Many studies stress the importance of building a mutual trusting relationship between involved persons and academic researchers [ 13 , 23 , 31 , 33 , 34 , 37 ]. A good relationship is believed to break down social barriers [ 37 ], foster freedom of expression [ 33 ], and thereby avoiding tokenistic involvement [ 13 ]. In addition, spending time with these persons is important to become familiar with an individual’s strengths and limitations [ 31 ].

Opting for naturally evolving involvement roles was mentioned as a barrier, as this may result in conflicting expectations and irrelevant tasks [ 37 ]. A clear role description and clarification of tasks is key to balancing potentially different expectations of the involved persons and researchers [ 26 , 28 , 29 , 32 , 38 ]. When designing a role for involvement in research, good practices dictate taking into account personal skills, preferences, development goals, and motivation for involvement [ 13 , 32 ]. This role should ideally be designed in collaboration with involved persons [ 13 , 32 ].

The perception of providing training to involved persons is ambivalent. Studies cited that training should not aim to transform them into “pseudo-scientist” [ 32 , 37 ] and that it raises the costs for involvement [ 28 ]. However, multiple scholars emphasize the importance of providing iterative training to facilitate meaningful involvement and development opportunities [ 13 , 23 , 28 , 31 , 32 , 33 , 36 , 37 ]. Training can empower involved persons to engage in the research process equally and with confidence, with the skills to fulfil their role [ 13 , 33 , 38 ]. However, the implementation of training may present a potential conflict with the fundamental principle of valuing experiential knowledge [ 37 ] and should avoid the objective of transforming co-researchers into 'expert' researchers [ 32 ]. Academic researchers should also be offered training on how to facilitate meaningful involvement [ 13 , 23 , 28 , 31 ].

Limited time and resources were mentioned as barriers to involvement that can delay the research process [ 13 , 33 , 36 , 39 ], restrict the involvement [ 28 ] and hinder the implementation of developed ideas [ 39 ]. Financial compensation for involvement is encouraged [ 25 , 26 , 27 , 32 ], as it acknowledges the contribution of involved persons [ 13 ]. Thus, meaningful involvement in research requires adequate funding and infrastructure to support the involvement activities [ 13 , 28 , 33 , 37 ].

Execution phase

The use of academic jargon and rapid paced discussions [ 13 , 37 ], power differentials, and the dominant discourse in biomedical research on what is considered “good science” can limit the impact of involvement [ 13 , 24 , 32 , 36 , 37 ]. Facilitating researchers should reflect on power differentials [ 35 ] and how decision-making power is shared [ 31 ]. Other facilitating factors are making a glossary of terms used and planning separate meetings for “technical topics” [ 37 ]. In addition, an emergent research design [ 35 ] or a design with flexible elements [ 28 ] can increase ownership in the research project and provide space for involvement to inform the research agenda [ 28 , 35 ]. This requires academic researchers to value experiential knowledge and to have an open mind towards the evolving research process [ 13 , 23 , 31 ].

Furthermore, managing the involvement process and ensuring equity in the collaboration [ 13 , 32 , 33 ], facilitating researchers must encourage involved persons to voice their perspectives. This means that they sometimes need to be convinced that they are experts of lived experience [ 32 , 33 , 36 , 37 , 39 ]. To enable involvement of PLWD, the use of visual and creative tools to prompt memories can be considered [ 24 , 30 , 33 , 34 ], as well as flexibility in relation to time frames and planning regular breaks to avoid too fast a pace for people who may tire easily [ 24 , 25 , 29 , 30 ].

Involvement can be experienced as stressful [ 13 , 32 , 38 ] and caring responsibilities may interfere [ 26 ]. Tailored [ 29 ] physical and emotional support should therefore be offered [ 13 , 23 , 38 ] without making assumptions about the meaning of burden [ 30 , 31 ]. Moreover, being the only PLWD involved in an advisory group was experienced as intimidating [ 25 ] and, ideally, a larger team of PLWD is involved to mitigate responsibilities [ 37 ]. PLWD having a focal point of contact [ 28 , 37 ] and involving nurses or other staff with experience working with PLWD and their carers [ 29 , 30 ] are mentioned as being beneficial. Some stress the importance of involving carers when engaging with PLWD in research [ 25 , 29 , 30 ].

To avoid an overload of information that is shared with the involved persons, tailoring information-sharing formats to individual preferences and abilities is essential to make communication effective [ 27 , 37 ].

Translation

Two studies indicated a need for more robust evaluation measures to assess the effect of involvement [ 28 , 33 ]. Reflection and evaluation of the involvement serves to improve the collaboration and to foster introspective learning [ 13 , 23 , 26 , 31 ]. The included studies evaluated involvement through the use of reflective diaries [ 13 ] or a template [ 38 ] with open-ended questions [ 33 ].

Two studies postulate that findings should benefit and be accessible to PLWD [ 23 , 31 ]. The use of creative tools not only enables involvement of PLWD, but can also increase accessibility of research findings and expand the present representation of PLWD [ 23 ].

The 18 included studies presented multiple methods for involvement in all three research phases. We found five types of involvement: advisory groups, (formal and informal) research team meetings, action groups, workshops, and co-conducting interviews. Only two studies described methods for involvement of LTC users in research. Involved persons were most often involved in consulting and advisory roles, but also as co-analysts, co-researchers, and partners. Involved persons’ roles can evolve and change over time. Especially as involved persons grow into their role, and gain confidence and knowledge of the specific research project, a more active role with shared responsibilities can become part of the research project. In addition, multiple involvement roles can be used throughout the research depending on the research phase.

Compared to the five types of involvement that we identified, other literature reviews about involvement methods for LTC users and PLWD in research also described advisory groups [ 10 ] and workshops [ 5 , 11 ], and methods that were similar to research team meetings (drop-in sessions and meetings [ 11 ]). Methods for action research (action groups) and co-conducting research (interviews) were not included by these other review studies. In addition to our findings, these other reviews also described as involvement methods interviews and focus groups [ 5 , 10 ] surveys [ 10 ], reader consultation [ 11 ]. Those types of methods were excluded from our study, because our definition of involvement is more strict; collecting opinions is not involvement per se, but sometimes only study participation. Moreover, compared to these previous reviews we set a high standard for transparency about the participation methods and the level of detail at which they are described.

Engaging the target group in research, particularly when collaborating with PLWD, LTC users, and carers, involves navigating unforeseen challenges [ 40 ]. This requires academic researchers to carefully balance academic research goals and expectations, and the expectations, personal circumstances and development goals related to the involved person. The aim is to maximize involvement while being attentive to the individual’s needs and avoiding a deficit perspective. Effective communication should be established, promoting respect, equality, and regular feedback between all stakeholders, including individuals living with dementia and LTCF staff. Building a mutual trusting relationship between involved persons and academic researchers through social interaction and clear communication is key to overcome barriers and ensure meaningful involvement. Inclusivity and empowerment, along with fostering an environment where diverse voices are heard, are crucial for the success of involvement in research. Our results are in line with a recent study concerning the experiences of frail older persons with involvement in research, confirming the importance of avoiding stereotypic views of ageing and frailty, building a trusting relationship, and being sensitive to older persons’ preferences and needs [ 41 ].

Furthermore, our results show that training academic researchers and involved persons is essential to develop the skills to facilitate involvement and to fulfil their role with confidence, respectively. Whilst the need for training is acknowledged by others [ 41 , 42 ], there are legitimate objections to the idea of training involved persons, as the professionalization underpinning the concept of training is at odds with voicing a lay perspective [ 43 , 44 ]. Furthermore, it is argued that experiential knowledge is compromised when training is structured according to the dominant professional epistemology of objectivity [ 45 ]. Therefore, training of involved persons should not focus on what researchers think they ought to know, but on what they want to learn [ 41 ].

Academic culture was frequently mentioned as a barrier to meaningful involvement. This result resonates with the wider debate related to involvement in health research which is concerned about active or “authentic involvement” being replaced with the appropriation of the patient voice as an add-on to conventional research designs [ 12 , 46 ]. It is argued that such tokenistic involvement limits the involved persons’ ability to shape research outcomes [ 46 ]. To reduce tokenism requires a culture shift [ 13 ]. We believe that due to the strict definition of involvement and high transparency standard used in this review, tokenistic approaches were excluded. This may set an example for how to stimulate making this culture shift.

Furthermore, the importance of practical aspects such as funding and, by extension, the availability of time should not be underestimated. Adequate funding is necessary for compensation of involvement, but also to ensure that researchers have ample time to plan involvement activities and provide personalized support for PLWD, LTC residents and their carers. Funding bodies increasingly require involvement of the public to be part of research proposals. Yet, support in terms of financial compensation and time for the implementation of involvement in research is rarely part of funding grants [ 42 ]. In addition, whereas an emergent design could aid the impact of involvement, funders often require a pre-set research proposal in which individual components are already fixed [ 5 , 47 ]. This indicates that not only do academic researchers and culture need to change, academic systems also need to be modified in order to facilitate and nurture meaningful involvement [ 47 ].

Strengths and limitations

A key strength of this review is the inclusion of over ten scientific databases, with a reach beyond the conventional biomedical science databases often consulted in systematic reviews. Besides, we believe that we have overcome the inconsistent use of terminology of involvement in research by including also other terms used, such as participation and engagement, in our search strategy. However, there was also inconsistency in length of publications and precision of the explanation of the process of involvement. E.g., involvement in the execution phase was often elaborated on, contributions to the research proposal and co-authoring research findings were only stated and not described. This presented challenges for data extraction and analysis, as it was not always possible to identify how the target group was involved. Involvement in these research phases is therefore not fully represented in this review.

The included studies in this review, the majority of which are of high quality, provide methods for involvement of PLWD and LTC users in research and they do not explicitly attend to the effectiveness or impact of the method for involvement used. Therefore, a limitation of this review is that it cannot make any statements regarding the effectiveness of the involvement methods included. Moreover, our target population was broad, although PLWD and LTC users are largely overlapping in their care needs and share important features, this may have led to heterogeneous results. In future research, it would be interesting to interpret potential differences between involvement of PLWD, LTC users, and their carers. However, as we expected, the amount of literature included in our analyses was too limited to do so. Furthermore, whereas the broad target group is a limitation it is also a strength of our review. Limiting our search to specifically persons living in LTC facilities would have provided limited methods for involvement of persons living with dementia. Our broad target groups enabled us to learn from research projects in which people living with early staged dementia are directly involved from which we can draw lessons on the involvement of people with more advanced stages of dementia and persons living with cognitive problems who live within LTC facilities.

Since January 2021 quite some research has been published about the importance of involvement in research. Although we had quickly screened for new methods, we realise that we may have missed some involvement methods in the past years. There will be a need for a search update in the future.

Implications for future research

Our review shows that a flexible and emergent design may help to increase involved persons' influence on and ownership in the research process. However, not all research objectives may be suitable for the implementation of an emergent design. Future research should therefore examine how aspects of a flexible emergent design can be integrated in, e.g., clinical research without compromising the validity of research outcomes.

Alzheimer Europe has called for the direct involvement of persons living with dementia in research [ 48 ]. In addition, Swarbrick et al. (this review) advise to involve persons regardless of their cognitive abilities [ 23 ]. These statements question the involvement of proxies, such as carers, professional caregivers and others involved in the care of PLWD. While PLWD and persons with other cognitive problems constitute a significant group within residential and nursing homes [ 7 ], none of the studies included in this review have provided methods to directly involve persons with more advanced stages of dementia. This raises the question if research methods should be adapted to allow those with more advanced stages of dementia to be involved themselves or if, concerning the progressive nature of the disease, it is more appropriate to involve proxies. And secondly who should these proxies be? Those that care for and live with persons with an advanced stage of dementia, or for example a person living with an early stage of dementia to represent the voices of persons with more advanced stages of dementia [ 31 ]?

Future research should adopt our example for stricter requirements for involvement and transparency about the involvement methods used. This will reduce tokenistic involvement and further promote the culture shift towards meaningful involvement. In addition, future research should assess the impact of the involvement methods that are described in this review. One of the first instruments that that may be used to do so in varying healthcare settings is the Public and Patient Engagement Evaluation Tool (PPEET) [ 49 ]. Moreover, scholars in this review stress, and we agree with this, that future research is needed on the involvement of persons with more advanced stages of dementia to ensure their voices are not excluded from research [ 33 , 34 ].

This review provides an overview of the existing methods used to actively involve PLWD, LTC users, and carers in scientific research. Our findings show that their involvement is feasible throughout all research phases. We have identified five different methods for involvement, four different roles, and two models for co-research. Our results suggest that planning enough time for involving PLWD, LTC users, and carers in research, is important to ensure that researchers have time to build a trusting relationship and meet their personal needs and preferences. In addition, researchers are advised not to presume the meaning of burden and to avoid a deficit perspective. A flexible or emergent design could aid involved persons’ ownership in the research process.

Availability of data and materials

The full search strategy is provided in supplement 1 . The data extraction form can be provided by the corresponding author on reasonable request.

Abbreviations

Critical Appraisal Skills Programme

Guidance for Reporting Involvement of Patients and the Public, long form version 2

• Long-term care

Persons living with dementia

Domecq JP, Prutsky G, Elraiyah T, et al. Patient engagement in research: a systematic review. BMC Health Serv Res. 2014;14(1):1–9. https://doi.org/10.1186/1472-6963-14-89 .

Article   Google Scholar  

Brett J, Staniszewska S, Mockford C, et al. A systematic review of the impact of patient and public involvement on service users, researchers and communities. Patient-Patient-Centered Outcomes Res. 2014;7(4):387–95. https://doi.org/10.1007/s40271-014-0065-0 .

Staley K. Exploring impact: Public involvement in NHS, public health and social care research. 2009. Available from: https://www.invo.org.uk/wp-content/uploads/2011/11/Involve_Exploring_Impactfinal28.10.09.pdf . Accessed 17 Jan 2024

Ashcroft J, Wykes T, Taylor J, et al. Impact on the individual: what do patients and carers gain, lose and expect from being involved in research? J Ment Health. 2016;25(1):28–35. https://doi.org/10.3109/09638237.2015.1101424 .

Article   PubMed   PubMed Central   Google Scholar  

Backhouse T, Kenkmann A, Lane K, et al. Older care-home residents as collaborators or advisors in research: a systematic review. Age Ageing. 2016;45(3):337–45. https://doi.org/10.1093/ageing/afv201 .

Bendien E, Groot B, Abma T. Circles of impacts within and beyond participatory action research with older people. Ageing Soc 2020:1–21. https://doi.org/10.1017/S0144686X20001336 .

Lepore MED, Meyer J, Igarashi A. How long-term care quality assurance measures address dementia in Australia, England, Japan, and the United States. Ageing Health Res 2021;1(2). https://doi.org/10.1016/j.ahr.2021.100013 .

Freedman VA, Spillman BC. Disability and care needs among older Americans. Milbank Q. 2014;92(3):509–41. https://doi.org/10.1111/1468-0009.12076 .

Bethell J, Commisso E, Rostad HM, et al. Patient engagement in research related to dementia: a scoping review. Dementia. 2018;17(8):944–75. https://doi.org/10.1177/1471301218789292 .

Article   PubMed   Google Scholar  

Miah J, Dawes P, Edwards S, et al. Patient and public involvement in dementia research in the European Union: a scoping review. BMC Geriatr. 2019;19(1):1–20. https://doi.org/10.1186/s12877-019-1217-9 .

Schilling I, Gerhardus A. Methods for involving older people in health research—a review of the literature. Int J Environ Res Public Health. 2017;14(12):1476. https://doi.org/10.3390/ijerph14121476 .

Andersson N. Participatory research—A modernizing science for primary health care. J Gen Fam Med. 2018;19(5):154–9. https://doi.org/10.1002/jgf2.187 .

Di Lorito C, Godfrey M, Dunlop M, et al. Adding to the knowledge on patient and public involvement: reflections from an experience of co-research with carers of people with dementia. Health Expect. 2020;23(3):691–706. https://doi.org/10.1111/hex.13049 .

Islam S, Small N. An annotated and critical glossary of the terminology of inclusion in healthcare and health research. Res Involve Engage. 2020;6(1):1–9. https://doi.org/10.1186/s40900-020-00186-6 .

Rose D. Patient and public involvement in health research: Ethical imperative and/or radical challenge? J Health Psychol. 2014;19(1):149–58. https://doi.org/10.1177/1359105313500249 .

NIHR. Briefing notes for researchers: public involvement in NHS, health and social care research. https://www.nihr.ac.uk/documents/briefing-notes-for-researchers-public-involvement-in-nhs-health-and-social-care-research/27371#Involvement . Accessed 12 Jan 2023.

Critical Appraisal Skills Programme. CASP Qualitative Checklist. 2018. https://casp-uk.net/images/checklist/documents/CASP-Qualitative-Studies-Checklist/CASP-Qualitative-Checklist-2018_fillable_form.pdf . Accessed 12 Jan 2023.

Staniszewska S, Brett J, Simera I, et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. Res Involv Engagem. 2017;3:13. https://doi.org/10.1186/s40900-017-0062-2 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Shippee ND, Domecq Garces JP, Prutsky Lopez GJ, et al. Patient and service user engagement in research: a systematic review and synthesized framework. Health Expect. 2015;18(5):1151–66. https://doi.org/10.1111/hex.12090 .

Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008;8(1):1–10. https://doi.org/10.1186/1471-2288-8-45 .

Saldana JM. The coding manual for qualitative researchers. London: SAGE Publications; 2015.

Google Scholar  

Page MJ, McKenzie JE, Bossuyt PM, The PRISMA, et al. statement: an updated guideline for reporting systematic reviews. BMJ. 2020;2021:372. https://doi.org/10.1136/bmj.n71 .

Swarbrick C, Doors O, et al. Developing the co-researcher involvement and engagement in dementia model (COINED): A co-operative inquiry. In: Keady J, Hydén L-C, Johnson A, et al., editors. Social research methods in dementia studies: Inclusion and innovation. New York, Routledge: Taylor & Francis Group; 2018. p. 8–19.

Clarke CL, Wilkinson H, Watson J, et al. A seat around the table: participatory data analysis with people living with dementia. Qual Health Res. 2018;28(9):1421–33. https://doi.org/10.1177/1049732318774768 .

Flavin T, Sinclair C. Reflections on involving people living with dementia in research in the Australian context. Australas J Ageing. 2019;38(Suppl 2):6–8. https://doi.org/10.1111/ajag.12596 .

Giebel C, Roe B, Hodgson A, et al. Effective public involvement in the HoST-D programme for dementia home care support: From proposal and design to methods of data collection (innovative practice). Dementia (London). 2019;18(7–8):3173–86. https://doi.org/10.1177/1471301216687698 .

Goeman DP, Corlis M, Swaffer K, et al. Partnering with people with dementia and their care partners, aged care service experts, policymakers and academics: a co-design process. Australas J Ageing. 2019;38(Suppl 2):53–8. https://doi.org/10.1111/ajag.12635 .

Gregory S, Bunnik EM, Callado AB, et al. Involving research participants in a pan-European research initiative: the EPAD participant panel experience. Res Involv Engagem. 2020;6:62. https://doi.org/10.1186/s40900-020-00236-z .

Hanson E, Magnusson L, Arvidsson H, et al. Working together with persons with early stage dementia and their family members to design a user-friendly technology-based support service. Dementia. 2007;6(3):411–34. https://doi.org/10.1177/1471301207081572 .

Hassan L, Swarbrick C, Sanders C, et al. Tea, talk and technology: patient and public involvement to improve connected health “wearables” research in dementia. Res Involv Engagem. 2017;3:12. https://doi.org/10.1186/s40900-017-0063-1 .

Mann J, Hung L. Co-research with people living with dementia for change. Action Research. 2019;17(4):573–90. https://doi.org/10.1177/1476750318787005 .

Poland F, Charlesworth G, Leung P, et al. Embedding patient and public involvement: managing tacit and explicit expectations. Health Expect. 2019;22(6):1231–9. https://doi.org/10.1111/hex.12952 .

Stevenson M, Taylor BJ. Involving individuals with dementia as co-researchers in analysis of findings from a qualitative study. Dementia (London). 2019;18(2):701–12. https://doi.org/10.1177/1471301217690904 .

Tanner D. Co-research with older people with dementia: experience and reflections. J Ment Health. 2012;21(3):296–306. https://doi.org/10.3109/09638237.2011.651658 .

Baur V, Abma T. “The Taste Buddies”: participation and empowerment in a residential home for older people. Ageing Soc. 2012;32(6):1055–78. https://doi.org/10.1017/S0144686X11000766 .

Beukema L, Valkenburg B. Demand-driven elderly care in the Netherlands. Action Research. 2007;5(2):161–80. https://doi.org/10.1177/1476750307077316 .

Brown LJE, Dickinson T, Smith S, et al. Openness, inclusion and transparency in the practice of public involvement in research: a reflective exercise to develop best practice recommendations. Health Expect. 2018;21(2):441–7. https://doi.org/10.1111/hex.12609 .

Froggatt K, Goodman C, Morbey H, et al. Public involvement in research within care homes: benefits and challenges in the APPROACH study. Health Expect. 2016;19(6):1336–45. https://doi.org/10.1111/hex.12431 .

Shura R, Siders RA, Dannefer D. Culture change in long-term care: participatory action research and the role of the resident. Gerontologist. 2011;51(2):212–25. https://doi.org/10.1093/geront/gnq099 .

Cook T. Participatory research: Its meaning and messiness. Beleidsonderzoek Online. 2020;3:1–21. https://doi.org/10.5553/BO/221335502021000003001 .

Haak M, Ivanoff S, Barenfeld E, et al. Research as an essentiality beyond one’s own competence: an interview study on frail older people’s view of research. Res Involv Engage. 2021;7(1):1–8. https://doi.org/10.1186/s40900-021-00333-7 .

Chamberlain SA, Gruneir A, Keefe JM, et al. Evolving partnerships: engagement methods in an established health services research team. Res Involve Engage. 2021;7(1):1–11. https://doi.org/10.1186/s40900-021-00314-w .

Bélisle-Pipon J-C, Rouleau G, Birko S. Early-career researchers’ views on ethical dimensions of patient engagement in research. BMC Med Ethics. 2018;19(1):1–10. https://doi.org/10.1186/s12910-018-0260-y .

Ives J, Damery S, Redwod S. PPI, paradoxes and Plato: who’s sailing the ship? J Med Ethics. 2013;39(3):181–5. https://doi.org/10.1136/medethics-2011-100150 .

De Graaff M, Stoopendaal A, Leistikow I. Transforming clients into experts-by-experience: a pilot in client participation in Dutch long-term elderly care homes inspectorate supervision. Health Policy. 2019;123(3):275–80. https://doi.org/10.1016/j.healthpol.2018.11.006 .

Cook T. Where Participatory Approaches Meet Pragmatism in Funded (Health) Research: The Challenge of Finding Meaningful Spaces. Forum Qualitative Sozialforschung Forum: Qualitative Social Research. 2012;13(1). https://doi.org/10.17169/fqs-13.1.1783 .

Paylor J, McKevitt C. The possibilities and limits of “co-producing” research. Front Sociol. 2019;4:23. https://doi.org/10.3389/fsoc.2019.00023 .

Gove D, Diaz-Ponce A, Georges J, et al. Alzheimer Europe’s position on involving people with dementia in research through PPI (patient and public involvement). Aging Ment Health. 2018;22(6):723–9. https://doi.org/10.1080/13607863.2017.1317334 .

Abelson J, Li K, Wilson G, et al. Supporting quality public and patient engagement in health system organizations: development and usability testing of the Public and Patient Engagement Evaluation Tool. Health Expect. 2016;19(4):817–27. https://doi.org/10.1111/hex.12378 .

Download references

Acknowledgements

We thank Jan W. Schoones, information specialist Directorate of Research Policy (formerly: Walaeus Library, Leiden University Medical Centre, Leiden, the Netherlands), for helping with the search.

This systematic review received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Author information

Authors and affiliations.

Department of Public Health and Primary Care, Leiden University Medical Center, P.O. Box 9600, 2300, RC, Leiden, the Netherlands

Janneke M. Groothuijse, Lisa S. van Tol, C. C. M. (Toos) Hoeksel-van Leeuwen, Monique A. A. Caljouw & Wilco P. Achterberg

University Network for the Care Sector Zuid-Holland, Leiden University Medical Center, Leiden, The Netherlands

Department of Medical Humanities, University Medical Center, Utrecht, The Netherlands

Johannes J. M. van Delden

You can also search for this author in PubMed   Google Scholar

Contributions

Concept: HD, LT, MC, WA; design of study protocol: JG, LT, MC; data collection and extraction: JG, LT, MC, THL; data analysis and interpretation: all authors; writing, editing and final approval of the manuscript: all authors.

Corresponding author

Correspondence to Monique A. A. Caljouw .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Ethics approval and consent to participate

Not applicable.

Consent for publication

Competing interest.

There are no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Groothuijse, J.M., van Tol, L.S., Leeuwen, C.C.M.(.Hv. et al. Active involvement in scientific research of persons living with dementia and long-term care users: a systematic review of existing methods with a specific focus on good practices, facilitators and barriers of involvement. BMC Geriatr 24 , 324 (2024). https://doi.org/10.1186/s12877-024-04877-7

Download citation

Received : 03 February 2023

Accepted : 05 March 2024

Published : 09 April 2024

DOI : https://doi.org/10.1186/s12877-024-04877-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Involvement in research
• Older residents

BMC Geriatrics

ISSN: 1471-2318

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Open access
• Published: 12 April 2024

Perceptions and experiences of undergraduate medical students regarding social accountability: a cross-sectional study at a Subsaharan African medical school

• Lorraine Oriokot 1 ,
• Ian Guyton Munabi 2 ,
• Sarah Kiguli 1 &
• Aloysius Gonzaga Mubuuke 3  

BMC Medical Education volume  24 , Article number:  409 ( 2024 ) Cite this article

Metrics details

Medical schools are called to be socially accountable by medical education and healthcare system stakeholders. Social accountability is a feature of excellent medical education. Medical students are essential to the development of socially accountable medical schools. Therefore, understanding the perceptions and experiences of medical students regarding social accountability is critical for efforts to improve social accountability practices and outcomes.

This cross-sectional online questionnaire-based survey used Google Forms and involved medical students in their fourth and fifth years of study at the Makerere University School of Medicine. The survey was conducted between September 2022 and October 2023. We used a study questionnaire and a validated toolkit designed by students as part of The Training for Health Equity Collaborative to gauge a school’s progress towards social accountability in medical schools to collect data on demographics, perceptions and experiences and evaluate social accountability.

Out of 555 eligible medical students, 426 responded to the online questionnaire. The response rate was 77%. The mean age of the students was 25.24 ± 4.4 years. Almost three fourths of the students were male (71.3%), and slightly less than two thirds were in their fourth year of study (65%). Almost half of the students (48.1%%) evaluated the school as doing well with regard to social accountability. The evaluation items referring to community-based research and positive impact on the community had the highest mean scores. Only 6 (3.6%) students who reported hearing of social accountability had a clear understanding of social accountability. Students receiving career guidance in secondary school was associated with evaluating social accountability in the medical school as strong (p-0.003).

Conclusions

Medical students evaluated the medical school favorably forsocial accountability despite lacking a clear understanding of social accountability. Receiving career guidance in secondary school was significantly associated with a positive evaluation of social accountability.

Peer Review reports

Medical students are essential to the development of a socially accountable medical school [ 1 , 2 ]. Social accountability for medical schools is defined as the obligation of the medical school to direct education, research, and service activities toward the most important needs of the community served by the medical school and its graduates [ 3 ]. While medical students are expected to learn about social accountability and become socially accountable practitioners, little has been documented about their perceptions and experiences of social accountability.

Medical students have distinct and unique experiences during training, which may influence their future practice and choices following graduation [ 4 , 5 ]. It is important to understand these experiences to improve the teaching and learning of social accountability. Previous studies have shown that medical students have a limited understanding of social accountability [ 5 , 6 , 7 ]. The partnership pentagram identifies five key stakeholders for social accountability in health professions education including policymakers, health administrators, health professionals, academics and community members [ 2 , 8 ]. A survey of Deans in Korea reported that interactions between partners had the greatest influence on social accountability in medical education [ 9 ]. Medical students are the future of the healthcare system; therefore, understanding their perceptions and experiences could be a starting point for improving their learning and adoption of social accountability [ 1 , 4 ].

The Students’ Toolkit on Social Accountability of Medical Schools was developed through a collaboration between the International Federation of Medical Students Associations (IFMSA) and the Training for Health Equity Network (THEnet) [ 1 , 4 , 10 ]. The toolkit aims to provide a brief introduction to social accountability for medical students and empower them to make a difference in their schools in the area of social accountability. The toolkit is an evaluation tool for students to assess the progress made by their medical school in terms of social accountability and to create action plans to improve social accountability at the medical school [ 1 ]. This kit has been applied in various settings most often in high-income countries [ 4 , 11 ].

The perceptions and experiences of medical students regarding social accountability are unique because of contextual differences that may influence their learning and adoption of social accountability [ 12 , 13 ]. There is a dearth of published literature about these perceptions and experiences, particularly from sub-Saharan Africa. The purpose of this study is to determine the perceptions and experiences of medical students regarding social accountability at the Makerere University School of Medicine.

Study design and questionnaire

We conducted a cross-sectional online questionnaire-based survey between September 2022 and October 2023 using Google Forms. This study collected responses from medical students from the Makerere University School of Medicine. The validated Students’ Toolkit on Social Accountability in Medical Schools, which has been applied widely to study perceptions of social accountability, was used for this study, with some modifications [ 1 ]. The final study questionnaire had three sections: demographic information, perceptions, and experiences related to social accountability; and the evaluation section from the toolkit. The questionnaire was pretested on eight undergraduate medical students in their fourth or fifth year of study. The link to the questionnaire on Google Forms was sent to each student’s email. Each link was unique to each email address to avoid reuse and double enrollment.

Study setting

We conducted this study at the Makerere University School of Medicine in Uganda. Makerere University is a government-owned university, and the Makerere University School of Medicine is the oldest medical school in East Africa. The Bachelor of Medicine and Bachelor of Surgery (MBCHB) program spans five years. In their third and fourth year of study, medical students spend time in the community for Community Based Education, Research and Service (COBERS). Before COBERS, students undergo preparatory sessions at the university and are assigned site and university supervisors to follow their progress. During COBERS, students identify a community problem, develop and implement interventions. Thereafter, the students evaluate the impact of the intervention [ 14 ].

Characteristics of participants

We invited all students in their fourth or fifth year of medical school to participate in the study through trained research assistants. A total of 555 students were eligible to participate in the study. We selected fourth- and fifth-year students because they were more likely to have had experiences related to social accountability in medical school. We identified twelve research assistants from among the fourth- and fifth-year students and trained them in the study procedures. The research assistants obtained written informed consent from the participants and registered their email addresses. We sent links to the electronic survey to the registered email addresses. The research assistants provided weekly phone call or in-person reminders to students who had not responded to the survey. We collected responses over two academic years therefore two sets of fourth year students were enrolled. However, the fifth years in the second academic year of the study had already been enrolled as fourth year students.

Statistical analysis

We collected the data using Google Forms. We exported the data to Microsoft Excel, checked for completeness and missing data. We then cleaned the data. The data were analyzed with R statistical software. The descriptive statistics are reported. The total score for the students’ toolkit 12 social accountability evaluation items was computed, and the means and standard deviations for the individual items are presented. The total scores were categorized according to the key provided in the student toolkit. The categories included 0–8, weak foundation; 9–17, some evidence of social accountability; 18–26, doing well (identify areas of improvement); and 27–36, strong foundation. For the regression analysis to determine the factors associated, the categories defined in the toolkit were classified in two groups (1) Limited social accountability (weak and some social accountability) and (2) Strong social accountability (looking for areas of improvement and strong social accountability). The chi-square test and Fisher’s exact test were used. A p value < 0.05 indicated statistical significance.

There were 555 eligible medical students at the Makerere University School of Medicine during the study period. This study involved 426 medical students therefore the response rate was 77%. The mean age of the students was 25.24 ± 4.4 years. Most of the participants were in their fourth year of study, as the link was available over two academic years. The characteristics of the study participants are summarized in Table  1 .

Student perceptions and experiences of social accountability

When asked if they had ever heard about social accountability, 165 (38.73%) students responded ‘yes’. To build on this previous question, participants were asked ‘What do you understand by the term social accountability?’. Only 6 (3.6%) of those who reported having heard about social accountability clearly defined the term capturing the active and multidimensional nature of social accountability. Of the 165 (39%) participants who reported hearing about social accountability before, 91 (55%) encountered the term in personal reading. The average time spent in community-based education research and service was 6.8 weeks, and most (40.1%) of the students reported feeling moderately prepared for their last COBERS experience. (Fig.  1 )

Level of preparedness for last community-based education research and service experience

Students’ evaluation of social accountability at medical schools

Using the Students’ Toolkit for Social Accountability in Medical Schools, 48.1% of the medical students evaluated the medical school as doing well in social accountability, with a total score between 18 and 26. In contrast, 1.4% of the students felt that the medical school had a weak foundation for social accountability. (Fig.  2 )

Evaluation categories of social accountability at the medical school

Of the twelve items used to assess social accountability, seven items (1, 2, 3, 7, 8, 11 and 12) were related to perceptions, while five items (4, 5, 6, 9 and 10) were related to experiences. Item 10 (Does your school have community-based research? ) 2.57  ±  0.62 and Item 12 (Does your school have a positive impact on the community? ) 2.434 ± 0.67 had the highest mean scores. Item 4 (Do you learn about other cultures? ) 1.37  ±  0.93 and Item 8 (Do your teachers reflect the sociodemographic characteristics of the reference population? ) 1.60 ± 0.92 had the lowest mean scores. (Table  2 )

The only factor that was significantly associated with evaluation of social accountability in medical school as strong was receiving career guidance in secondary school (p 0.003). (Table  3 )

We evaluated the perceptions and experiences of medical students at the Makerere University School of Medicine regarding social accountability. Medical students are central to the adoption and practice of social accountability efforts in medical schools. The perceptions and experiences of the medical students in this study reflect exposure to the concept and practice of social accountability. Learning social accountability requires deliberate and meaningful efforts in which medical students are considered partners. Our study findings suggest that medical students at the Makerere University School of Medicine have varied perceptions and experiences of social accountability; most of the students evaluated the medical school favorably, and receiving career guidance in secondary school was associated with a positive evaluation of social accountability. Most medical students felt that the medical school was doing well in social accountability and needed to look at areas of weakness and ways to advocate for improving social accountability. The recommended actions for students to take concerning each scoring category of social accountability in the student toolkit reflect the continuous quality improvement approach that is vital for social accountability [ 1 ]. The highest mean scores for items in the students’ toolkit for social accountability in medical schools were for community-based research and for the positive impact of the medical school on the community. The least favorable assessment items include learning about other cultures and teachers reflecting the reference population. Medical students come from varied backgrounds and have different experiences during medical education; therefore, it is not surprising that the perceptions, experiences and evaluations of the medical school were diverse. While career guidance in secondary school was associated with a good evaluation of social accountability at the medical school, we cannot confirm a causal relationship between the two. This relationship may be an area for further evaluation. This study helps us to characterise medical students’ views regarding social accountability and the factors that may influence these views. These findings provide a starting point for improving student experiences of social accountability in medical education.

The high percentage of students who gave a good evaluation of social accountability at medical school reflects efforts by the medical school to achieve this goal. These efforts include community-based education, research and services and adopting a competency-based medical education curriculum to better meet the community’s needs [ 15 , 16 ]. Our findings are comparable to those of a study conducted at a Saudi Arabian government-funded medical school where most students felt that the medical school was performing well in terms of social accountability [ 4 ].

There have been gains in pursuing social accountability goals, and more needs to be done to enable students to understand the concept and demonstrate its values. Our findings concur with a previous study that showed a poor understanding of social accountability among stakeholders. This previous study also provided evidence of social accountability in medical school activities [ 6 ]. Similarly, a qualitative study in the United Kingdom also showed that students did not understand the concept of social accountability or feel that it has implications for their medical education or future practice [ 5 ].

The major strength of this study lies in the relatively large sample size, as we tried to enroll all eligible participants. The limitation of this study is that it was conducted at one study site, which may limit the generalizability of the findings. However, the presence of similar findings in other settings supports the generalizability of our findings. Recall bias was minimized by enrolling students who were still in medical school.

Medical students at the Makerere University School of Medicine have varied perceptions and experiences of social accountability. However, the students lack a clear understanding of social accountability. Most students felt moderately prepared for COBERS and evaluated the medical school positively for social accountability. Regarding individual items in the student toolkit, the presence of community-based research and the school having a positive impact on the community had higher mean evaluation scores. Receiving career guidance in secondary school was associated with a positive evaluation of the medical school.

Recommendations

The medical school should provide students with more opportunities to learn about social accountability and routinely evaluate the perceptions and experiences of medical students regarding social accountability. In addition, the medical school should effectively prepare students for Community-Based Education Research and Service.

Data availability

The datasets used and/or analyzed during the current study are available from the corresponding author upon reasonable request.

Abbreviations

Community Based Education Research and Service

Dijk S, Pálsdóttir B, Ross SJ, Bhiri M, Ramalho R, Glasner J. Students’ Toolkit on Social Accountability in Medical Schools. Amst IFMSA. 2017.

Rourke J. Social accountability in theory and practice. Ann Fam Med. 2006;4(Suppl 1):S45–8. discussion S58.

Article   Google Scholar  

Boelen C, Heck JE, World Health Organization. Defining and measuring the social accountability of medical schools. World Health Organization; 1995.

Masud N, Alenezi S, Alsayari O, Alghaith D, Alshehri R, Albarrak D et al. Social accountability in Medical Education: students’ perspective. In Frontiers; 2022. p. 868245.

McCrea ML, Murdoch-Eaton D. How do undergraduate medical students perceive social accountability? Med Teach. 2014;36(10):867–75.

Galukande M, Nakasujja N, Sewankambo NK. Social accountability: a survey of perceptions and evidence of its expression at a sub saharan African university. BMC Med Educ. 2012;12:1–6.

Clithero-Eridon A, Ross A, Albright D. Conceptualising social accountability as an attribute of medical education. Afr J Prim Health Care Fam Med. 2020;12(1):1–8.

Boelen C. Towards unity for health: challenges and opportunities for partnership in health development [Internet]. Geneva: World Health Organization; 2000 [cited 2023 Jul 1] p. 84 p. Report No.: WHO/EIP/OSD/2000.9. https://apps.who.int/iris/handle/10665/66566 .

Lee ST, Yang EB. Factors affecting social accountability of medical schools in the Korean context: exploratory factor and multiple regression analyses. Med Educ Online. 2022;27(1):2054049.

Swed S, Alibrahim H, Bohsas H, Nasif MN, Abouainain Y, Jabban YKE, et al. Assessing social accountability perspectives among Syrian medical students: a cross-sectional study. BMC Med Educ. 2023;23(1):980.

Coşkun Ö, Timurçin U, Kıyak YS, Budakoğlu Iİ. Validation of IFMSA social accountability assessment tool: exploratory and confirmatory factor analysis. BMC Med Educ. 2023;23(1):138.

Mohammadi M, Bagheri M, Jafari P, Bazrafkan L. Motivating medical students for social accountability in medical schools. J Adv Med Educ Prof. 2020;8(2):90.

Google Scholar  

Boelen C, Woollard R. Social accountability: the extra leap to excellence for educational institutions. Med Teach. 2011;33(8):614–9.

Kizito S, Baingana R, Mugagga K, Akera P, Sewankambo NK. Influence of community-based education on undergraduate health professions students’ decision to work in underserved areas in Uganda. BMC Res Notes. 2017;10(1):726.

McKenzie-White J, Mubuuke AG, Westergaard S, Munabi IG, Bollinger RC, Opoka R, et al. Evaluation of a competency based medical curriculum in a sub-saharan African medical school. BMC Med Educ. 2022;22(1):724.

Nakanjako D, Omaswa F. A century of health professions’ education, training, academic research and service at Makerere University, Kampala, Uganda. Afr Health Sci. 2022;22(2):i–v.

Download references

Acknowledgements

Not applicable.

The research reported in this publication was supported by the Fogarty International Center of the National Institutes of Health, U.S. Department of State’s Office of the U.S. Global AIDS Coordinator and Health Diplomacy (S/GAC) and the President’s Emergency Plan for AIDS Relief (PEPFAR) under Award Number 1R25TW011213. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Author information

Authors and affiliations.

Department of Pediatrics and Child Health, Makerere University College of Health Sciences, Kampala, Uganda

Lorraine Oriokot & Sarah Kiguli

Department of Anatomy, Makerere University College of Health Sciences, Kampala, Uganda

Ian Guyton Munabi

Department of Radiology, Makerere University College of Health Sciences, Kampala, Uganda

Aloysius Gonzaga Mubuuke

You can also search for this author in PubMed   Google Scholar

Contributions

LO conception of the study, acquisition and analysis of data, drafting the paper and writing the final version. IGM, SK and AGM- refined the study concept, reviewed the results, drafted the paper and approved its final version.

Corresponding author

Correspondence to Lorraine Oriokot .

Ethics declarations

Ethical approval and consent to participate.

Ethical approval for the study was obtained from the Makerere University School of Medicine Institutional Review Board (Mak-SOMREC-2021-77) and the Uganda National Council for Science and Technology. Written informed consent was obtained from the study participants.

Consent for publication

Competing interests.

AGM is a member of the BMC Medical Education editorial board. The other authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Oriokot, L., Munabi, I.G., Kiguli, S. et al. Perceptions and experiences of undergraduate medical students regarding social accountability: a cross-sectional study at a Subsaharan African medical school. BMC Med Educ 24 , 409 (2024). https://doi.org/10.1186/s12909-024-05412-3

Download citation

Received : 15 December 2023

Accepted : 10 April 2024

Published : 12 April 2024

DOI : https://doi.org/10.1186/s12909-024-05412-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Social accountability
• Medical school
• Medical education
• Medical students
• Community-based education research and service

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• S&P Dow Jones Indices
• S&P Global Market Intelligence
• S&P Global Mobility
• S&P Global Commodity Insights
• S&P Global Ratings
• S&P Global Sustainable1
• *BRC Ratings – S&P Global
• India-CRISIL
• Taiwan Ratings
• S&P Capital IQ Pro
• Ratings360®

Entity & Insights

CreditWeek: Why Is EBITDA Addback Analysis Critical For Investors?

Table Of Contents: S&P Global Ratings Corporate And Infrastructure Finance Criteria

Credit FAQ: The Rise of Repeat Defaulters

Instant Insights: Key Takeaways From Our Research

Credit FAQ: Will The TMX Expansion Project Move The Needle On U.S. Refiners' Credit Quality?

• 11 Apr, 2024 | 13:25
• Sector Corporates Credit Markets Research

What We're Watching

What we think and why, what could change.

(Editor's Note: CreditWeek is a weekly research offering from S&P Global Ratings, providing actionable and forward-looking insights on emerging credit risks and exploring the questions that matter to markets today. Subscribe to receive a new edition every Thursday at: https://www.linkedin.com/newsletters/creditweek-7115686044951273472/ )

Our latest analysis of company-adjusted EBITDA continues to show that U.S. speculative-grade corporate issuers typically present unreliable and overly optimistic earnings, debt, and leverage forecasts in their marketing materials when launching deals. In the absence of a standardized definition of EBITDA, addbacks can create challenges for investors by inflating EBITDA, which can understate debt leverage and expand the flexibility embedded in debt documents to add debt or make restricted payments. S&P Global Ratings bases its ratings on its independent projections of a company's expected earnings.

Our sixth annual analysis of EBITDA addbacks reinforces our view that companies' EBITDA adjustments at the inception of a deal generally don't provide a realistic view of their future earnings. We continue to find a positive correlation between the magnitude of addbacks at deal inception and the severity of projection misses.

This is crucial for investors to understand because addbacks represent a significant percentage of management-adjusted EBITDA at deal inception, at approximately 30% on a median basis over the study's 2015-2020 span.

In our analysis, we assessed the relationship between the magnitude of addbacks (which are adjustments to income and cash flow, for operating costs characterized as nonrecurring, unusual, or discretionary, or projected earnings boosts from cost savings or synergies) and projected earnings performance (as measured in terms of projected leverage misses over the two years following deal inception).

The results of our studies over the last six years show that company-adjusted EBITDA continues to show a vast difference between management projections and what companies actually report.

During the first year following deal inception, historically 95% of the companies failed to meet their forecasts, debt was understated across the six-year sample by 2%, and management missed leverage projections on a median basis by 2.3x. In the second year, over 50% missed earnings projections by more than 33%, the median miss in projected debt rose to 13%, and actual leverage exceeded management projections by 2.7x.

Notwithstanding a slight reduction in the magnitude of misses in our latest study, aggressive addbacks correlate to persistently unreliable projections. As a result, management-adjusted EBITDA is generally an unrealistic view of future profitability .

Data for the latest cohort of borrowers showed a slight improvement in managements' earnings projections. Whether this is an anomaly or an early sign of a fundamental shift toward more realizable projections at deal inception is up for debate--as is the reason for the notable improvement. While we've seen some variation around the baseline in our previous studies, this iteration featured the first notable decline in miss percentage, with 2020 transactions showing marginal improvement in accuracy.

The most recent cohort in our six-year study represents a small portion of our aggregate dataset and may not necessarily represent a change in management mindset.

The latest cohort's origination during the height of the COVID-19 pandemic, when investors were in "risk-off" mode, is likely a potential driver. It's also possible that companies, whether consciously or otherwise, were inclined to make more accurate projections at that time because lenders were especially eager to spot projections that were (or appeared) unattainable.

Either way, our findings clearly warn of the potential perils of accepting management forecasts at face value. Because the absence of a standardized definition of EBITDA can create challenges for investors directly comparing transactions, unrealistic projections may weaken credit quality--and elevate the potential for significant future event risk.

We base our ratings on our independent projections of a company's expected earnings, a tempered view of its capacity and appetite for debt repayment , and our analysis and assessment of business and financial factors (such as management and board governance). Marketing leverage and deal-specific language around addbacks don't determine our view of credit risk (other than in assessing headroom regarding financial-maintenance covenants). There's often a vast difference between management projections and our own projections.

The addition of the 2021 cohort of companies in our study next year will be telling, given the dramatic swing in credit market and economic sentiment and robust issuance volumes in that year--especially since market dynamics may have already evolved.

We will also be watching to see whether the heightened competition between syndicated and private leveraged credit markets since 2020 leads to looser underwriting standards and affords borrowers more flexibility to pad EBITDA addbacks.

It's possible that the persistently large underperformance could lead to a tempering of addbacks. Nonetheless, this appears unlikely, given limited lender negotiating power to dictate document terms (in syndicated credit markets, at least).

There is a greater chance that we will continue to see sizable EBITDA addbacks, persistent underperformance of actual EBITDA relative to projections, and variation around the baseline depending on the underlying credit market and economic conditions in any given year.

Writers: Joe Maguire and Molly Mintz

This report does not constitute a rating action.

No content (including ratings, credit-related analyses and data, valuations, model, software, or other application or output therefrom) or any part thereof (Content) may be modified, reverse engineered, reproduced, or distributed in any form by any means, or stored in a database or retrieval system, without the prior written permission of Standard & Poor’s Financial Services LLC or its affiliates (collectively, S&P). The Content shall not be used for any unlawful or unauthorized purposes. S&P and any third-party providers, as well as their directors, officers, shareholders, employees, or agents (collectively S&P Parties) do not guarantee the accuracy, completeness, timeliness, or availability of the Content. S&P Parties are not responsible for any errors or omissions (negligent or otherwise), regardless of the cause, for the results obtained from the use of the Content, or for the security or maintenance of any data input by the user. The Content is provided on an “as is” basis. S&P PARTIES DISCLAIM ANY AND ALL EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, ANY WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE OR USE, FREEDOM FROM BUGS, SOFTWARE ERRORS OR DEFECTS, THAT THE CONTENT’S FUNCTIONING WILL BE UNINTERRUPTED, OR THAT THE CONTENT WILL OPERATE WITH ANY SOFTWARE OR HARDWARE CONFIGURATION. In no event shall S&P Parties be liable to any party for any direct, indirect, incidental, exemplary, compensatory, punitive, special or consequential damages, costs, expenses, legal fees, or losses (including, without limitation, lost income or lost profits and opportunity costs or losses caused by negligence) in connection with any use of the Content even if advised of the possibility of such damages.

Credit-related and other analyses, including ratings, and statements in the Content are statements of opinion as of the date they are expressed and not statements of fact. S&P’s opinions, analyses, and rating acknowledgment decisions (described below) are not recommendations to purchase, hold, or sell any securities or to make any investment decisions, and do not address the suitability of any security. S&P assumes no obligation to update the Content following publication in any form or format. The Content should not be relied on and is not a substitute for the skill, judgment, and experience of the user, its management, employees, advisors, and/or clients when making investment and other business decisions. S&P does not act as a fiduciary or an investment advisor except where registered as such. While S&P has obtained information from sources it believes to be reliable, S&P does not perform an audit and undertakes no duty of due diligence or independent verification of any information it receives. Rating-related publications may be published for a variety of reasons that are not necessarily dependent on action by rating committees, including, but not limited to, the publication of a periodic update on a credit rating and related analyses.

To the extent that regulatory authorities allow a rating agency to acknowledge in one jurisdiction a rating issued in another jurisdiction for certain regulatory purposes, S&P reserves the right to assign, withdraw, or suspend such acknowledgement at any time and in its sole discretion. S&P Parties disclaim any duty whatsoever arising out of the assignment, withdrawal, or suspension of an acknowledgment as well as any liability for any damage alleged to have been suffered on account thereof.

S&P keeps certain activities of its business units separate from each other in order to preserve the independence and objectivity of their respective activities. As a result, certain business units of S&P may have information that is not available to other S&P business units. S&P has established policies and procedures to maintain the confidentiality of certain nonpublic information received in connection with each analytical process.

S&P may receive compensation for its ratings and certain analyses, normally from issuers or underwriters of securities or from obligors. S&P reserves the right to disseminate its opinions and analyses. S&P's public ratings and analyses are made available on its Web sites, www.spglobal.com/ratings (free of charge), and www.ratingsdirect.com (subscription), and may be distributed through other means, including via S&P publications and third-party redistributors. Additional information about our ratings fees is available at www.spglobal.com/usratingsfees .

• Corporates Credit Markets Research

Create a free account to unlock the article.

Gain access to exclusive research, events and more.

Already have an account?     Sign in