critical thinking skills research papers

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Review Article
• Open access
• Published: 11 January 2023

The effectiveness of collaborative problem solving in promoting students’ critical thinking: A meta-analysis based on empirical literature

• Enwei Xu   ORCID: orcid.org/0000-0001-6424-8169 1 ,
• Wei Wang 1 &
• Qingxia Wang 1  

Humanities and Social Sciences Communications volume  10 , Article number:  16 ( 2023 ) Cite this article

14k Accesses

13 Citations

3 Altmetric

Metrics details

• Science, technology and society

Collaborative problem-solving has been widely embraced in the classroom instruction of critical thinking, which is regarded as the core of curriculum reform based on key competencies in the field of education as well as a key competence for learners in the 21st century. However, the effectiveness of collaborative problem-solving in promoting students’ critical thinking remains uncertain. This current research presents the major findings of a meta-analysis of 36 pieces of the literature revealed in worldwide educational periodicals during the 21st century to identify the effectiveness of collaborative problem-solving in promoting students’ critical thinking and to determine, based on evidence, whether and to what extent collaborative problem solving can result in a rise or decrease in critical thinking. The findings show that (1) collaborative problem solving is an effective teaching approach to foster students’ critical thinking, with a significant overall effect size (ES = 0.82, z  = 12.78, P  < 0.01, 95% CI [0.69, 0.95]); (2) in respect to the dimensions of critical thinking, collaborative problem solving can significantly and successfully enhance students’ attitudinal tendencies (ES = 1.17, z  = 7.62, P  < 0.01, 95% CI[0.87, 1.47]); nevertheless, it falls short in terms of improving students’ cognitive skills, having only an upper-middle impact (ES = 0.70, z  = 11.55, P  < 0.01, 95% CI[0.58, 0.82]); and (3) the teaching type (chi 2  = 7.20, P  < 0.05), intervention duration (chi 2  = 12.18, P  < 0.01), subject area (chi 2  = 13.36, P  < 0.05), group size (chi 2  = 8.77, P  < 0.05), and learning scaffold (chi 2  = 9.03, P  < 0.01) all have an impact on critical thinking, and they can be viewed as important moderating factors that affect how critical thinking develops. On the basis of these results, recommendations are made for further study and instruction to better support students’ critical thinking in the context of collaborative problem-solving.

Similar content being viewed by others

Fostering twenty-first century skills among primary school students through math project-based learning

A meta-analysis to gauge the impact of pedagogies employed in mixed-ability high school biology classrooms

A guide to critical thinking: implications for dental education

Introduction.

Although critical thinking has a long history in research, the concept of critical thinking, which is regarded as an essential competence for learners in the 21st century, has recently attracted more attention from researchers and teaching practitioners (National Research Council, 2012 ). Critical thinking should be the core of curriculum reform based on key competencies in the field of education (Peng and Deng, 2017 ) because students with critical thinking can not only understand the meaning of knowledge but also effectively solve practical problems in real life even after knowledge is forgotten (Kek and Huijser, 2011 ). The definition of critical thinking is not universal (Ennis, 1989 ; Castle, 2009 ; Niu et al., 2013 ). In general, the definition of critical thinking is a self-aware and self-regulated thought process (Facione, 1990 ; Niu et al., 2013 ). It refers to the cognitive skills needed to interpret, analyze, synthesize, reason, and evaluate information as well as the attitudinal tendency to apply these abilities (Halpern, 2001 ). The view that critical thinking can be taught and learned through curriculum teaching has been widely supported by many researchers (e.g., Kuncel, 2011 ; Leng and Lu, 2020 ), leading to educators’ efforts to foster it among students. In the field of teaching practice, there are three types of courses for teaching critical thinking (Ennis, 1989 ). The first is an independent curriculum in which critical thinking is taught and cultivated without involving the knowledge of specific disciplines; the second is an integrated curriculum in which critical thinking is integrated into the teaching of other disciplines as a clear teaching goal; and the third is a mixed curriculum in which critical thinking is taught in parallel to the teaching of other disciplines for mixed teaching training. Furthermore, numerous measuring tools have been developed by researchers and educators to measure critical thinking in the context of teaching practice. These include standardized measurement tools, such as WGCTA, CCTST, CCTT, and CCTDI, which have been verified by repeated experiments and are considered effective and reliable by international scholars (Facione and Facione, 1992 ). In short, descriptions of critical thinking, including its two dimensions of attitudinal tendency and cognitive skills, different types of teaching courses, and standardized measurement tools provide a complex normative framework for understanding, teaching, and evaluating critical thinking.

Cultivating critical thinking in curriculum teaching can start with a problem, and one of the most popular critical thinking instructional approaches is problem-based learning (Liu et al., 2020 ). Duch et al. ( 2001 ) noted that problem-based learning in group collaboration is progressive active learning, which can improve students’ critical thinking and problem-solving skills. Collaborative problem-solving is the organic integration of collaborative learning and problem-based learning, which takes learners as the center of the learning process and uses problems with poor structure in real-world situations as the starting point for the learning process (Liang et al., 2017 ). Students learn the knowledge needed to solve problems in a collaborative group, reach a consensus on problems in the field, and form solutions through social cooperation methods, such as dialogue, interpretation, questioning, debate, negotiation, and reflection, thus promoting the development of learners’ domain knowledge and critical thinking (Cindy, 2004 ; Liang et al., 2017 ).

Collaborative problem-solving has been widely used in the teaching practice of critical thinking, and several studies have attempted to conduct a systematic review and meta-analysis of the empirical literature on critical thinking from various perspectives. However, little attention has been paid to the impact of collaborative problem-solving on critical thinking. Therefore, the best approach for developing and enhancing critical thinking throughout collaborative problem-solving is to examine how to implement critical thinking instruction; however, this issue is still unexplored, which means that many teachers are incapable of better instructing critical thinking (Leng and Lu, 2020 ; Niu et al., 2013 ). For example, Huber ( 2016 ) provided the meta-analysis findings of 71 publications on gaining critical thinking over various time frames in college with the aim of determining whether critical thinking was truly teachable. These authors found that learners significantly improve their critical thinking while in college and that critical thinking differs with factors such as teaching strategies, intervention duration, subject area, and teaching type. The usefulness of collaborative problem-solving in fostering students’ critical thinking, however, was not determined by this study, nor did it reveal whether there existed significant variations among the different elements. A meta-analysis of 31 pieces of educational literature was conducted by Liu et al. ( 2020 ) to assess the impact of problem-solving on college students’ critical thinking. These authors found that problem-solving could promote the development of critical thinking among college students and proposed establishing a reasonable group structure for problem-solving in a follow-up study to improve students’ critical thinking. Additionally, previous empirical studies have reached inconclusive and even contradictory conclusions about whether and to what extent collaborative problem-solving increases or decreases critical thinking levels. As an illustration, Yang et al. ( 2008 ) carried out an experiment on the integrated curriculum teaching of college students based on a web bulletin board with the goal of fostering participants’ critical thinking in the context of collaborative problem-solving. These authors’ research revealed that through sharing, debating, examining, and reflecting on various experiences and ideas, collaborative problem-solving can considerably enhance students’ critical thinking in real-life problem situations. In contrast, collaborative problem-solving had a positive impact on learners’ interaction and could improve learning interest and motivation but could not significantly improve students’ critical thinking when compared to traditional classroom teaching, according to research by Naber and Wyatt ( 2014 ) and Sendag and Odabasi ( 2009 ) on undergraduate and high school students, respectively.

The above studies show that there is inconsistency regarding the effectiveness of collaborative problem-solving in promoting students’ critical thinking. Therefore, it is essential to conduct a thorough and trustworthy review to detect and decide whether and to what degree collaborative problem-solving can result in a rise or decrease in critical thinking. Meta-analysis is a quantitative analysis approach that is utilized to examine quantitative data from various separate studies that are all focused on the same research topic. This approach characterizes the effectiveness of its impact by averaging the effect sizes of numerous qualitative studies in an effort to reduce the uncertainty brought on by independent research and produce more conclusive findings (Lipsey and Wilson, 2001 ).

This paper used a meta-analytic approach and carried out a meta-analysis to examine the effectiveness of collaborative problem-solving in promoting students’ critical thinking in order to make a contribution to both research and practice. The following research questions were addressed by this meta-analysis:

What is the overall effect size of collaborative problem-solving in promoting students’ critical thinking and its impact on the two dimensions of critical thinking (i.e., attitudinal tendency and cognitive skills)?

How are the disparities between the study conclusions impacted by various moderating variables if the impacts of various experimental designs in the included studies are heterogeneous?

This research followed the strict procedures (e.g., database searching, identification, screening, eligibility, merging, duplicate removal, and analysis of included studies) of Cooper’s ( 2010 ) proposed meta-analysis approach for examining quantitative data from various separate studies that are all focused on the same research topic. The relevant empirical research that appeared in worldwide educational periodicals within the 21st century was subjected to this meta-analysis using Rev-Man 5.4. The consistency of the data extracted separately by two researchers was tested using Cohen’s kappa coefficient, and a publication bias test and a heterogeneity test were run on the sample data to ascertain the quality of this meta-analysis.

Data sources and search strategies

There were three stages to the data collection process for this meta-analysis, as shown in Fig. 1 , which shows the number of articles included and eliminated during the selection process based on the statement and study eligibility criteria.

This flowchart shows the number of records identified, included and excluded in the article.

First, the databases used to systematically search for relevant articles were the journal papers of the Web of Science Core Collection and the Chinese Core source journal, as well as the Chinese Social Science Citation Index (CSSCI) source journal papers included in CNKI. These databases were selected because they are credible platforms that are sources of scholarly and peer-reviewed information with advanced search tools and contain literature relevant to the subject of our topic from reliable researchers and experts. The search string with the Boolean operator used in the Web of Science was “TS = (((“critical thinking” or “ct” and “pretest” or “posttest”) or (“critical thinking” or “ct” and “control group” or “quasi experiment” or “experiment”)) and (“collaboration” or “collaborative learning” or “CSCL”) and (“problem solving” or “problem-based learning” or “PBL”))”. The research area was “Education Educational Research”, and the search period was “January 1, 2000, to December 30, 2021”. A total of 412 papers were obtained. The search string with the Boolean operator used in the CNKI was “SU = (‘critical thinking’*‘collaboration’ + ‘critical thinking’*‘collaborative learning’ + ‘critical thinking’*‘CSCL’ + ‘critical thinking’*‘problem solving’ + ‘critical thinking’*‘problem-based learning’ + ‘critical thinking’*‘PBL’ + ‘critical thinking’*‘problem oriented’) AND FT = (‘experiment’ + ‘quasi experiment’ + ‘pretest’ + ‘posttest’ + ‘empirical study’)” (translated into Chinese when searching). A total of 56 studies were found throughout the search period of “January 2000 to December 2021”. From the databases, all duplicates and retractions were eliminated before exporting the references into Endnote, a program for managing bibliographic references. In all, 466 studies were found.

Second, the studies that matched the inclusion and exclusion criteria for the meta-analysis were chosen by two researchers after they had reviewed the abstracts and titles of the gathered articles, yielding a total of 126 studies.

Third, two researchers thoroughly reviewed each included article’s whole text in accordance with the inclusion and exclusion criteria. Meanwhile, a snowball search was performed using the references and citations of the included articles to ensure complete coverage of the articles. Ultimately, 36 articles were kept.

Two researchers worked together to carry out this entire process, and a consensus rate of almost 94.7% was reached after discussion and negotiation to clarify any emerging differences.

Eligibility criteria

Since not all the retrieved studies matched the criteria for this meta-analysis, eligibility criteria for both inclusion and exclusion were developed as follows:

The publication language of the included studies was limited to English and Chinese, and the full text could be obtained. Articles that did not meet the publication language and articles not published between 2000 and 2021 were excluded.

The research design of the included studies must be empirical and quantitative studies that can assess the effect of collaborative problem-solving on the development of critical thinking. Articles that could not identify the causal mechanisms by which collaborative problem-solving affects critical thinking, such as review articles and theoretical articles, were excluded.

The research method of the included studies must feature a randomized control experiment or a quasi-experiment, or a natural experiment, which have a higher degree of internal validity with strong experimental designs and can all plausibly provide evidence that critical thinking and collaborative problem-solving are causally related. Articles with non-experimental research methods, such as purely correlational or observational studies, were excluded.

The participants of the included studies were only students in school, including K-12 students and college students. Articles in which the participants were non-school students, such as social workers or adult learners, were excluded.

The research results of the included studies must mention definite signs that may be utilized to gauge critical thinking’s impact (e.g., sample size, mean value, or standard deviation). Articles that lacked specific measurement indicators for critical thinking and could not calculate the effect size were excluded.

Data coding design

In order to perform a meta-analysis, it is necessary to collect the most important information from the articles, codify that information’s properties, and convert descriptive data into quantitative data. Therefore, this study designed a data coding template (see Table 1 ). Ultimately, 16 coding fields were retained.

The designed data-coding template consisted of three pieces of information. Basic information about the papers was included in the descriptive information: the publishing year, author, serial number, and title of the paper.

The variable information for the experimental design had three variables: the independent variable (instruction method), the dependent variable (critical thinking), and the moderating variable (learning stage, teaching type, intervention duration, learning scaffold, group size, measuring tool, and subject area). Depending on the topic of this study, the intervention strategy, as the independent variable, was coded into collaborative and non-collaborative problem-solving. The dependent variable, critical thinking, was coded as a cognitive skill and an attitudinal tendency. And seven moderating variables were created by grouping and combining the experimental design variables discovered within the 36 studies (see Table 1 ), where learning stages were encoded as higher education, high school, middle school, and primary school or lower; teaching types were encoded as mixed courses, integrated courses, and independent courses; intervention durations were encoded as 0–1 weeks, 1–4 weeks, 4–12 weeks, and more than 12 weeks; group sizes were encoded as 2–3 persons, 4–6 persons, 7–10 persons, and more than 10 persons; learning scaffolds were encoded as teacher-supported learning scaffold, technique-supported learning scaffold, and resource-supported learning scaffold; measuring tools were encoded as standardized measurement tools (e.g., WGCTA, CCTT, CCTST, and CCTDI) and self-adapting measurement tools (e.g., modified or made by researchers); and subject areas were encoded according to the specific subjects used in the 36 included studies.

The data information contained three metrics for measuring critical thinking: sample size, average value, and standard deviation. It is vital to remember that studies with various experimental designs frequently adopt various formulas to determine the effect size. And this paper used Morris’ proposed standardized mean difference (SMD) calculation formula ( 2008 , p. 369; see Supplementary Table S3 ).

Procedure for extracting and coding data

According to the data coding template (see Table 1 ), the 36 papers’ information was retrieved by two researchers, who then entered them into Excel (see Supplementary Table S1 ). The results of each study were extracted separately in the data extraction procedure if an article contained numerous studies on critical thinking, or if a study assessed different critical thinking dimensions. For instance, Tiwari et al. ( 2010 ) used four time points, which were viewed as numerous different studies, to examine the outcomes of critical thinking, and Chen ( 2013 ) included the two outcome variables of attitudinal tendency and cognitive skills, which were regarded as two studies. After discussion and negotiation during data extraction, the two researchers’ consistency test coefficients were roughly 93.27%. Supplementary Table S2 details the key characteristics of the 36 included articles with 79 effect quantities, including descriptive information (e.g., the publishing year, author, serial number, and title of the paper), variable information (e.g., independent variables, dependent variables, and moderating variables), and data information (e.g., mean values, standard deviations, and sample size). Following that, testing for publication bias and heterogeneity was done on the sample data using the Rev-Man 5.4 software, and then the test results were used to conduct a meta-analysis.

Publication bias test

When the sample of studies included in a meta-analysis does not accurately reflect the general status of research on the relevant subject, publication bias is said to be exhibited in this research. The reliability and accuracy of the meta-analysis may be impacted by publication bias. Due to this, the meta-analysis needs to check the sample data for publication bias (Stewart et al., 2006 ). A popular method to check for publication bias is the funnel plot; and it is unlikely that there will be publishing bias when the data are equally dispersed on either side of the average effect size and targeted within the higher region. The data are equally dispersed within the higher portion of the efficient zone, consistent with the funnel plot connected with this analysis (see Fig. 2 ), indicating that publication bias is unlikely in this situation.

This funnel plot shows the result of publication bias of 79 effect quantities across 36 studies.

Heterogeneity test

To select the appropriate effect models for the meta-analysis, one might use the results of a heterogeneity test on the data effect sizes. In a meta-analysis, it is common practice to gauge the degree of data heterogeneity using the I 2 value, and I 2  ≥ 50% is typically understood to denote medium-high heterogeneity, which calls for the adoption of a random effect model; if not, a fixed effect model ought to be applied (Lipsey and Wilson, 2001 ). The findings of the heterogeneity test in this paper (see Table 2 ) revealed that I 2 was 86% and displayed significant heterogeneity ( P  < 0.01). To ensure accuracy and reliability, the overall effect size ought to be calculated utilizing the random effect model.

The analysis of the overall effect size

This meta-analysis utilized a random effect model to examine 79 effect quantities from 36 studies after eliminating heterogeneity. In accordance with Cohen’s criterion (Cohen, 1992 ), it is abundantly clear from the analysis results, which are shown in the forest plot of the overall effect (see Fig. 3 ), that the cumulative impact size of cooperative problem-solving is 0.82, which is statistically significant ( z  = 12.78, P  < 0.01, 95% CI [0.69, 0.95]), and can encourage learners to practice critical thinking.

This forest plot shows the analysis result of the overall effect size across 36 studies.

In addition, this study examined two distinct dimensions of critical thinking to better understand the precise contributions that collaborative problem-solving makes to the growth of critical thinking. The findings (see Table 3 ) indicate that collaborative problem-solving improves cognitive skills (ES = 0.70) and attitudinal tendency (ES = 1.17), with significant intergroup differences (chi 2  = 7.95, P  < 0.01). Although collaborative problem-solving improves both dimensions of critical thinking, it is essential to point out that the improvements in students’ attitudinal tendency are much more pronounced and have a significant comprehensive effect (ES = 1.17, z  = 7.62, P  < 0.01, 95% CI [0.87, 1.47]), whereas gains in learners’ cognitive skill are slightly improved and are just above average. (ES = 0.70, z  = 11.55, P  < 0.01, 95% CI [0.58, 0.82]).

The analysis of moderator effect size

The whole forest plot’s 79 effect quantities underwent a two-tailed test, which revealed significant heterogeneity ( I 2  = 86%, z  = 12.78, P  < 0.01), indicating differences between various effect sizes that may have been influenced by moderating factors other than sampling error. Therefore, exploring possible moderating factors that might produce considerable heterogeneity was done using subgroup analysis, such as the learning stage, learning scaffold, teaching type, group size, duration of the intervention, measuring tool, and the subject area included in the 36 experimental designs, in order to further explore the key factors that influence critical thinking. The findings (see Table 4 ) indicate that various moderating factors have advantageous effects on critical thinking. In this situation, the subject area (chi 2  = 13.36, P  < 0.05), group size (chi 2  = 8.77, P  < 0.05), intervention duration (chi 2  = 12.18, P  < 0.01), learning scaffold (chi 2  = 9.03, P  < 0.01), and teaching type (chi 2  = 7.20, P  < 0.05) are all significant moderators that can be applied to support the cultivation of critical thinking. However, since the learning stage and the measuring tools did not significantly differ among intergroup (chi 2  = 3.15, P  = 0.21 > 0.05, and chi 2  = 0.08, P  = 0.78 > 0.05), we are unable to explain why these two factors are crucial in supporting the cultivation of critical thinking in the context of collaborative problem-solving. These are the precise outcomes, as follows:

Various learning stages influenced critical thinking positively, without significant intergroup differences (chi 2  = 3.15, P  = 0.21 > 0.05). High school was first on the list of effect sizes (ES = 1.36, P  < 0.01), then higher education (ES = 0.78, P  < 0.01), and middle school (ES = 0.73, P  < 0.01). These results show that, despite the learning stage’s beneficial influence on cultivating learners’ critical thinking, we are unable to explain why it is essential for cultivating critical thinking in the context of collaborative problem-solving.

Different teaching types had varying degrees of positive impact on critical thinking, with significant intergroup differences (chi 2  = 7.20, P  < 0.05). The effect size was ranked as follows: mixed courses (ES = 1.34, P  < 0.01), integrated courses (ES = 0.81, P  < 0.01), and independent courses (ES = 0.27, P  < 0.01). These results indicate that the most effective approach to cultivate critical thinking utilizing collaborative problem solving is through the teaching type of mixed courses.

Various intervention durations significantly improved critical thinking, and there were significant intergroup differences (chi 2  = 12.18, P  < 0.01). The effect sizes related to this variable showed a tendency to increase with longer intervention durations. The improvement in critical thinking reached a significant level (ES = 0.85, P  < 0.01) after more than 12 weeks of training. These findings indicate that the intervention duration and critical thinking’s impact are positively correlated, with a longer intervention duration having a greater effect.

Different learning scaffolds influenced critical thinking positively, with significant intergroup differences (chi 2  = 9.03, P  < 0.01). The resource-supported learning scaffold (ES = 0.69, P  < 0.01) acquired a medium-to-higher level of impact, the technique-supported learning scaffold (ES = 0.63, P  < 0.01) also attained a medium-to-higher level of impact, and the teacher-supported learning scaffold (ES = 0.92, P  < 0.01) displayed a high level of significant impact. These results show that the learning scaffold with teacher support has the greatest impact on cultivating critical thinking.

Various group sizes influenced critical thinking positively, and the intergroup differences were statistically significant (chi 2  = 8.77, P  < 0.05). Critical thinking showed a general declining trend with increasing group size. The overall effect size of 2–3 people in this situation was the biggest (ES = 0.99, P  < 0.01), and when the group size was greater than 7 people, the improvement in critical thinking was at the lower-middle level (ES < 0.5, P  < 0.01). These results show that the impact on critical thinking is positively connected with group size, and as group size grows, so does the overall impact.

Various measuring tools influenced critical thinking positively, with significant intergroup differences (chi 2  = 0.08, P  = 0.78 > 0.05). In this situation, the self-adapting measurement tools obtained an upper-medium level of effect (ES = 0.78), whereas the complete effect size of the standardized measurement tools was the largest, achieving a significant level of effect (ES = 0.84, P  < 0.01). These results show that, despite the beneficial influence of the measuring tool on cultivating critical thinking, we are unable to explain why it is crucial in fostering the growth of critical thinking by utilizing the approach of collaborative problem-solving.

Different subject areas had a greater impact on critical thinking, and the intergroup differences were statistically significant (chi 2  = 13.36, P  < 0.05). Mathematics had the greatest overall impact, achieving a significant level of effect (ES = 1.68, P  < 0.01), followed by science (ES = 1.25, P  < 0.01) and medical science (ES = 0.87, P  < 0.01), both of which also achieved a significant level of effect. Programming technology was the least effective (ES = 0.39, P  < 0.01), only having a medium-low degree of effect compared to education (ES = 0.72, P  < 0.01) and other fields (such as language, art, and social sciences) (ES = 0.58, P  < 0.01). These results suggest that scientific fields (e.g., mathematics, science) may be the most effective subject areas for cultivating critical thinking utilizing the approach of collaborative problem-solving.

The effectiveness of collaborative problem solving with regard to teaching critical thinking

According to this meta-analysis, using collaborative problem-solving as an intervention strategy in critical thinking teaching has a considerable amount of impact on cultivating learners’ critical thinking as a whole and has a favorable promotional effect on the two dimensions of critical thinking. According to certain studies, collaborative problem solving, the most frequently used critical thinking teaching strategy in curriculum instruction can considerably enhance students’ critical thinking (e.g., Liang et al., 2017 ; Liu et al., 2020 ; Cindy, 2004 ). This meta-analysis provides convergent data support for the above research views. Thus, the findings of this meta-analysis not only effectively address the first research query regarding the overall effect of cultivating critical thinking and its impact on the two dimensions of critical thinking (i.e., attitudinal tendency and cognitive skills) utilizing the approach of collaborative problem-solving, but also enhance our confidence in cultivating critical thinking by using collaborative problem-solving intervention approach in the context of classroom teaching.

Furthermore, the associated improvements in attitudinal tendency are much stronger, but the corresponding improvements in cognitive skill are only marginally better. According to certain studies, cognitive skill differs from the attitudinal tendency in classroom instruction; the cultivation and development of the former as a key ability is a process of gradual accumulation, while the latter as an attitude is affected by the context of the teaching situation (e.g., a novel and exciting teaching approach, challenging and rewarding tasks) (Halpern, 2001 ; Wei and Hong, 2022 ). Collaborative problem-solving as a teaching approach is exciting and interesting, as well as rewarding and challenging; because it takes the learners as the focus and examines problems with poor structure in real situations, and it can inspire students to fully realize their potential for problem-solving, which will significantly improve their attitudinal tendency toward solving problems (Liu et al., 2020 ). Similar to how collaborative problem-solving influences attitudinal tendency, attitudinal tendency impacts cognitive skill when attempting to solve a problem (Liu et al., 2020 ; Zhang et al., 2022 ), and stronger attitudinal tendencies are associated with improved learning achievement and cognitive ability in students (Sison, 2008 ; Zhang et al., 2022 ). It can be seen that the two specific dimensions of critical thinking as well as critical thinking as a whole are affected by collaborative problem-solving, and this study illuminates the nuanced links between cognitive skills and attitudinal tendencies with regard to these two dimensions of critical thinking. To fully develop students’ capacity for critical thinking, future empirical research should pay closer attention to cognitive skills.

The moderating effects of collaborative problem solving with regard to teaching critical thinking

In order to further explore the key factors that influence critical thinking, exploring possible moderating effects that might produce considerable heterogeneity was done using subgroup analysis. The findings show that the moderating factors, such as the teaching type, learning stage, group size, learning scaffold, duration of the intervention, measuring tool, and the subject area included in the 36 experimental designs, could all support the cultivation of collaborative problem-solving in critical thinking. Among them, the effect size differences between the learning stage and measuring tool are not significant, which does not explain why these two factors are crucial in supporting the cultivation of critical thinking utilizing the approach of collaborative problem-solving.

In terms of the learning stage, various learning stages influenced critical thinking positively without significant intergroup differences, indicating that we are unable to explain why it is crucial in fostering the growth of critical thinking.

Although high education accounts for 70.89% of all empirical studies performed by researchers, high school may be the appropriate learning stage to foster students’ critical thinking by utilizing the approach of collaborative problem-solving since it has the largest overall effect size. This phenomenon may be related to student’s cognitive development, which needs to be further studied in follow-up research.

With regard to teaching type, mixed course teaching may be the best teaching method to cultivate students’ critical thinking. Relevant studies have shown that in the actual teaching process if students are trained in thinking methods alone, the methods they learn are isolated and divorced from subject knowledge, which is not conducive to their transfer of thinking methods; therefore, if students’ thinking is trained only in subject teaching without systematic method training, it is challenging to apply to real-world circumstances (Ruggiero, 2012 ; Hu and Liu, 2015 ). Teaching critical thinking as mixed course teaching in parallel to other subject teachings can achieve the best effect on learners’ critical thinking, and explicit critical thinking instruction is more effective than less explicit critical thinking instruction (Bensley and Spero, 2014 ).

In terms of the intervention duration, with longer intervention times, the overall effect size shows an upward tendency. Thus, the intervention duration and critical thinking’s impact are positively correlated. Critical thinking, as a key competency for students in the 21st century, is difficult to get a meaningful improvement in a brief intervention duration. Instead, it could be developed over a lengthy period of time through consistent teaching and the progressive accumulation of knowledge (Halpern, 2001 ; Hu and Liu, 2015 ). Therefore, future empirical studies ought to take these restrictions into account throughout a longer period of critical thinking instruction.

With regard to group size, a group size of 2–3 persons has the highest effect size, and the comprehensive effect size decreases with increasing group size in general. This outcome is in line with some research findings; as an example, a group composed of two to four members is most appropriate for collaborative learning (Schellens and Valcke, 2006 ). However, the meta-analysis results also indicate that once the group size exceeds 7 people, small groups cannot produce better interaction and performance than large groups. This may be because the learning scaffolds of technique support, resource support, and teacher support improve the frequency and effectiveness of interaction among group members, and a collaborative group with more members may increase the diversity of views, which is helpful to cultivate critical thinking utilizing the approach of collaborative problem-solving.

With regard to the learning scaffold, the three different kinds of learning scaffolds can all enhance critical thinking. Among them, the teacher-supported learning scaffold has the largest overall effect size, demonstrating the interdependence of effective learning scaffolds and collaborative problem-solving. This outcome is in line with some research findings; as an example, a successful strategy is to encourage learners to collaborate, come up with solutions, and develop critical thinking skills by using learning scaffolds (Reiser, 2004 ; Xu et al., 2022 ); learning scaffolds can lower task complexity and unpleasant feelings while also enticing students to engage in learning activities (Wood et al., 2006 ); learning scaffolds are designed to assist students in using learning approaches more successfully to adapt the collaborative problem-solving process, and the teacher-supported learning scaffolds have the greatest influence on critical thinking in this process because they are more targeted, informative, and timely (Xu et al., 2022 ).

With respect to the measuring tool, despite the fact that standardized measurement tools (such as the WGCTA, CCTT, and CCTST) have been acknowledged as trustworthy and effective by worldwide experts, only 54.43% of the research included in this meta-analysis adopted them for assessment, and the results indicated no intergroup differences. These results suggest that not all teaching circumstances are appropriate for measuring critical thinking using standardized measurement tools. “The measuring tools for measuring thinking ability have limits in assessing learners in educational situations and should be adapted appropriately to accurately assess the changes in learners’ critical thinking.”, according to Simpson and Courtney ( 2002 , p. 91). As a result, in order to more fully and precisely gauge how learners’ critical thinking has evolved, we must properly modify standardized measuring tools based on collaborative problem-solving learning contexts.

With regard to the subject area, the comprehensive effect size of science departments (e.g., mathematics, science, medical science) is larger than that of language arts and social sciences. Some recent international education reforms have noted that critical thinking is a basic part of scientific literacy. Students with scientific literacy can prove the rationality of their judgment according to accurate evidence and reasonable standards when they face challenges or poorly structured problems (Kyndt et al., 2013 ), which makes critical thinking crucial for developing scientific understanding and applying this understanding to practical problem solving for problems related to science, technology, and society (Yore et al., 2007 ).

Suggestions for critical thinking teaching

Other than those stated in the discussion above, the following suggestions are offered for critical thinking instruction utilizing the approach of collaborative problem-solving.

First, teachers should put a special emphasis on the two core elements, which are collaboration and problem-solving, to design real problems based on collaborative situations. This meta-analysis provides evidence to support the view that collaborative problem-solving has a strong synergistic effect on promoting students’ critical thinking. Asking questions about real situations and allowing learners to take part in critical discussions on real problems during class instruction are key ways to teach critical thinking rather than simply reading speculative articles without practice (Mulnix, 2012 ). Furthermore, the improvement of students’ critical thinking is realized through cognitive conflict with other learners in the problem situation (Yang et al., 2008 ). Consequently, it is essential for teachers to put a special emphasis on the two core elements, which are collaboration and problem-solving, and design real problems and encourage students to discuss, negotiate, and argue based on collaborative problem-solving situations.

Second, teachers should design and implement mixed courses to cultivate learners’ critical thinking, utilizing the approach of collaborative problem-solving. Critical thinking can be taught through curriculum instruction (Kuncel, 2011 ; Leng and Lu, 2020 ), with the goal of cultivating learners’ critical thinking for flexible transfer and application in real problem-solving situations. This meta-analysis shows that mixed course teaching has a highly substantial impact on the cultivation and promotion of learners’ critical thinking. Therefore, teachers should design and implement mixed course teaching with real collaborative problem-solving situations in combination with the knowledge content of specific disciplines in conventional teaching, teach methods and strategies of critical thinking based on poorly structured problems to help students master critical thinking, and provide practical activities in which students can interact with each other to develop knowledge construction and critical thinking utilizing the approach of collaborative problem-solving.

Third, teachers should be more trained in critical thinking, particularly preservice teachers, and they also should be conscious of the ways in which teachers’ support for learning scaffolds can promote critical thinking. The learning scaffold supported by teachers had the greatest impact on learners’ critical thinking, in addition to being more directive, targeted, and timely (Wood et al., 2006 ). Critical thinking can only be effectively taught when teachers recognize the significance of critical thinking for students’ growth and use the proper approaches while designing instructional activities (Forawi, 2016 ). Therefore, with the intention of enabling teachers to create learning scaffolds to cultivate learners’ critical thinking utilizing the approach of collaborative problem solving, it is essential to concentrate on the teacher-supported learning scaffolds and enhance the instruction for teaching critical thinking to teachers, especially preservice teachers.

Implications and limitations

There are certain limitations in this meta-analysis, but future research can correct them. First, the search languages were restricted to English and Chinese, so it is possible that pertinent studies that were written in other languages were overlooked, resulting in an inadequate number of articles for review. Second, these data provided by the included studies are partially missing, such as whether teachers were trained in the theory and practice of critical thinking, the average age and gender of learners, and the differences in critical thinking among learners of various ages and genders. Third, as is typical for review articles, more studies were released while this meta-analysis was being done; therefore, it had a time limit. With the development of relevant research, future studies focusing on these issues are highly relevant and needed.

Conclusions

The subject of the magnitude of collaborative problem-solving’s impact on fostering students’ critical thinking, which received scant attention from other studies, was successfully addressed by this study. The question of the effectiveness of collaborative problem-solving in promoting students’ critical thinking was addressed in this study, which addressed a topic that had gotten little attention in earlier research. The following conclusions can be made:

Regarding the results obtained, collaborative problem solving is an effective teaching approach to foster learners’ critical thinking, with a significant overall effect size (ES = 0.82, z  = 12.78, P  < 0.01, 95% CI [0.69, 0.95]). With respect to the dimensions of critical thinking, collaborative problem-solving can significantly and effectively improve students’ attitudinal tendency, and the comprehensive effect is significant (ES = 1.17, z  = 7.62, P  < 0.01, 95% CI [0.87, 1.47]); nevertheless, it falls short in terms of improving students’ cognitive skills, having only an upper-middle impact (ES = 0.70, z  = 11.55, P  < 0.01, 95% CI [0.58, 0.82]).

As demonstrated by both the results and the discussion, there are varying degrees of beneficial effects on students’ critical thinking from all seven moderating factors, which were found across 36 studies. In this context, the teaching type (chi 2  = 7.20, P  < 0.05), intervention duration (chi 2  = 12.18, P  < 0.01), subject area (chi 2  = 13.36, P  < 0.05), group size (chi 2  = 8.77, P  < 0.05), and learning scaffold (chi 2  = 9.03, P  < 0.01) all have a positive impact on critical thinking, and they can be viewed as important moderating factors that affect how critical thinking develops. Since the learning stage (chi 2  = 3.15, P  = 0.21 > 0.05) and measuring tools (chi 2  = 0.08, P  = 0.78 > 0.05) did not demonstrate any significant intergroup differences, we are unable to explain why these two factors are crucial in supporting the cultivation of critical thinking in the context of collaborative problem-solving.

Data availability

All data generated or analyzed during this study are included within the article and its supplementary information files, and the supplementary information files are available in the Dataverse repository: https://doi.org/10.7910/DVN/IPFJO6 .

Bensley DA, Spero RA (2014) Improving critical thinking skills and meta-cognitive monitoring through direct infusion. Think Skills Creat 12:55–68. https://doi.org/10.1016/j.tsc.2014.02.001

Article   Google Scholar  

Castle A (2009) Defining and assessing critical thinking skills for student radiographers. Radiography 15(1):70–76. https://doi.org/10.1016/j.radi.2007.10.007

Chen XD (2013) An empirical study on the influence of PBL teaching model on critical thinking ability of non-English majors. J PLA Foreign Lang College 36 (04):68–72

Google Scholar  

Cohen A (1992) Antecedents of organizational commitment across occupational groups: a meta-analysis. J Organ Behav. https://doi.org/10.1002/job.4030130602

Cooper H (2010) Research synthesis and meta-analysis: a step-by-step approach, 4th edn. Sage, London, England

Cindy HS (2004) Problem-based learning: what and how do students learn? Educ Psychol Rev 51(1):31–39

Duch BJ, Gron SD, Allen DE (2001) The power of problem-based learning: a practical “how to” for teaching undergraduate courses in any discipline. Stylus Educ Sci 2:190–198

Ennis RH (1989) Critical thinking and subject specificity: clarification and needed research. Educ Res 18(3):4–10. https://doi.org/10.3102/0013189x018003004

Facione PA (1990) Critical thinking: a statement of expert consensus for purposes of educational assessment and instruction. Research findings and recommendations. Eric document reproduction service. https://eric.ed.gov/?id=ed315423

Facione PA, Facione NC (1992) The California Critical Thinking Dispositions Inventory (CCTDI) and the CCTDI test manual. California Academic Press, Millbrae, CA

Forawi SA (2016) Standard-based science education and critical thinking. Think Skills Creat 20:52–62. https://doi.org/10.1016/j.tsc.2016.02.005

Halpern DF (2001) Assessing the effectiveness of critical thinking instruction. J Gen Educ 50(4):270–286. https://doi.org/10.2307/27797889

Hu WP, Liu J (2015) Cultivation of pupils’ thinking ability: a five-year follow-up study. Psychol Behav Res 13(05):648–654. https://doi.org/10.3969/j.issn.1672-0628.2015.05.010

Huber K (2016) Does college teach critical thinking? A meta-analysis. Rev Educ Res 86(2):431–468. https://doi.org/10.3102/0034654315605917

Kek MYCA, Huijser H (2011) The power of problem-based learning in developing critical thinking skills: preparing students for tomorrow’s digital futures in today’s classrooms. High Educ Res Dev 30(3):329–341. https://doi.org/10.1080/07294360.2010.501074

Kuncel NR (2011) Measurement and meaning of critical thinking (Research report for the NRC 21st Century Skills Workshop). National Research Council, Washington, DC

Kyndt E, Raes E, Lismont B, Timmers F, Cascallar E, Dochy F (2013) A meta-analysis of the effects of face-to-face cooperative learning. Do recent studies falsify or verify earlier findings? Educ Res Rev 10(2):133–149. https://doi.org/10.1016/j.edurev.2013.02.002

Leng J, Lu XX (2020) Is critical thinking really teachable?—A meta-analysis based on 79 experimental or quasi experimental studies. Open Educ Res 26(06):110–118. https://doi.org/10.13966/j.cnki.kfjyyj.2020.06.011

Liang YZ, Zhu K, Zhao CL (2017) An empirical study on the depth of interaction promoted by collaborative problem solving learning activities. J E-educ Res 38(10):87–92. https://doi.org/10.13811/j.cnki.eer.2017.10.014

Lipsey M, Wilson D (2001) Practical meta-analysis. International Educational and Professional, London, pp. 92–160

Liu Z, Wu W, Jiang Q (2020) A study on the influence of problem based learning on college students’ critical thinking-based on a meta-analysis of 31 studies. Explor High Educ 03:43–49

Morris SB (2008) Estimating effect sizes from pretest-posttest-control group designs. Organ Res Methods 11(2):364–386. https://doi.org/10.1177/1094428106291059

Article   ADS   Google Scholar  

Mulnix JW (2012) Thinking critically about critical thinking. Educ Philos Theory 44(5):464–479. https://doi.org/10.1111/j.1469-5812.2010.00673.x

Naber J, Wyatt TH (2014) The effect of reflective writing interventions on the critical thinking skills and dispositions of baccalaureate nursing students. Nurse Educ Today 34(1):67–72. https://doi.org/10.1016/j.nedt.2013.04.002

National Research Council (2012) Education for life and work: developing transferable knowledge and skills in the 21st century. The National Academies Press, Washington, DC

Niu L, Behar HLS, Garvan CW (2013) Do instructional interventions influence college students’ critical thinking skills? A meta-analysis. Educ Res Rev 9(12):114–128. https://doi.org/10.1016/j.edurev.2012.12.002

Peng ZM, Deng L (2017) Towards the core of education reform: cultivating critical thinking skills as the core of skills in the 21st century. Res Educ Dev 24:57–63. https://doi.org/10.14121/j.cnki.1008-3855.2017.24.011

Reiser BJ (2004) Scaffolding complex learning: the mechanisms of structuring and problematizing student work. J Learn Sci 13(3):273–304. https://doi.org/10.1207/s15327809jls1303_2

Ruggiero VR (2012) The art of thinking: a guide to critical and creative thought, 4th edn. Harper Collins College Publishers, New York

Schellens T, Valcke M (2006) Fostering knowledge construction in university students through asynchronous discussion groups. Comput Educ 46(4):349–370. https://doi.org/10.1016/j.compedu.2004.07.010

Sendag S, Odabasi HF (2009) Effects of an online problem based learning course on content knowledge acquisition and critical thinking skills. Comput Educ 53(1):132–141. https://doi.org/10.1016/j.compedu.2009.01.008

Sison R (2008) Investigating Pair Programming in a Software Engineering Course in an Asian Setting. 2008 15th Asia-Pacific Software Engineering Conference, pp. 325–331. https://doi.org/10.1109/APSEC.2008.61

Simpson E, Courtney M (2002) Critical thinking in nursing education: literature review. Mary Courtney 8(2):89–98

Stewart L, Tierney J, Burdett S (2006) Do systematic reviews based on individual patient data offer a means of circumventing biases associated with trial publications? Publication bias in meta-analysis. John Wiley and Sons Inc, New York, pp. 261–286

Tiwari A, Lai P, So M, Yuen K (2010) A comparison of the effects of problem-based learning and lecturing on the development of students’ critical thinking. Med Educ 40(6):547–554. https://doi.org/10.1111/j.1365-2929.2006.02481.x

Wood D, Bruner JS, Ross G (2006) The role of tutoring in problem solving. J Child Psychol Psychiatry 17(2):89–100. https://doi.org/10.1111/j.1469-7610.1976.tb00381.x

Wei T, Hong S (2022) The meaning and realization of teachable critical thinking. Educ Theory Practice 10:51–57

Xu EW, Wang W, Wang QX (2022) A meta-analysis of the effectiveness of programming teaching in promoting K-12 students’ computational thinking. Educ Inf Technol. https://doi.org/10.1007/s10639-022-11445-2

Yang YC, Newby T, Bill R (2008) Facilitating interactions through structured web-based bulletin boards: a quasi-experimental study on promoting learners’ critical thinking skills. Comput Educ 50(4):1572–1585. https://doi.org/10.1016/j.compedu.2007.04.006

Yore LD, Pimm D, Tuan HL (2007) The literacy component of mathematical and scientific literacy. Int J Sci Math Educ 5(4):559–589. https://doi.org/10.1007/s10763-007-9089-4

Zhang T, Zhang S, Gao QQ, Wang JH (2022) Research on the development of learners’ critical thinking in online peer review. Audio Visual Educ Res 6:53–60. https://doi.org/10.13811/j.cnki.eer.2022.06.08

Download references

Acknowledgements

This research was supported by the graduate scientific research and innovation project of Xinjiang Uygur Autonomous Region named “Research on in-depth learning of high school information technology courses for the cultivation of computing thinking” (No. XJ2022G190) and the independent innovation fund project for doctoral students of the College of Educational Science of Xinjiang Normal University named “Research on project-based teaching of high school information technology courses from the perspective of discipline core literacy” (No. XJNUJKYA2003).

Author information

Authors and affiliations.

College of Educational Science, Xinjiang Normal University, 830017, Urumqi, Xinjiang, China

Enwei Xu, Wei Wang & Qingxia Wang

You can also search for this author in PubMed   Google Scholar

Corresponding authors

Correspondence to Enwei Xu or Wei Wang .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Ethical approval

This article does not contain any studies with human participants performed by any of the authors.

Informed consent

Additional information.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary tables, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Xu, E., Wang, W. & Wang, Q. The effectiveness of collaborative problem solving in promoting students’ critical thinking: A meta-analysis based on empirical literature. Humanit Soc Sci Commun 10 , 16 (2023). https://doi.org/10.1057/s41599-023-01508-1

Download citation

Received : 07 August 2022

Accepted : 04 January 2023

Published : 11 January 2023

DOI : https://doi.org/10.1057/s41599-023-01508-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Impacts of online collaborative learning on students’ intercultural communication apprehension and intercultural communicative competence.

• Hoa Thi Hoang Chau
• Hung Phu Bui
• Quynh Thi Huong Dinh

Education and Information Technologies (2024)

Exploring the effects of digital technology on deep learning: a meta-analysis

Sustainable electricity generation and farm-grid utilization from photovoltaic aquaculture: a bibliometric analysis.

• A. A. Amusa
• M. Alhassan

International Journal of Environmental Science and Technology (2024)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

• Open access
• Published: 11 September 2019

Inquiry and critical thinking skills for the next generation: from artificial intelligence back to human intelligence

• Jonathan Michael Spector   ORCID: orcid.org/0000-0002-6270-3073 1 &
• Shanshan Ma 1  

Smart Learning Environments volume  6 , Article number:  8 ( 2019 ) Cite this article

31k Accesses

52 Citations

32 Altmetric

Metrics details

Along with the increasing attention to artificial intelligence (AI), renewed emphasis or reflection on human intelligence (HI) is appearing in many places and at multiple levels. One of the foci is critical thinking. Critical thinking is one of four key 21st century skills – communication, collaboration, critical thinking and creativity. Though most people are aware of the value of critical thinking, it lacks emphasis in curricula. In this paper, we present a comprehensive definition of critical thinking that ranges from observation and inquiry to argumentation and reflection. Given a broad conception of critical thinking, a developmental approach beginning with children is suggested as a way to help develop critical thinking habits of mind. The conclusion of this analysis is that more emphasis should be placed on developing human intelligence, especially in young children and with the support of artificial intelligence. While much funding and support goes to the development of artificial intelligence, this should not happen at the expense of human intelligence. Overall, the purpose of this paper is to argue for more attention to the development of human intelligence with an emphasis on critical thinking.

Introduction

In recent decades, advancements in Artificial Intelligence (AI) have developed at an incredible rate. AI has penetrated into people’s daily life on a variety of levels such as smart homes, personalized healthcare, security systems, self-service stores, and online shopping. One notable AI achievement was when AlphaGo, a computer program, defeated the World Go Champion Mr. Lee Sedol in 2016. In the previous year, AlphaGo won in a competition against a professional Go player (Silver et al. 2016 ). As Go is one of the most challenging games, the wins of AI indicated a breakthrough. Public attention has been further drawn to AI since then, and AlphaGo continues to improve. In 2017, a new version of AlphaGo beat Ke Jie, the current world No.1 ranking Go player. Clearly AI can manage high levels of complexity.

Given many changes and multiple lines of development and implement, it is somewhat difficult to define AI to include all of the changes since the 1980s (Luckin et al. 2016 ). Many definitions incorporate two dimensions as a starting point: (a) human-like thinking, and (b) rational action (Russell and Norvig 2009 ). Basically, AI is a term used to label machines (computers) that imitate human cognitive functions such as learning and problem solving, or that manage to deal with complexity as well as human experts.

AlphaGo’s wins against human players were seen as a comparison between artificial and human intelligence. One concern is that AI has already surpassed HI; other concerns are that AI will replace humans in some settings or that AI will become uncontrollable (Epstein 2016 ; Fang et al. 2018 ). Scholars worry that AI technology in the future might trigger the singularity (Good 1966 ), a hypothesized future that the development of technology becomes uncontrollable and irreversible, resulting in unfathomable changes to human civilization (Vinge 1993 ).

The famous theoretical physicist Stephen Hawking warned that AI might end mankind, yet the technology he used to communicate involved a basic form of AI (Cellan-Jones 2014 ). This example highlights one of the basic dilemmas of AI – namely, what are the overall benefits of AI versus its potential drawbacks, and how to move forward given its rapid development? Obviously, basic or controllable AI technologies are not what people are afraid of. Spector et al. 1993 distinguished strong AI and weak AI. Strong AI involves an application that is intended to replace an activity performed previously by a competent human, while weak AI involves an application that aims to enable a less experienced human to perform at a much higher level. Other researchers categorize AI into three levels: (a) artificial narrow intelligence (Narrow AI), (b) artificial general intelligence (General AI), and (c) artificial super intelligence (Super AI) (Siau and Yang 2017 ; Zhang and Xie 2018 ). Narrow AI, sometimes called weak AI, refers to a computer that focus on a narrow task such as AlphaZero or a self-driving car. General AI, sometimes referred to as strong AI, is the simulation of human-level intelligence, which can perform more cognitive tasks as well as most humans do. Super AI is defined by Bostrom ( 1998 ) as “an intellect that is much smarter than the best human brains in practically every field, including scientific creativity, general wisdom and social skills” (p.1).

Although the consequence of singularity and its potential benefits or harm to the human race have been intensely debated, an undeniable fact is that AI is capable of undertaking recursive self-improvement. With the increasing improvement of this capability, more intelligent generations of AI will appear rapidly. On the other hand, HI has its own limits and its development requires continuous efforts and investment from generation to generation. Education is the main approach humans use to develop and improve HI. Given the extraordinary growth gap between AI and HI, eventually AI can surpass HI. However, that is no reason to neglect the development and improvement of HI. In addition, in contrast to the slow development rate of HI, the growth of funding support to AI has been rapidly increasing according to the following comparison of support for artificial and human intelligence.

The funding support for artificial and human intelligence

There are challenges in comparing artificial and human intelligence by identifying funding for both. Both terms are somewhat vague and can include a variety of aspects. Some analyses will include big data and data analytics within the sphere of artificial intelligence and others will treat them separately. Some will include early childhood developmental research within the sphere of support for HI and others treat them separately. Education is a major way of human beings to develop and improve HI. The investments in education reflect the efforts put on the development of HI, and they pale in comparison with investments in AI.

Sources also vary from governmental funding of research and development to business and industry investments in related research and development. Nonetheless, there are strong indications of increased funding support for AI in North America, Europe and Asia, especially in China. The growth in funding for AI around the world is explosive. According to ZDNet, AI funding more than doubled from 2016 to 2017 and more than tripled from 2016 to 2018. The growth in funding for AI in the last 10 years has been exponential. According to Venture Scanner, there are approximately 2500 companies that have raised $60 billion in funding from 3400 investors in 72 different countries (see https://www.slideshare.net/venturescanner/artificial-intelligence-q1-2019-report-highlights ). Areas included in the Venture Scanner analysis included virtual assistants, recommendation engines, video recognition, context-aware computing, speech recognition, natural language processing, machine learning, and more.

The above data on AI funding focuses primarily on companies making products. There is no direct counterpart in the area of HI where the emphasis is on learning and education. What can be seen, however, are trends within each area. The above data suggest exponential growth in support for AI. In contrast, according to the Urban Institute, per-student funding in the USA has been relatively flat for nearly two decades, with a few states showing modest increases and others showing none (see http://apps.urban.org/features/education-funding-trends/ ). Funding for education is complicated due to the various sources. In the USA, there are local, state and federal sources to consider. While that mixture of funding sources is complex, it is clear that federal and state spending for education in the USA experienced an increase after World War II. However, since the 1980s, federal spending for education has steadily declined, and state spending on education in most states has declined since 2010 according to a government report (see https://www.usgovernmentspending.com/education_spending ). This decline in funding reflects the decreasing emphasis on the development of HI, which is a dangerous signal.

Decreased support for education funding in the USA is not typical of what is happening in other countries, according to The Hechinger Report (see https://hechingerreport.org/rest-world-invests-education-u-s-spends-less/ ). For example, in the period of 2010 to 2014, American spending on elementary and high school education declined 3%, whereas in the same period, education spending in the 35 countries in the OECD rose by 5% with some countries experiencing very significant increases (e.g., 76% in Turkey).

Such data can be questioned in terms of how effectively funds are being spent or how poorly a country was doing prior to experiencing a significant increase. However, given the performance of American students on the Program for International Student Assessment (PISA), the relative lack of funding support in the USA is roughly related with the mediocre performance on PISA tests (see https://nces.ed.gov/surveys/pisa/pisa2015/index.asp ). Research by Darling-Hammond ( 2014 ) indicated that in order to improve learning and reduce the achievement gap, systematic government investments in high-need schools would be more effective if the focus was on capacity building, improving the knowledge and skills of educators and the quality of curriculum opportunities.

Though HI could not be simply defined by the performance on PISA test, improving HI requires systematic efforts and funding support in high-need areas as well. So, in the following section, we present a reflection on HI.

Reflection on human intelligence

Though there is a variety of definitions of HI, from the perspective of psychology, according to Sternberg ( 1999 ), intelligence is a form of developing expertise, from a novice or less experienced person to an expert or more experienced person, a student must be through multiple learning (implicit and explicit) and thinking (critical and creative) processes. In this paper, we adopted such a view and reflected on HI in the following section by discussing learning and critical thinking.

What is learning?

We begin with Gagné’s ( 1985 ) definition of learning as characterized by stable and persistent changes in what a person knows or can do. How do humans learn? Do you recall how to prove that the square root of 2 is not a rational number, something you might have learned years ago? The method is intriguing and is called an indirect proof or a reduction to absurdity – assume that the square root of 2 is a rational number and then apply truth preserving rules to arrive at a contradiction to show that the square root of 2 cannot be a rational number. We recommend this as an exercise for those readers who have never encountered that method of learning and proof. (see https://artofproblemsolving.com/wiki/index.php/Proof_by_contradiction ). Yet another interesting method of learning is called the process of elimination, sometimes accredited to Arthur Conan Doyle’s ( 1926 ) in The Adventure of the Blanched Soldier – Sherlock Holmes says to Dr. Watson that the process of elimination “starts upon the supposition that when you have eliminated all which is impossible, that whatever remains, however improbable, must be the truth ” (see https://www.dfw-sherlock.org/uploads/3/7/3/8/37380505/1926_november_the_adventure_of_the_blanched_soldier.pdf ).

The reason to mention Sherlock Holmes early in this paper is to emphasize the role that observation plays in learning. The character Sherlock Holmes was famous for his observation skills that led to his so-called method of deductive reasoning (a process of elimination), which is what logicians would classify as inductive reasoning as the conclusions of that reasoning process are primarily probabilistic rather than certain, unlike the proof of the irrationality of the square root of 2 mentioned previously.

In dealing with uncertainty, it seems necessary to make observations and gather evidence that can lead one to a likely conclusion. Is that not what reasonable people and accomplished detectives do? It is certainly what card counters do at gambling houses; they observe high and low value cards that have already been played in order to estimate the likelihood of the next card being a high or low value card. Observation is a critical process in dealing with uncertainty.

Moreover, humans typically encounter many uncertain situations in the course of life. Few people encounter situations which require resolution using a mathematical proof such as the one with which this article began. Jonassen ( 2000 , 2011 ) argued that problem solving is one of the most important and frequent activities in which people engage. Moreover, many of the more challenging problems are ill-structured in the sense that (a) there is incomplete information pertaining to the situation, or (b) the ideal resolution of the problem is unknown, or (c) how to transform a problematic situation into an acceptable situation is unclear. In short, people are confronted with uncertainty nearly every day and in many different ways. The so called key 21st century skills of communication, collaboration, critical thinking and creativity (the 4 Cs; see http://www.battelleforkids.org/networks/p21 ) are important because uncertainty is a natural and inescapable aspect of the human condition. The 4 Cs are interrelated and have been presented by Spector ( 2018 ) as interrelated capabilities involving logic and epistemology in the form of the new 3Rs – namely, re-examining, reasoning, and reflecting. Re-examining is directly linked to observation as a beginning point for inquiry. The method of elimination is one form of reasoning in which a person engages to solve challenging problems. Reflecting on how well one is doing in the life-long enterprise of solving challenging problems is a higher kind of meta-cognitive activity in which accomplished problem-solvers engage (Ericsson et al. 1993 ; Flavell 1979 ).

Based on these initial comments, a comprehensive definition of critical thinking is presented next in the form of a framework.

A framework of critical thinking

Though there is variety of definitions of critical thinking, a concise definition of critical thinking remains elusive. For delivering a direct understanding of critical thinking to readers such as parents and school teachers, in this paper, we present a comprehensive definition of critical thinking through a framework that includes many of the definitions offered by others. Critical thinking, as treated broadly herein, is a multi-dimensioned and multifaceted human capability. Critical thinking has been interpreted from three perspectives: education, psychology, and epistemology, all of which are represented in the framework that follows.

In a developmental approach to critical thinking, Spector ( 2019 ) argues that critical thinking involves a series of cumulative and related abilities, dispositions and other variables (e.g., motivation, criteria, context, knowledge). This approach proceeds from experience (e.g., observing something unusual) and then to various forms of inquiry, investigation, examination of evidence, exploration of alternatives, argumentation, testing conclusions, rethinking assumptions, and reflecting on the entire process.

Experience and engagement are ongoing throughout the process which proceeds from relatively simple experiences (e.g., direct and immediate observation) to more complex interactions (e.g., manipulation of an actual or virtual artifact and observing effects).

The developmental approach involves a variety of mental processes and non-cognitive states, which help a person’s decision making to become purposeful and goal directed. The associated critical thinking skills enable individuals to be likely to achieve a desired outcome in a challenging situation.

In the process of critical thinking, apart from experience, there are two additional cognitive capabilities essential to critical thinking – namely, metacognition and self-regulation . Many researchers (e.g., Schraw et al. 2006 ) believe that metacognition has two components: (a) awareness and understanding of one’s own thoughts, and (b) the ability to regulate one’s own cognitive processes. Some other researchers put more emphasis on the latter component. For example, Davies ( 2015 ) described metacognition as the capacity to monitor the quality of one’s thinking process, and then to make appropriate changes. However, the American Psychology Association (APA) defines metacognition as an awareness and understanding of one’s own thought with the ability to control related cognitive processes (see https://psycnet.apa.org/record/2008-15725-005 ).

Although the definition and elaboration of these two concepts deserve further exploration, they are often used interchangeably (Hofer and Sinatra 2010 ; Schunk 2008 ). Many psychologists see the two related capabilities of metacognition and self-regulation as being closely related - two sides on one coin, so to speak. Metacognition involves or emphasizes awareness, whereas self-regulation involves and emphasizes appropriate control. These two concepts taken together enable a person to create a self-regulatory mechanism, which monitors and regulates the corresponding skills (e.g., observation, inquiry, interpretation, explanation, reasoning, analysis, evaluation, synthesis, reflection, and judgement).

As to the critical thinking skills, it should be noted that there is much discussion about the generalizability and domain specificity of them, just as there is about problem-solving skills in general (Chi et al. 1982 ; Chiesi et al. 1979 ; Ennis 1989 ; Fischer 1980 ). The research supports the notion that to achieve high levels of expertise and performance, one must develop high levels of domain knowledge. As a consequence, becoming a highly effective critical thinker in a particular domain of inquiry requires significant domain knowledge. One may achieve such levels in a domain in which one has significant domain knowledge and experience but not in a different domain in which one has little domain knowledge and experience. The processes involved in developing high levels of critical thinking are somewhat generic. Therefore, it is possible to develop critical thinking in nearly any domain when the two additional capabilities of metacognition and self-regulation are coupled with motivation and engagement and supportive emotional states (Ericsson et al. 1993 ).

Consequently, the framework presented here (see Fig. 1 ) is built around three main perspectives about critical thinking (i.e., educational, psychological and epistemological) and relevant learning theories. This framework provides a visual presentation of critical thinking with four dimensions: abilities (educational perspective), dispositions (psychological perspective), levels (epistemological perspective) and time. Time is added to emphasize the dynamic nature of critical thinking in terms of a specific context and a developmental approach.

Critical thinking often begins with simple experiences such as observing a difference, encountering a puzzling question or problem, questioning someone’s statement, and then leads, in some instances to an inquiry, and then to more complex experiences such as interactions and application of higher order thinking skills (e.g., logical reasoning, questioning assumptions, considering and evaluating alternative explanations).

If the individual is not interested in what was observed, an inquiry typically does not begin. Inquiry and critical thinking require motivation along with an inquisitive disposition. The process of critical thinking requires the support of corresponding internal indispositions such as open-mindedness and truth-seeking. Consequently, a disposition to initiate an inquiry (e.g., curiosity) along with an internal inquisitive disposition (e.g., that links a mental habit to something motivating to the individual) are both required (Hitchcock 2018 ). Initiating dispositions are those that contribute to the start of inquiry and critical thinking. Internal dispositions are those that initiate and support corresponding critical thinking skills during the process. Therefore, critical thinking dispositions consist of initiating dispositions and internal dispositions. Besides these factors, critical thinking also involves motivation. Motivation and dispositions are not mutually exclusive, for example, curiosity is a disposition and also a motivation.

Critical thinking abilities and dispositions are two main components of critical thinking, which involve such interrelated cognitive constructs as interpretation, explanation, reasoning, evaluation, synthesis, reflection, judgement, metacognition and self-regulation (Dwyer et al. 2014 ; Davies 2015 ; Ennis 2018 ; Facione 1990 ; Hitchcock 2018 ; Paul and Elder 2006 ). There are also some other abilities such as communication, collaboration and creativity, which are now essential in current society (see https://en.wikipedia.org/wiki/21st_century_skills ). Those abilities along with critical thinking are called the 4Cs; they are individually monitored and regulated through metacognitive and self-regulation processes.

The abilities involved in critical thinking are categorized in Bloom’s taxonomy into higher order skills (e.g., analyzing and synthesizing) and lower level skills (e.g., remembering and applying) (Anderson and Krathwohl 2001 ; Bloom et al. 1956 ).

The thinking process can be depicted as a spiral through both lower and higher order thinking skills. It encompasses several reasoning loops. Some of them might be iterative until a desired outcome is achieved. Each loop might be a mix of higher order thinking skills and lower level thinking skills. Each loop is subject to the self-regulatory mechanism of metacognition and self-regulation.

But, due to the complexity of human thinking, a specific spiral with reasoning loops is difficult to represent. Therefore, instead of a visualized spiral with an indefinite number of reasoning loops, the developmental stages of critical thinking are presented in the diagram (Fig. 1 ).

Besides, most of the definitions of critical thinking are based on the imagination about ideal critical thinkers such as the consensus generated from the Delphi report (Facione 1990 ). However, according to Dreyfus and Dreyfus ( 1980 ), in the course of developing an expertise, students would pass through five stages. Those five stages are “absolute beginner”, “advanced beginner”, “competent performer”, “proficient performer,” and “intuitive expert performer”. Dreyfus and Dreyfus ( 1980 ) described the five stages the result of the successive transformations of four mental functions: recollection, recognition, decision making, and awareness.

In the course of developing critical thinking and expertise, individuals will pass through similar stages which are accompanied with the increasing practices and accumulation of experience. Through the intervention and experience of developing critical thinking, as a novice, tasks are decomposed into context-free features which could be recognized by students without the experience of particular situations. For further improving, students need to be able to monitor their awareness, and with a considerable experience. They can note recurrent meaningful component patterns in some contexts. Gradually, increased practices expose students to a variety of whole situations which enable the students to recognize tasks in a more holistic manner as a professional. On the other hand, with the increasing accumulation of experience, individuals are less likely to depend simply on abstract principles. The decision will turn to something intuitive and highly situational as well as analytical. Students might unconsciously apply rules, principles or abilities. A high level of awareness is absorbed. At this stage, critical thinking is turned into habits of mind and in some cases expertise. The description above presents a process of critical thinking development evolving from a novice to an expert, eventually developing critical thinking into habits of mind.

We mention the five-stage model proposed by Dreyfus and Dreyfus ( 1980 ) to categorize levels of critical thinking and emphasize the developmental nature involved in becoming a critical thinker. Correspondingly, critical thinking is categorized into 5 levels: absolute beginner (novice), advanced beginner (beginner), competent performer (competent), proficient performer (proficient), and intuitive expert (expert).

Ability level and critical thinker (critical thinking) level together represent one of the four dimensions represented in Fig. 1 .

In addition, it is noteworthy that the other two elements of critical thinking are the context and knowledge in which the inquiry is based. Contextual and domain knowledge must be taken into account with regard to critical thinking, as previously argued. Besides, as Hitchcock ( 2018 ) argued, effective critical thinking requires knowledge about and experience applying critical thinking concepts and principles as well.

Critical thinking is considered valuable across disciplines. But except few courses such as philosophy, critical thinking is reported lacking in most school education. Most of researchers and educators thus proclaim that integrating critical thinking across the curriculum (Hatcher 2013 ). For example, Ennis ( 2018 ) provided a vision about incorporating critical thinking across the curriculum in higher education. Though people are aware of the value of critical thinking, few of them practice it. Between 2012 and 2015, in Australia, the demand of critical thinking as one of the enterprise skills for early-career job increased 125% (Statista Research Department, 2016). According to a survey across 1000 adults by The Reboot Foundation 2018 , more than 80% of respondents believed that critical thinking skills are lacking in today’s youth. Respondents were deeply concerned that schools do not teach critical thinking. Besides, the investigation also found that respondents were split over when and how to teach critical thinking, clearly.

In the previous analysis of critical thinking, we presented the mechanism of critical thinking instead of a concise definition. This is because, given the various perspectives of interpreting critical thinking, it is not easy to come out with an unitary definition, but it is essential for the public to understand how critical thinking works, the elements it involves and the relationships between them, so they can achieve an explicit understanding.

In the framework, critical thinking starts from simple experience such as observing a difference, then entering the stage of inquiry, inquiry does not necessarily turn the thinking process into critical thinking unless the student enters a higher level of thinking process or reasoning loops such as re-examining, reasoning, reflection (3Rs). Being an ideal critical thinker (or an expert) requires efforts and time.

According to the framework, simple abilities such as observational skills and inquiry are indispensable to lead to critical thinking, which suggests that paying attention to those simple skills at an early stage of children can be an entry point to critical thinking. Considering the child development theory by Piaget ( 1964 ), a developmental approach spanning multiple years can be employed to help children develop critical thinking at each corresponding development stage until critical thinking becomes habits of mind.

Although we emphasized critical thinking in this paper, for the improvement of intelligence, creative thinking and critical thinking are separable, they are both essential abilities that develop expertise, eventually drive the improvement of HI at human race level.

As previously argued, there is a similar pattern among students who think critically in different domains, but students from different domains might perform differently in creativity because of different thinking styles (Haller and Courvoisier 2010 ). Plus, students have different learning styles and preferences. Personalized learning has been the most appropriate approach to address those differences. Though the way of realizing personalized learning varies along with the development of technologies. Generally, personalized learning aims at customizing learning to accommodate diverse students based on their strengths, needs, interests, preferences, and abilities.

Meanwhile, the advancement of technology including AI is revolutionizing education; students’ learning environments are shifting from technology-enhanced learning environments to smart learning environments. Although lots of potentials are unrealized yet (Spector 2016 ), the so-called smart learning environments rely more on the support of AI technology such as neural networks, learning analytics and natural language processing. Personalized learning is better supported and realized in a smart learning environment. In short, in the current era, personalized learning is to use AI to help learners perform at a higher level making adjustments based on differences of learners. This is the notion with which we conclude – the future lies in using AI to improve HI and accommodating individual differences.

The application of AI in education has been a subject for decades. There are efforts heading to such a direction though personalized learning is not technically involved in them. For example, using AI technology to stimulate critical thinking (Zhu 2015 ), applying a virtual environment for building and assessing higher order inquiry skills (Ketelhut et al. 2010 ). Developing computational thinking through robotics (Angeli and Valanides 2019 ) is another such promising application of AI to support the development of HI.

However, almost all of those efforts are limited to laboratory experiments. For accelerating the development rate of HI, we argue that more emphasis should be given to the development of HI at scale with the support of AI, especially in young children focusing on critical and creative thinking.

In this paper, we argue that more emphasis should be given to HI development. Rather than decreasing the funding of AI, the analysis of progress in artificial and human intelligence indicates that it would be reasonable to see increased emphasis placed on using various AI techniques and technologies to improve HI on a large and sustainable scale. Well, most researchers might agree that AI techniques or the situation might be not mature enough to support such a large-scale development. But it would be dangerous if HI development is overlooked. Based on research and theory drawn from psychology as well as from epistemology, the framework is intended to provide a practical guide to the progressive development of inquiry and critical thinking skills in young children as children represent the future of our fragile planet. And we suggested a sustainable development approach for developing inquiry and critical thinking (See, Spector 2019 ). Such an approach could be realized through AI and infused into HI development. Besides, a project is underway in collaboration with NetDragon to develop gamified applications to develop the relevant skills and habits of mind. A game-based assessment methodology is being developed and tested at East China Normal University that is appropriate for middle school children. The intention of the effort is to refocus some of the attention on the development of HI in young children.

Availability of data and materials

Not applicable.

Abbreviations

Artificial Intelligence

Human Intelligence

L.W. Anderson, D.R. Krathwohl, A taxonomy for learning, teaching, and assessing: A revision of bloom’s taxonomy of educational objectives (Allyn & Bacon, Boston, 2001)

Google Scholar  

Angeli, C., & Valanides, N. (2019). Developing young children’s computational thinking with educational robotics: An interaction effect between gender and scaffolding strategy. Comput. Hum. Behav. Retrieved from https://doi.org/10.1016/j.chb.2019.03.018

B.S. Bloom, M.D. Engelhart, E.J. Furst, W.H. Hill, D.R. Krathwohl, Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive Domain (David McKay Company, New York, 1956)

Bostrom, N. (1998). How long before superintelligence? Retrieved from https://nickbostrom.com/superintelligence.html

R. Cellan-Jones, Stephen hawking warns artificial intelligence could end mankind. BBC. News. 2 , 2014 (2014)

M.T.H. Chi, R. Glaser, E. Rees, in Advances in the Psychology of Human Intelligence , ed. by R. S. Sternberg. Expertise in problem solving (Erlbaum, Hillsdale, 1982), pp. 7–77

H.L. Chiesi, G.J. Spliich, J.F. Voss, Acquisition of domain-related information in relation to high and low domain knowledge. J. Verbal Learn. Verbal Behav. 18 , 257–273 (1979)

Article   Google Scholar  

L. Darling-Hammond, What can PISA tell US about US education policy? N. Engl. J. Publ. Policy. 26 (1), 4 (2014)

M. Davies, in Higher education: Handbook of theory and research . A Model of Critical Thinking in Higher Education (Springer, Cham, 2015), pp. 41–92

Chapter   Google Scholar  

A.C. Doyle, in The Strand Magazine . The adventure of the blanched soldier (1926) Retrieved from https://www.dfw-sherlock.org/uploads/3/7/3/8/37380505/1926_november_the_adventure_of_the_blanched_soldier.pdf

S.E. Dreyfus, H.L. Dreyfus, A five-stage model of the mental activities involved in directed skill acquisition (no. ORC-80-2) (University of California-Berkeley Operations Research Center, Berkeley, 1980)

Book   Google Scholar  

C.P. Dwyer, M.J. Hogan, I. Stewart, An integrated critical thinking framework for the 21st century. Think. Skills Creat. 12 , 43–52 (2014)

R.H. Ennis, Critical thinking and subject specificity: Clarification and needed research. Educ. Res. 18 , 4–10 (1989)

R.H. Ennis, Critical thinking across the curriculum: A vision. Topoi. 37 (1), 165–184 (2018)

Epstein, Z. (2016). Has artificial intelligence already surpassed the human brain? Retrieved from https://bgr.com/2016/03/10/alphago-beats-lee-sedol-again/

K.A. Ericsson, R.T. Krampe, C. Tesch-Römer, The role of deliberate practice in the acquisition of expert performance. Psychol. Rev. 100 (3), 363–406 (1993)

Facione, P. A. (1990). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction [Report for the American Psychology Association]. Retrieved from https://files.eric.ed.gov/fulltext/ED315423.pdf

J. Fang, H. Su, Y. Xiao, Will Artificial Intelligence Surpass Human Intelligence? (2018). https://doi.org/10.2139/ssrn.3173876

K.W. Fischer, A theory of cognitive development: The control and construction of hierarchies of skills. Psychol. Rev. 87 , 477–431 (1980)

J.H. Flavell, Metacognition and cognitive monitoring: A new area of cognitive development inquiry. Am. Psychol. 34 (10), 906–911 (1979)

R.M. Gagné, The conditions of learning and theory of instruction , 4th edn. (Holt, Rinehart, & Winston, New York, 1985)

I.J. Good, Speculations concerning the first ultraintelligent machine. Adv Comput. 6 , 31-88 (1966)

C.S. Haller, D.S. Courvoisier, Personality and thinking style in different creative domains. Psychol. Aesthet. Creat. Arts. 4 (3), 149 (2010)

D.L. Hatcher, Is critical thinking across the curriculum a plausible goal? OSSA. 69 (2013) Retrieved from https://scholar.uwindsor.ca/ossaarchive/OSSA10/papersandcommentaries/69

Hitchcock, D. (2018). Critical thinking. Retrieved from https://plato.stanford.edu/entries/critical-thinking/

B.K. Hofer, G.M. Sinatra, Epistemology, metacognition, and self-regulation: Musings on an emerging field. Metacogn. Learn. 5 (1), 113–120 (2010)

D.H. Jonassen, Toward a design theory of problem solving. Educ. Technol. Res. Dev. 48 (4), 63–85 (2000)

D.H. Jonassen, Learning to Solve Problems: A Handbook for Designing Problem-Solving Learning Environments (Routledge, New York, 2011)

D.J. Ketelhut, B.C. Nelson, J. Clarke, C. Dede, A multi-user virtual environment for building and assessing higher order inquiry skills in science. Br. J. Educ. Technol. 41 (1), 56–68 (2010)

R. Luckin, W. Holmes, M. Griffiths, L.B. Forcier, Intelligence Unleashed: An Argument for AI in Education (Pearson Education, London, 2016) Retrieved from http://oro.open.ac.uk/50104/1/Luckin%20et%20al.%20-%202016%20-%20Intelligence%20Unleashed.%20An%20argument%20for%20AI%20in%20Educ.pdf

R. Paul, L. Elder, The miniature guide to critical thinking: Concepts and tools , 4th edn. (2006) Retrieved from https://www.criticalthinking.org/files/Concepts_Tools.pdf

J. Piaget, Part I: Cognitive development in children: Piaget development and learning. J. Res. Sci. Teach. 2 (3), 176–186 (1964)

S.J. Russell, P. Norvig, Artificial Intelligence: A Modern Approach , 3rd edn. (Prentice Hall, Upper Saddle River, 2009) ISBN 978-0-136042594

G. Schraw, K.J. Crippen, K. Hartley, Promoting self-regulation in science education: Metacognition as part of a broader perspective on learning. Res. Sci. Educ. 36 (1–2), 111–139 (2006)

D.H. Schunk, Metacognition, self-regulation, and self-regulated learning: Research recommendations. Educ. Psychol. Rev. 20 (4), 463–467 (2008)

K. Siau, Y. Yang, in Twelve Annual Midwest Association for Information Systems Conference (MWAIS 2017) . Impact of artificial intelligence, robotics, and machine learning on sales and marketing (2017), pp. 18–19

D. Silver, A. Huang, C.J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, et al., Mastering the game of Go with deep neural networks and tree search. Nature. 529 (7587), 484 (2016)

J. M. Spector, M. C. Polson, D. J. Muraida (eds.), Automating Instructional Design: Concepts and Issues (Educational Technology Publications, Englewood Cliffs, 1993)

J.M. Spector, Smart Learning Environments: Concepts and Issues . In G. Chamblee & L. Langub (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference (pp. 2728–2737). (Association for the Advancement of Computing in Education (AACE), Savannah, GA, United States, 2016). Retrieved June 4, 2019 from https://www.learntechlib.org/primary/p/172078/ .

J. M. Spector, Thinking and learning in the anthropocene: The new 3 Rs . Discussion paper presented at the International Big History Association Conference, Philadelphia, PA (2018). Retrieved from http://learndev.org/dl/HLAIBHA2018/Spector%2C%20J.%20M.%20(2018).%20Thinking%20and%20Learning%20in%20the%20Anthropocene.pdf .

J. M. Spector, Complexity, Inquiry Critical Thinking, and Technology: A Holistic and Developmental Approach . In Mind, Brain and Technology (pp. 17–25). (Springer, Cham, 2019).

R.J. Sternberg, Intelligence as developing expertise. Contemp. Educ. Psychol. 24 (4), 359–375 (1999)

The Reboot Foundation. (2018). The State of Critical Thinking: A New Look at Reasoning at Home, School, and Work. Retrieved from https://reboot-foundation.org/wp-content/uploads/_docs/REBOOT_FOUNDATION_WHITE_PAPER.pdf

V. Vinge, The Coming Technological Singularity: How to Survive in the Post-Human Era . Resource document. NASA Technical report server. Retrieved from https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/19940022856.pdf . Accessed 20 June 2019.

D. Zhang, M. Xie, Artificial Intelligence’s Digestion and Reconstruction for Humanistic Feelings . In 2018 International Seminar on Education Research and Social Science (ISERSS 2018) (Atlantis Press, Paris, 2018)

X. Zhu, in Twenty-Ninth AAAI Conference on Artificial Intelligence . Machine Teaching: An Inverse Problem to Machine Learning and an Approach toward Optimal Education (2015)

Download references

Acknowledgements

We wish to acknowledge the generous support of NetDragon and the Digital Research Centre at the University of North Texas.

Initial work is being funded through the NetDragon Digital Research Centre at the University of North Texas with Author as the Principal Investigator.

Author information

Authors and affiliations.

Department of Learning Technologies, University of North Texas Denton, Texas, TX, 76207, USA

Jonathan Michael Spector & Shanshan Ma

You can also search for this author in PubMed   Google Scholar

Contributions

The authors contributed equally to the effort. Both authors read and approved the final manuscript.

Corresponding author

Correspondence to Jonathan Michael Spector .

Ethics declarations

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Cite this article.

Spector, J.M., Ma, S. Inquiry and critical thinking skills for the next generation: from artificial intelligence back to human intelligence. Smart Learn. Environ. 6 , 8 (2019). https://doi.org/10.1186/s40561-019-0088-z

Download citation

Received : 06 June 2019

Accepted : 27 August 2019

Published : 11 September 2019

DOI : https://doi.org/10.1186/s40561-019-0088-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Artificial intelligence
• Critical thinking
• Developmental model
• Human intelligence
• Inquiry learning

Critical Thinking in Academic Research - Second Edition

(4 reviews)

Cindy Gruwell, University of West Florida

Robin Ewing, St. Cloud State University

Copyright Year: 2022

Last Update: 2023

Publisher: Minnesota State Colleges and Universities

Language: English

Formats Available

Conditions of use.

Learn more about reviews.

Reviewed by Julie Jaszkowiak, Community Faculty, Metropolitan State University on 12/22/23

Organized in 11 parts, this his textbook includes introductory information about critical thinking and details about the academic research process. The basics of critical thinking related to doing academic research in Parts I and II. Parts III –... read more

Comprehensiveness rating: 5 see less

Organized in 11 parts, this his textbook includes introductory information about critical thinking and details about the academic research process. The basics of critical thinking related to doing academic research in Parts I and II. Parts III – XI provide specifics on various steps in doing academic research including details on finding and citing source material. There is a linked table of contents so the reader is able to jump to a specific section as needed. There is also a works cited page with information and links to works used for this textbook.

Content Accuracy rating: 5

The content of this textbook is accurate and error free. It contains examples that demonstrate concepts from a variety of disciplines such as “hard science” or “popular culture” that assist in eliminating bias. The authors are librarians so it is clear that their experience as such leads to clear and unbiased content.

Relevance/Longevity rating: 5

General concepts about critical thinking and academic research methodology is well defined and should not become obsolete. Specific content regarding use of citation tools and attribution structure may change but the links to various research sites allow for simple updates.

Clarity rating: 5

This textbook is written in a conversational manner that allows for a more personal interaction with the textbook. It is like the reader is having a conversation with a librarian. Each part has an introduction section that fully defines concepts and terms used for that part.

Consistency rating: 5

In addition to the written content, this textbook contains links to short quizzes at the end of each section. This is consistent throughout each part. Embedded links to additional information are included as necessary.

Modularity rating: 4

This textbook is arranged in 11 modular parts with each part having multiple sections. All of these are linked so a reader can go to a distinct part or section to find specific information. There are some links that refer back to previous sections in the document. It can be challenging to return to where you were once you have jumped to a different section.

Organization/Structure/Flow rating: 5

There is clear definition as to what information is contained within each of the parts and subsequent sections. The textbook follows the logical flow of the process of researching and writing a research paper.

Interface rating: 4

The pictures have alternative text that appears when you hover over the text. There is one picture on page 102 that is a link to where the downloaded picture is from. The pictures are clear and supportive of the text for a visual learner. All the links work and go to either the correct area of the textbook or to a valid website. If you are going to use the embedded links to go to other sections of the textbook you need to keep track of where you are as it can sometimes get confusing as to where you went based on clicking links.

Grammatical Errors rating: 4

This is not really a grammatical error but I did notice on some of the quizzes if you misspelled a work for fill in the blank it was incorrect. It was also sometimes challenging to come up with the correct word for the fill in the blanks.

Cultural Relevance rating: 5

There are no examples or text that are culturally insensitive or offensive. The examples are general and would be applicable to a variety of students study many different academic subjects. There are references and information to many research tools from traditional such as checking out books and articles from the library to more current such as blogs and other electronic sources. This information appeals to a wide expanse of student populations.

I really enjoyed the quizzes at the end of each section. It is very beneficial to test your knowledge and comprehension of what you just read. Often I had to return and reread the content more critically based on my quiz results! They are just the right length to not disrupt the overall reading of the textbook and cover the important content and learning objectives.

Reviewed by Sara Stigberg, Adjunct Reference Librarian, Truman College, City Colleges of Chicago on 3/15/23

Critical Thinking in Academic Research thoroughly covers the basics of academic research for undergraduates, including well-guided deeper dives into relevant areas. The authors root their introduction to academic research principles and practices... read more

Critical Thinking in Academic Research thoroughly covers the basics of academic research for undergraduates, including well-guided deeper dives into relevant areas. The authors root their introduction to academic research principles and practices in the Western philosophical tradition, focused on developing students' critical thinking skills and habits around inquiry, rationales, and frameworks for research.

This text conforms to the principles and frames of the Framework for Information Literacy for Higher Education, published by the Association of College and Research Libraries. It includes excellent, clear, step-by-step guides to help students understand rationales and techniques for academic research.

Essential for our current information climate, the authors present relevant information for students who may be new to academic research, in ways and with content that is not too broad or too narrow, or likely to change drastically in the near future.

The authors use clear and well-considered language and explanations of ideas and terms, contextualizing the scholarly research process and tools in a relatable manner. As mentioned earlier, this text includes excellent step-by-step guides, as well as illustrations, visualizations, and videos to instruct students in conducting academic research.

(4.75) The terminology and framework of this text are consistent. Early discussions of critical thinking skills are tied in to content in later chapters, with regard to selecting different types of sources and search tools, as well as rationales for choosing various formats of source references. Consciously making the theme of critical thinking as applied to the stages of academic research more explicit and frequent within the text would further strengthen it, however.

Modularity rating: 5

Chapters are divided in a logical, progressive manner throughout the text. The use of embedded links to further readings and some other relevant sections of the text are an excellent way of providing references and further online information, without overwhelming or side-tracking the reader.

Topics in the text are organized in logical, progressive order, transitioning cleanly from one focus to the next. Each chapter begins with a helpful outline of topics that will be covered within it.

There are no technical issues with the interface for this text. Interactive learning tools such as the many self-checks and short quizzes that are included throughout the text are a great bonus for reinforcing student learning, and the easily-accessible table of contents was very helpful. There are some slight inconsistencies across chapters, however, relative to formatting images and text and spacing, and an image was missing in the section on Narrowing a Topic. Justifying copy rather than aligning-left would prevent hyphenation, making the text more streamlined.

Grammatical Errors rating: 5

(4.75) A few minor punctuation errors are present.

The authors of this text use culturally-relevant examples and inclusive language. The chapter on Barriers to Critical Thinking works directly to break down bias and preconceived notions.

Overall, Critical Thinking in Academic Research is an excellent general textbook for teaching the whys and hows of academic research to undergraduates. A discussion of annotated bibliographies would be a great addition for future editions of the text. ---- (As an aside for the authors, I am curious if the anonymous data from the self-checks and quizzes is being collected and analyzed for assessment purposes. I'm sure it would be interesting!)

Reviewed by Ann Bell-Pfeifer, Program Director/ Instructor, Minnesota State Community and Technical College on 2/15/23

The book has in depth coverage of academic research. A formal glossary and index were not included. read more

Comprehensiveness rating: 4 see less

The book has in depth coverage of academic research. A formal glossary and index were not included.

The book appears error free and factual.

The content is current and would support students who are pursuing writing academic research papers.

Excellent explanations for specific terms were included throughout the text.

The text is easy to follow with a standardized format and structure.

The text contains headings and topics in each section.

It is easy to follow the format and review each section.

Interface rating: 5

The associated links were useful and not distracting.

No evidence of grammatical errors were found in the book.

The book is inclusive.

The book was informative, easy to follow, and sequential allowing the reader to digest each section before moving into another.

Reviewed by Jenny Inker, Assistant Professor, Virginia Commonwealth University on 8/23/22

This book provides a comprehensive yet easily comprehensible introduction to critical thinking in academic research. The author lays a foundation with an introduction to the concepts of critical thinking and analyzing and making arguments, and... read more

This book provides a comprehensive yet easily comprehensible introduction to critical thinking in academic research. The author lays a foundation with an introduction to the concepts of critical thinking and analyzing and making arguments, and then moves into the details of developing research questions and identifying and appropriately using research sources. There are many wonderful links to other open access publications for those who wish to read more or go deeper.

The content of the book appears to be accurate and free of bias.

The examples used throughout the book are relevant and up-to-date, making it easy to see how to apply the concepts in real life.

The text is very accessibly written and the content is presented in a simple, yet powerful way that helps the reader grasp the concepts easily. There are many short, interactive exercises scattered throughout each chapter of the book so that the reader can test their own knowledge as they go along. It would be even better if the author had provided some simple feedback explaining why quiz answers are correct or incorrect in order to bolster learning, but this is a very minor point and the interactive exercises still work well without this.

The book appears consistent throughout with regard to use of terminology and tone of writing. The basic concepts introduced in the early chapters are used consistently throughout the later chapters.

This book has been wonderfully designed into bite sized chunks that do not overwhelm the reader. This is perhaps its best feature, as this encourages the reader to take in a bit of information, digest it, check their understanding of it, and then move on to the next concept. I loved this!

The book is organized in a manner that introduces the basic architecture of critical thinking first, and then moves on to apply it to the subject of academic research. While the entire book would be helpful for college students (undergraduates particularly), the earlier chapters on critical thinking and argumentation also stand well on their own and would be of great utility to students in general.

This book was extremely easy to navigate with a clear, drop down list of chapters and subheadings on the left side of the screen. When the reader clicks on links to additional material, these open up in a new tab which keeps things clear and organized. Images and charts were clear and the overall organization is very easy to follow.

I came across no grammatical errors in the text.

Cultural Relevance rating: 4

This is perhaps an area where the book could do a little more. I did not come across anything that seemed culturally insensitive or offensive but on the other hand, the book might have taken more opportunities to represent a greater diversity of races, ethnicities, and backgrounds.

This book seems tailor made for undergraduate college students and I would highly recommend it. I think it has some use for graduate students as well, although some of the examples are perhaps little basic for this purpose. As well as using this book to guide students on doing academic research, I think it could also be used as a very helpful introduction to the concept of critical thinking by focusing solely on chapters 1-4.

Table of Contents

• Introduction
• Part I. What is Critical Thinking?
• Part II. Barriers to Critical Thinking
• Part III. Analyzing Arguments
• Part IV. Making an Argument
• Part V. Research Questions
• Part VI. Sources and Information Needs
• Part VII. Types of Sources
• Part VIII. Precision Searching
• Part IX. Evaluating Sources
• Part X. Ethical Use and Citing Sources
• Part XI. Copyright Basics
• Works Cited
• About the Authors

Ancillary Material

About the book.

Critical Thinking in Academic Research - 2nd Edition provides examples and easy-to-understand explanations to equip students with the skills to develop research questions, evaluate and choose the right sources, search for information, and understand arguments. This 2nd Edition includes new content based on student feedback as well as additional interactive elements throughout the text.

About the Contributors

Cindy Gruwell is an Assistant Librarian/Coordinator of Scholarly Communication at the University of West Florida. She is the library liaison to the department of biology and the College of Health which has extensive nursing programs, public health, health administration, movement, and medical laboratory sciences. In addition to supporting health sciences faculty, she oversees the Argo IRCommons (Institutional Repository) and provides scholarly communication services to faculty across campus. Cindy graduated with her BA (history) and MLS from the University of California, Los Angeles and has a Masters in Education from Bemidji State University. Cindy’s research interests include academic research support, publishing, and teaching.

Robin Ewing is a Professor/Collections Librarian at St. Cloud State University. Robin is the liaison to the College of Education and Learning Design. She oversees content selection for the Library’s collections. Robin graduated with her BBA (Management) and MLIS from the University of Oklahoma. She also has a Masters of Arts in Teaching from Bemidji State University. Robin’s research interests include collection analysis, assessment, and online teaching.

Contribute to this Page

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

11.5 Critical Thinking and Research Applications

Learning objectives.

• Analyze source materials to determine how they support or refute the working thesis.
• Identify connections between source materials and eliminate redundant or irrelevant source materials.
• Identify instances when it is appropriate to use human sources, such as interviews or eyewitness testimony.
• Select information from sources to begin answering the research questions.
• Determine an appropriate organizational structure for the research paper that uses critical analysis to connect the writer’s ideas and information taken from sources.

At this point in your project, you are preparing to move from the research phase to the writing phase. You have gathered much of the information you will use, and soon you will be ready to begin writing your draft. This section helps you transition smoothly from one phase to the next.

Beginning writers sometimes attempt to transform a pile of note cards into a formal research paper without any intermediary step. This approach presents problems. The writer’s original question and thesis may be buried in a flood of disconnected details taken from research sources. The first draft may present redundant or contradictory information. Worst of all, the writer’s ideas and voice may be lost.

An effective research paper focuses on the writer’s ideas—from the question that sparked the research process to how the writer answers that question based on the research findings. Before beginning a draft, or even an outline, good writers pause and reflect. They ask themselves questions such as the following:

• How has my thinking changed based on my research? What have I learned?
• Was my working thesis on target? Do I need to rework my thesis based on what I have learned?
• How does the information in my sources mesh with my research questions and help me answer those questions? Have any additional important questions or subtopics come up that I will need to address in my paper?
• How do my sources complement each other? What ideas or facts recur in multiple sources?
• Where do my sources disagree with each other, and why?

In this section, you will reflect on your research and review the information you have gathered. You will determine what you now think about your topic. You will synthesize , or put together, different pieces of information that help you answer your research questions. Finally, you will determine the organizational structure that works best for your paper and begin planning your outline.

Review the research questions and working thesis you developed in Chapter 11 “Writing from Research: What Will I Learn?” , Section 11.2 “Steps in Developing a Research Proposal” . Set a timer for ten minutes and write about your topic, using your questions and thesis to guide your writing. Complete this exercise without looking over your notes or sources. Base your writing on the overall impressions and concepts you have absorbed while conducting research. If additional, related questions come to mind, jot them down.

Selecting Useful Information

At this point in the research process, you have gathered information from a wide variety of sources. Now it is time to think about how you will use this information as a writer.

When you conduct research, you keep an open mind and seek out many promising sources. You take notes on any information that looks like it might help you answer your research questions. Often, new ideas and terms come up in your reading, and these, too, find their way into your notes. You may record facts or quotations that catch your attention even if they did not seem immediately relevant to your research question. By now, you have probably amassed an impressively detailed collection of notes.

You will not use all of your notes in your paper.

Good researchers are thorough. They look at multiple perspectives, facts, and ideas related to their topic, and they gather a great deal of information. Effective writers, however, are selective. They determine which information is most relevant and appropriate for their purpose. They include details that develop or explain their ideas—and they leave out details that do not. The writer, not the pile of notes, is the controlling force. The writer shapes the content of the research paper.

While working through Chapter 11 “Writing from Research: What Will I Learn?” , Section 11.4 “Strategies for Gathering Reliable Information” , you used strategies to filter out unreliable or irrelevant sources and details. Now you will apply your critical-thinking skills to the information you recorded—analyzing how it is relevant, determining how it meshes with your ideas, and finding how it forms connections and patterns.

Writing at Work

When you create workplace documents based on research, selectivity remains important. A project team may spend months conducting market surveys to prepare for rolling out a new product, but few managers have time to read the research in its entirety. Most employees want the research distilled into a few well-supported points. Focused, concise writing is highly valued in the workplace.

Identify Information That Supports Your Thesis

In Note 11.81 “Exercise 1” , you revisited your research questions and working thesis. The process of writing informally helped you see how you might begin to pull together what you have learned from your research. Do not feel anxious, however, if you still have trouble seeing the big picture. Systematically looking through your notes will help you.

Begin by identifying the notes that clearly support your thesis. Mark or group these, either physically or using the cut-and-paste function in your word-processing program. As you identify the crucial details that support your thesis, make sure you analyze them critically. Ask the following questions to focus your thinking:

• Is this detail from a reliable, high-quality source? Is it appropriate for me to cite this source in an academic paper? The bulk of the support for your thesis should come from reliable, reputable sources. If most of the details that support your thesis are from less-reliable sources, you may need to do additional research or modify your thesis.
• Is the link between this information and my thesis obvious—or will I need to explain it to my readers? Remember, you have spent more time thinking and reading about this topic than your audience. Some connections might be obvious to both you and your readers. More often, however, you will need to provide the analysis or explanation that shows how the information supports your thesis. As you read through your notes, jot down ideas you have for making those connections clear.
• What personal biases or experiences might affect the way I interpret this information? No researcher is 100 percent objective. We all have personal opinions and experiences that influence our reactions to what we read and learn. Good researchers are aware of this human tendency. They keep an open mind when they read opinions or facts that contradict their beliefs.

It can be tempting to ignore information that does not support your thesis or that contradicts it outright. However, such information is important. At the very least, it gives you a sense of what has been written about the issue. More importantly, it can help you question and refine your own thinking so that writing your research paper is a true learning process.

Find Connections between Your Sources

As you find connections between your ideas and information in your sources, also look for information that connects your sources. Do most sources seem to agree on a particular idea? Are some facts mentioned repeatedly in many different sources? What key terms or major concepts come up in most of your sources regardless of whether the sources agree on the finer points? Identifying these connections will help you identify important ideas to discuss in your paper.

Look for subtler ways your sources complement one another, too. Does one author refer to another’s book or article? How do sources that are more recent build upon the ideas developed in earlier sources?

Be aware of any redundancies in your sources. If you have amassed solid support from a reputable source, such as a scholarly journal, there is no need to cite the same facts from an online encyclopedia article that is many steps removed from any primary research. If a given source adds nothing new to your discussion and you can cite a stronger source for the same information, use the stronger source.

Determine how you will address any contradictions found among different sources. For instance, if one source cites a startling fact that you cannot confirm anywhere else, it is safe to dismiss the information as unreliable. However, if you find significant disagreements among reliable sources, you will need to review them and evaluate each source. Which source presents a sounder argument or more solid evidence? It is up to you to determine which source is the most credible and why.

Finally, do not ignore any information simply because it does not support your thesis. Carefully consider how that information fits into the big picture of your research. You may decide that the source is unreliable or the information is not relevant, or you may decide that it is an important point you need to bring up. What matters is that you give it careful consideration.

As Jorge reviewed his research, he realized that some of the information was not especially useful for his purpose. His notes included several statements about the relationship between soft drinks that are high in sugar and childhood obesity—a subtopic that was too far outside of the main focus of the paper. Jorge decided to cut this material.

Reevaluate Your Working Thesis

A careful analysis of your notes will help you reevaluate your working thesis and determine whether you need to revise it. Remember that your working thesis was the starting point—not necessarily the end point—of your research. You should revise your working thesis if your ideas changed based on what you read. Even if your sources generally confirmed your preliminary thinking on the topic, it is still a good idea to tweak the wording of your thesis to incorporate the specific details you learned from research.

Jorge realized that his working thesis oversimplified the issues. He still believed that the media was exaggerating the benefits of low-carb diets. However, his research led him to conclude that these diets did have some advantages. Read Jorge’s revised thesis.

Although following a low-carbohydrate diet can benefit some people, these diets are not necessarily the best option for everyone who wants to lose weight or improve their health.

Synthesizing and Organizing Information

By now your thinking on your topic is taking shape. You have a sense of what major ideas to address in your paper, what points you can easily support, and what questions or subtopics might need a little more thought. In short, you have begun the process of synthesizing information—that is, of putting the pieces together into a coherent whole.

It is normal to find this part of the process a little difficult. Some questions or concepts may still be unclear to you. You may not yet know how you will tie all of your research together. Synthesizing information is a complex, demanding mental task, and even experienced researchers struggle with it at times. A little uncertainty is often a good sign! It means you are challenging yourself to work thoughtfully with your topic instead of simply restating the same information.

Use Your Research Questions to Synthesize Information

You have already considered how your notes fit with your working thesis. Now, take your synthesis a step further. Analyze how your notes relate to your major research question and the subquestions you identified in Chapter 11 “Writing from Research: What Will I Learn?” , Section 11.2 “Steps in Developing a Research Proposal” . Organize your notes with headings that correspond to those questions. As you proceed, you might identify some important subtopics that were not part of your original plan, or you might decide that some questions are not relevant to your paper.

Categorize information carefully and continue to think critically about the material. Ask yourself whether the sources are reliable and whether the connections between ideas are clear.

Remember, your ideas and conclusions will shape the paper. They are the glue that holds the rest of the content together. As you work, begin jotting down the big ideas you will use to connect the dots for your reader. (If you are not sure where to begin, try answering your major research question and subquestions. Add and answer new questions as appropriate.) You might record these big ideas on sticky notes or type and highlight them within an electronic document.

Jorge looked back on the list of research questions that he had written down earlier. He changed a few to match his new thesis, and he began a rough outline for his paper.

Review your research questions and working thesis again. This time, keep them nearby as you review your research notes.

• Identify information that supports your working thesis.
• Identify details that call your thesis into question. Determine whether you need to modify your thesis.
• Use your research questions to identify key ideas in your paper. Begin categorizing your notes according to which topics are addressed. (You may find yourself adding important topics or deleting unimportant ones as you proceed.)
• Write out your revised thesis and at least two or three big ideas.

You may be wondering how your ideas are supposed to shape the paper, especially since you are writing a research paper based on your research. Integrating your ideas and your information from research is a complex process, and sometimes it can be difficult to separate the two.

Some paragraphs in your paper will consist mostly of details from your research. That is fine, as long as you explain what those details mean or how they are linked. You should also include sentences and transitions that show the relationship between different facts from your research by grouping related ideas or pointing out connections or contrasts. The result is that you are not simply presenting information; you are synthesizing, analyzing, and interpreting it.

Plan How to Organize Your Paper

The final step to complete before beginning your draft is to choose an organizational structure. For some assignments, this may be determined by the instructor’s requirements. For instance, if you are asked to explore the impact of a new communications device, a cause-and-effect structure is obviously appropriate. In other cases, you will need to determine the structure based on what suits your topic and purpose. For more information about the structures used in writing, see Chapter 10 “Rhetorical Modes” .

The purpose of Jorge’s paper was primarily to persuade. With that in mind, he planned the following outline.

Review the organizational structures discussed in this section and Chapter 10 “Rhetorical Modes” . Working with the notes you organized earlier, follow these steps to begin planning how to organize your paper.

• Create an outline that includes your thesis, major subtopics, and supporting points.
• The major headings in your outline will become sections or paragraphs in your paper. Remember that your ideas should form the backbone of the paper. For each major section of your outline, write out a topic sentence stating the main point you will make in that section.
• As you complete step 2, you may find that some points are too complex to explain in a sentence. Consider whether any major sections of your outline need to be broken up and jot down additional topic sentences as needed.
• Review your notes and determine how the different pieces of information fit into your outline as supporting points.

Collaboration

Please share the outline you created with a classmate. Examine your classmate’s outline and see if any questions come to mind or if you see any area that would benefit from an additional point or clarification. Return the outlines to each other and compare observations.

The structures described in this section and Chapter 10 “Rhetorical Modes” can also help you organize information in different types of workplace documents. For instance, medical incident reports and police reports follow a chronological structure. If the company must choose between two vendors to provide a service, you might write an e-mail to your supervisor comparing and contrasting the choices. Understanding when and how to use each organizational structure can help you write workplace documents efficiently and effectively.

Key Takeaways

• An effective research paper focuses on presenting the writer’s ideas using information from research as support.
• Effective writers spend time reviewing, synthesizing, and organizing their research notes before they begin drafting a research paper.
• It is important for writers to revisit their research questions and working thesis as they transition from the research phase to the writing phrase of a project. Usually, the working thesis will need at least minor adjustments.
• To organize a research paper, writers choose a structure that is appropriate for the topic and purpose. Longer papers may make use of more than one structure.

Writing for Success Copyright © 2015 by University of Minnesota is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Translate this page from English...

*Machine translated pages not guaranteed for accuracy. Click Here for our professional translations.

Research in Critical Thinking

Each year it sponsors an annual International Conference on Critical Thinking and Educational Reform. It has worked with the College Board, the National Education Association, the U.S. Department of Education, as well as numerous colleges, universities, and school districts to facilitate the implementation of critical thinking instruction focused on intellectual standards.

The following three studies demonstrate:

• the fact that, as a rule, critical thinking is not presently being effectively taught at the high school, college and university level, and yet
• it is possible to do so.

To assess students' understanding of critical thinking, we recommend use of the International Critical Thinking Test as well as the Critical Thinking Interview Profile for College Students . To assess faculty understanding of critical thinking and its importance to instruction, we recommend the Critical Thinking Interview Profile For Teachers and Faculty . By registering as a member of the community, you will have access to streaming video, which includes a sample student interview with Dr. Richard Paul and Rush Cosgrove.

RESEARCH TITLES

View Abstract  -  View Full Dissertation (Adobe Acrobat PDF)

  A Critical Analysis of Richard Paul's Substantive Trans-disciplinary Conception of Critical Thinking

by Enoch Hale, Ph.D.

Union Institute & University - Cincinnati, Ohio - October 2008

View Abstract      Dissertation Table of Contents

Effect of a Model for Critical Thinking on Student Achievement in Primary Source Document Analysis and Interpretation, Argumentative Reasoning, Critical Thinking Dispositions and History Content in a Community College History Course Abstract of the Study, conducted by Jenny Reed, in partial fulfillment for her dissertation (October 26, 1998) View Abstract   -   View Full Dissertation (Adobe Acrobat PDF)

The Effect of Richard Paul's Universal Elements and Standards of Reasoning on Twelfth Grade Composition A Research Proposal Presented to the Faculty Of the School of Education Alliant International University In Partial Fulfillment of the Requirements for the Degree of Master of Arts in Education: Teaching Study conducted by J. Stephen Scanlan, San Diego (2006) View Abstract   -    View Full Dissertation (Adobe Acrobat PDF)

Study of 38 Public Universities and 28 Private Universities To Determine Faculty Emphasis on Critical Thinking In Instruction

Principal Researchers: Dr. Richard Paul, Dr. Linda Elder, and Dr. Ted Bartell

View Abstract    -    View the full study

Substantive Critical Thinking as Developed by the Foundation for Critical Thinking Proves Effective in Raising SAT and ACT Test Scores at West Side High School:  Staff Development Program Utilizes Critical Thinking Instruction to Improve Student Performance on ACT and SAT Tests, and in Critical Reading, Writing and Math Dr.   John Crook, West Side High School Principal View the Report

Teaching Critical Thinking Skills to Fourth Grade Students Identified as Gifted and Talented by Debra Connerly Graceland University - Cedar Rapids, Iowa - December 2006 View the Report

The Loss of the Space Shuttle Columbia: Portaging Leadership Lessons with a Critical Thinking Model

by Rob Niewoehner, Ph.D. U.S. Navy Graceland University - Cedar Rapids, Iowa - December 2006 View the Report

The Importance of Critical Thinking Skills in Research

Why is Critical Thinking Important: A Disruptive Force

Research anxiety seems to be taking an increasingly dominant role in the world of academic research. The pressure to publish or perish can warp your focus into thinking that the only good research is publishable research!

Today, your role as the researcher appears to take a back seat to the perceived value of the topic and the extent to which the results of the study will be cited around the world. Due to financial pressures and a growing tendency of risk aversion, studies are increasingly going down the path of applied research rather than basic or pure research . The potential for breakthroughs is being deliberately limited to incremental contributions from researchers who are forced to worry more about job security and pleasing their paymasters than about making a significant contribution to their field.

A Slow Decline

So what lead the researchers to their love of science and scientific research in the first place? The answer is critical thinking skills. The more that academic research becomes governed by policies outside of the research process, the less opportunity there will be for researchers to exercise such skills.

True research demands new ideas , perspectives, and arguments based on willingness and confidence to revisit and directly challenge existing schools of thought and established positions on theories and accepted codes of practice. Success comes from a recursive approach to the research question with an iterative refinement based on constant reflection and revision.

The importance of critical thinking skills in research is therefore huge, without which researchers may even lack the confidence to challenge their own assumptions.

A Misunderstood Skill

Critical thinking is widely recognized as a core competency and as a precursor to research. Employers value it as a requirement for every position they post, and every survey of potential employers for graduates in local markets rate the skill as their number one concern.

Related: Do you have questions on research idea or manuscript drafting? Get personalized answers on the FREE Q&A Forum!

When asked to clarify what critical thinking means to them, employers will use such phrases as “the ability to think independently,” or “the ability to think on their feet,” or “to show some initiative and resolve a problem without direct supervision.” These are all valuable skills, but how do you teach them?

For higher education institutions in particular, when you are being assessed against dropout, graduation, and job placement rates, where does a course in critical thinking skills fit into the mix? Student Success courses as a precursor to your first undergraduate course will help students to navigate the campus and whatever online resources are available to them (including the tutoring center), but that doesn’t equate to raising critical thinking competencies.

The Dependent Generation

As education becomes increasingly commoditized and broken-down into components that can be delivered online for maximum productivity and profitability, we run the risk of devaluing academic discourse and independent thought. Larger class sizes preclude substantive debate, and the more that content is broken into sound bites that can be tested in multiple-choice questions, the less requirement there will be for original thought.

Academic journals value citation above all else, and so content is steered towards the type of articles that will achieve high citation volume. As such, students and researchers will perpetuate such misuse by ensuring that their papers include only highly cited works. And the objective of high citation volume is achieved.

We expand the body of knowledge in any field by challenging the status quo. Denying the veracity of commonly accepted “facts” or playing devil’s advocate with established rules supports a necessary insurgency that drives future research. If we do not continue to emphasize the need for critical thinking skills to preserve such rebellion, academic research may begin to slowly fade away.

Rate this article Cancel Reply

Your email address will not be published.

Enago Academy's Most Popular Articles

• Reporting Research

Choosing the Right Analytical Approach: Thematic analysis vs. content analysis for data interpretation

In research, choosing the right approach to understand data is crucial for deriving meaningful insights.…

Comparing Cross Sectional and Longitudinal Studies: 5 steps for choosing the right approach

The process of choosing the right research design can put ourselves at the crossroads of…

• Career Corner

Unlocking the Power of Networking in Academic Conferences

Embarking on your first academic conference experience? Fear not, we got you covered! Academic conferences…

Research Recommendations – Guiding policy-makers for evidence-based decision making

Research recommendations play a crucial role in guiding scholars and researchers toward fruitful avenues of…

• AI in Academia

Disclosing the Use of Generative AI: Best practices for authors in manuscript preparation

The rapid proliferation of generative and other AI-based tools in research writing has ignited an…

Intersectionality in Academia: Dealing with diverse perspectives

Meritocracy and Diversity in Science: Increasing inclusivity in STEM education

Avoiding the AI Trap: Pitfalls of relying on ChatGPT for PhD applications

Sign-up to read more

Subscribe for free to get unrestricted access to all our resources on research writing and academic publishing including:

• 2000+ blog articles
• 50+ Webinars
• 10+ Expert podcasts
• 50+ Infographics
• 10+ Checklists
• Research Guides

We hate spam too. We promise to protect your privacy and never spam you.

I am looking for Editing/ Proofreading services for my manuscript Tentative date of next journal submission:

What should universities' stance be on AI tools in research and academic writing?

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Int J Exerc Sci

Research and Critical Thinking : An Important Link for Exercise Science Students Transitioning to Physical Therapy

Harvey w. wallmann.

1 Department of Physical Therapy, Western Kentucky University, Bowling Green, KY, USA

DONALD L. HOOVER

2 Department of Physical Therapy Education, Rockhurst University, Kansas City, MO, USA

Critical thinking skills are increasingly necessary for success in professional health care careers. Changes in the contemporary healthcare system in the United States arguably make these critical thinking skills more important than they have ever been, as clinicians are required on a daily basis to evaluate multiple bits of information about patients with multiple-systemic health concerns and make appropriate treatment decisions based on this information. We believe the IJES, with its emphasis on engaging undergraduate and graduate students in research and scholarly activity, is a valuable resource for promoting the higher-order critical thinking skills necessary for preparing exercise science students with an interest in professional healthcare careers such as physical therapy.

Higher-order critical thinking skills are necessary for students preparing for and/or enrolled in professional programs, especially the ability to evaluate and synthesize information, which are vital for problem-solving. Essentially, critical thinking is learning to think independently and to develop one’s own opinions supported by existing evidence. In learning scenarios that promote and foster problem-solving and critical thinking skills, it is much more difficult for the student to simply adhere to the role of the passive student; rather, this type of learning prompts the student to assume the role of a self-reliant thinker and researcher.

However, attaining critical thinking skills does not come without its challenges as students must be able to manage a vast array of resources within a series of complex network systems. This is especially true when students are asked to write a research paper, which is one of the most common methods for teaching critical thinking skills. Inherent within writing a research paper are various levels of reasoning with each level becoming progressively more abstract, complex, and effortful. This, according to Bloom’s taxonomy, promotes higher-order thinking skill and more critical thought in the form of synthesis-level thinking and builds upon the prior skill levels in a hierarchical fashion ( 1 ). However, when confronted with this seemingly daunting task, many college students shy away; presumably, because they lack these skills and therefore need to be taught how to learn and apply them ( 2 , 4 ).

Upon closer scrutiny, deficiencies in critical thinking skills among students may rest with the educational system itself, which often stresses memorization of voluminous amounts of material essentially unrelated to any type of application at all ( 2 ). The question then arises as to the extent which critical thinking is initiated during a student’s education in any given institution in higher education. As such, any focus on learning without critical thought becomes less meaningful, thereby disengaging students from any formal training and experience specifically as it relates to critically reviewing and evaluating research ( 3 ).

Arguably, an important component of critical thinking skills is the ability to critically examine and understand published research in one’s professional area of interest ( 7 ). Requiring students to critique published research is one way of addressing the goal of teaching students to critically evaluate research while gaining experience doing it ( 3 ). At its very essence, scientific research is a problem-based learning activity that sharpens critical thinking skills.

An even greater challenge, and one that provides a framework for differentiating between different levels of learning and thought by incorporating reasoning and critical thinking skills to a greater degree, is to actually engage students in the scientific method. Here, students actively participate in the formulation of a research question, data collection, and statistical analysis as a means of creating a learning environment that encourages or even forces them to engage in critical thinking and higher level reasoning. This process is arguably complete only when students are encouraged to complete the manuscript submission process in order to publish their research. Additionally, the manuscript submission process teaches students to be consumers of information while constantly examining, questioning, and evaluating the credibility of sources as they make sense of their own work ( 6 ).

Thus, we see the International Journal of Exercise Science (IJES), with its aim on engaging undergraduate and graduate students in scholarly activity, as a quite suitable vehicle for promoting critical thinking skills in exercise science students interested in entering professional programs such as physical therapy. For example, a very meaningful way to engage students is to enlist their support in a research effort of interest to them and for them to assist in the publication process. Given changes seen among Kinesiology majors on the undergraduate level in recent years, the IJES, with its emphasis on student involvement in the research process, is a great venue for disseminating research findings emphasizing this type of undergraduate student involvement ( 5 ). The research findings typically published in this journal are highly relevant to physical therapy given the central role of exercise within this healthcare profession.

We encourage all authors who work with undergraduate students interested in physical therapy to publish in this journal. Doing so will help to “raise the level” of critical thinking skills for all students involved. Among other things, doing so would also provide another valuable measure for evaluating applicants to physical therapy programs. We believe that student experiences of this nature are helpful when making admissions decisions for physical therapy programs, in part because evidence of prior research experiences provide some indication of a given student’s ability to handle the level of critical thinking necessary for success within a physical therapy education program.

In other words, while measures such as undergraduate GPA and exam scores on standardized aptitude tests are helpful in the selection process, they are certainly finite and incomplete measures for predicting which students are most capable of handling the rigors of these graduate professional programs. We believe that undergraduate research experiences provide an emphasis on higher-order critical thinking skills that are often hard to replicate in other parts of the typical undergraduate educational experience, and these experiences typically translate broadly into academic success when these students matriculate into graduate professional programs such as physical therapy.

When viewed from another vantage point, the IJES may also serve as a vehicle for further refining critical thinking skills once students are enrolled in graduate professional programs. In this same vein, we also encourage researchers working with students enrolled in Doctor of Physical Therapy (DPT) programs to publish in the IJES. Physical therapy curricula typically employ a research course sequence as part of the overall curriculum, as a means of fostering critical thinking skills for all students involved, and many projects completed in this manner are particularly suitable for publication in this journal. Many of the manuscripts published to date in the IJES are similarly highly generalizable to therapeutic exercise scenarios regularly encountered in physical therapy practice, providing a valuable resource for students and practicing clinicians alike.

The free, full-text format of the IJES further increases the attractiveness of this journal, as anecdotal evidence suggests that both students and practicing clinicians are mostly likely to use the resources they can access most easily. Thus, DPT faculty can confidently point to manuscripts in this journal as 1) resources for promoting evidence-based clinical practice as well as 2) an attainable target for publishing their own work. Realizing any of these aims on a consistent basis can contribute to stronger critical thinking skills and perhaps higher clinical outcomes for all involved.

In summary, higher-order critical thinking skills are increasingly necessary for success in professional health care careers. Changes in the contemporary healthcare system in the United States arguably make these critical thinking skills more important than they’ve ever been, as clinicians are required on a daily basis to evaluate multiple bits of information about patients with multiple-systemic health concerns and make appropriate treatment decisions based on this information.

We believe the IJES, with its emphasis on engaging undergraduate and graduate students in research and scholarly activity, is a valuable resource for promoting the higher-order critical thinking skills necessary for preparing exercise science students with an interest in professional healthcare careers such as physical therapy. This viewpoint is based not only upon our experience working with students who enter DPT programs possessing strong higher-order critical thinking skills honed through undergraduate research activities, but also partly upon the many research projects students complete in DPT programs that are highly suitable for dissemination in this journal. The IJES has much potential for strengthening the existing bonds between exercise science and physical therapy that benefit all involved.