research paper about computer system servicing

Effects of Computer-Based Training in Computer Hardware Servicing on Students' Academic Performance

363  citations

256  citations

241  citations

134  citations

Related Papers (5)

Trending questions (3).

The study did not address the lack of prior knowledge encountered by students when practicing computer system servicing tasks in a hands-on environment.

Not addressed in the paper.

Students exhibited gaming behavior during hands-on tasks but reduced it after intervention. Returning students to lectures effectively discouraged this behavior. Specific difficulties were not addressed in the paper.

Ask Copilot

Related papers

Related topics

Problems Met by Computer System Servicing Grade 10 Students in Gov. Feliciano Leviste Memorial National High School

• Emerson Punzalan

INTRODUCTION

Technology and Livelihood Education under the K-12 curriculum offers the subject Computer System Servicing for grades 9 and 10. The researcher as one of the teachers teaching this subject encounter different attitudes of students towards the subject. The factors teacher, competency, environment, and family were tested to know the highest factor affecting the students.

A descriptive method of research using a questionnaire was utilized to gather empirical data for the study on problems met by CSS students in learning the lesson. The questionnaire was used to make the survey on the most factor that affects the learning of students in the subject. The respondents of the study were forty-five (45) CSS students in GFLMNHS.

Most of the respondents were in the average age bracket, female, and single when it comes to their profile. They also agreed on the problems stated on the questionnaire giving all the factors the verbal interpretation of Agree in their composite mean. The teacher's factor which had the highest composite mean entails that the students were greatly affected by the way on how the teacher acts as the facilitator n class.

DISCUSSIONS

The result demonstrated the need for an action plan to be utilized in the field of Education where all the factors will be dealt with. Teachers, who had been the greatest factor, must be very careful in everything he does in the classroom to make the learning possible and easy.

Information

• For Readers
• For Authors
• For Librarians

©2017 by Ascendens Asia Pte. Ltd. | NLB Singapore-Registered Publisher.

• Review article
• Open access
• Published: 02 October 2017

Computer-based technology and student engagement: a critical review of the literature

• Laura A. Schindler   ORCID: orcid.org/0000-0001-8730-5189 1 ,
• Gary J. Burkholder 2 , 3 ,
• Osama A. Morad 1 &
• Craig Marsh 4  

International Journal of Educational Technology in Higher Education volume  14 , Article number:  25 ( 2017 ) Cite this article

384k Accesses

137 Citations

39 Altmetric

Metrics details

Computer-based technology has infiltrated many aspects of life and industry, yet there is little understanding of how it can be used to promote student engagement, a concept receiving strong attention in higher education due to its association with a number of positive academic outcomes. The purpose of this article is to present a critical review of the literature from the past 5 years related to how web-conferencing software, blogs, wikis, social networking sites ( Facebook and Twitter ), and digital games influence student engagement. We prefaced the findings with a substantive overview of student engagement definitions and indicators, which revealed three types of engagement (behavioral, emotional, and cognitive) that informed how we classified articles. Our findings suggest that digital games provide the most far-reaching influence across different types of student engagement, followed by web-conferencing and Facebook . Findings regarding wikis, blogs, and Twitter are less conclusive and significantly limited in number of studies conducted within the past 5 years. Overall, the findings provide preliminary support that computer-based technology influences student engagement, however, additional research is needed to confirm and build on these findings. We conclude the article by providing a list of recommendations for practice, with the intent of increasing understanding of how computer-based technology may be purposefully implemented to achieve the greatest gains in student engagement.

Introduction

The digital revolution has profoundly affected daily living, evident in the ubiquity of mobile devices and the seamless integration of technology into common tasks such as shopping, reading, and finding directions (Anderson, 2016 ; Smith & Anderson, 2016 ; Zickuhr & Raine, 2014 ). The use of computers, mobile devices, and the Internet is at its highest level to date and expected to continue to increase as technology becomes more accessible, particularly for users in developing countries (Poushter, 2016 ). In addition, there is a growing number of people who are smartphone dependent, relying solely on smartphones for Internet access (Anderson & Horrigan, 2016 ) rather than more expensive devices such as laptops and tablets. Greater access to and demand for technology has presented unique opportunities and challenges for many industries, some of which have thrived by effectively digitizing their operations and services (e.g., finance, media) and others that have struggled to keep up with the pace of technological innovation (e.g., education, healthcare) (Gandhi, Khanna, & Ramaswamy, 2016 ).

Integrating technology into teaching and learning is not a new challenge for universities. Since the 1900s, administrators and faculty have grappled with how to effectively use technical innovations such as video and audio recordings, email, and teleconferencing to augment or replace traditional instructional delivery methods (Kaware & Sain, 2015 ; Westera, 2015 ). Within the past two decades, however, this challenge has been much more difficult due to the sheer volume of new technologies on the market. For example, in the span of 7 years (from 2008 to 2015), the number of active apps in Apple’s App Store increased from 5000 to 1.75 million. Over the next 4 years, the number of apps is projected to rise by 73%, totaling over 5 million (Nelson, 2016 ). Further compounding this challenge is the limited shelf life of new devices and software combined with significant internal organizational barriers that hinder universities from efficiently and effectively integrating new technologies (Amirault, 2012 ; Kinchin, 2012 ; Linder-VanBerschot & Summers 2015 ; Westera, 2015 ).

Many organizational barriers to technology integration arise from competing tensions between institutional policy and practice and faculty beliefs and abilities. For example, university administrators may view technology as a tool to attract and retain students, whereas faculty may struggle to determine how technology coincides with existing pedagogy (Lawrence & Lentle-Keenan, 2013 ; Lin, Singer, & Ha, 2010 ). In addition, some faculty may be hesitant to use technology due to lack of technical knowledge and/or skepticism about the efficacy of technology to improve student learning outcomes (Ashrafzadeh & Sayadian, 2015 ; Buchanan, Sainter, & Saunders, 2013 ; Hauptman, 2015 ; Johnson, 2013 ; Kidd, Davis, & Larke, 2016 ; Kopcha, Rieber, & Walker, 2016 ; Lawrence & Lentle-Keenan, 2013 ; Lewis, Fretwell, Ryan, & Parham, 2013 ; Reid, 2014 ). Organizational barriers to technology adoption are particularly problematic given the growing demands and perceived benefits among students about using technology to learn (Amirault, 2012 ; Cassidy et al., 2014 ; Gikas & Grant, 2013 ; Paul & Cochran, 2013 ). Surveys suggest that two-thirds of students use mobile devices for learning and believe that technology can help them achieve learning outcomes and better prepare them for a workforce that is increasingly dependent on technology (Chen, Seilhamer, Bennett, & Bauer, 2015 ; Dahlstrom, 2012 ). Universities that fail to effectively integrate technology into the learning experience miss opportunities to improve student outcomes and meet the expectations of a student body that has grown accustomed to the integration of technology into every facet of life (Amirault, 2012 ; Cook & Sonnenberg, 2014 ; Revere & Kovach, 2011 ; Sun & Chen, 2016 ; Westera, 2015 ).

The purpose of this paper is to provide a literature review on how computer-based technology influences student engagement within higher education settings. We focused on computer-based technology given the specific types of technologies (i.e., web-conferencing software, blogs, wikis, social networking sites, and digital games) that emerged from a broad search of the literature, which is described in more detail below. Computer-based technology (hereafter referred to as technology) requires the use of specific hardware, software, and micro processing features available on a computer or mobile device. We also focused on student engagement as the dependent variable of interest because it encompasses many different aspects of the teaching and learning process (Bryson & Hand, 2007 ; Fredricks, Blumenfeld, & Parks, 1994; Wimpenny & Savin-Baden, 2013 ), compared narrower variables in the literature such as final grades or exam scores. Furthermore, student engagement has received significant attention over the past several decades due to shifts towards student-centered, constructivist instructional methods (Haggis, 2009 ; Wright, 2011 ), mounting pressures to improve teaching and learning outcomes (Axelson & Flick, 2011 ; Kuh, 2009 ), and promising studies suggesting relationships between student engagement and positive academic outcomes (Carini, Kuh, & Klein, 2006 ; Center for Postsecondary Research, 2016 ; Hu & McCormick, 2012 ). Despite the interest in student engagement and the demand for more technology in higher education, there are no articles offering a comprehensive review of how these two variables intersect. Similarly, while many existing student engagement conceptual models have expanded to include factors that influence student engagement, none highlight the overt role of technology in the engagement process (Kahu, 2013 ; Lam, Wong, Yang, & Yi, 2012 ; Nora, Barlow, & Crisp, 2005 ; Wimpenny & Savin-Baden, 2013 ; Zepke & Leach, 2010 ).

Our review aims to address existing gaps in the student engagement literature and seeks to determine whether student engagement models should be expanded to include technology. The review also addresses some of the organizational barriers to technology integration (e.g., faculty uncertainty and skepticism about technology) by providing a comprehensive account of the research evidence regarding how technology influences student engagement. One limitation of the literature, however, is the lack of detail regarding how teaching and learning practices were used to select and integrate technology into learning. For example, the methodology section of many studies does not include a pedagogical justification for why a particular technology was used or details about the design of the learning activity itself. Therefore, it often is unclear how teaching and learning practices may have affected student engagement levels. We revisit this issue in more detail at the end of this paper in our discussions of areas for future research and recommendations for practice. We initiated our literature review by conducting a broad search for articles published within the past 5 years, using the key words technology and higher education , in Google Scholar and the following research databases: Academic Search Complete, Communication & Mass Media Complete, Computers & Applied Sciences Complete, Education Research Complete, ERIC, PsycARTICLES, and PsycINFO . Our initial search revealed themes regarding which technologies were most prevalent in the literature (e.g., social networking, digital games), which then lead to several, more targeted searches of the same databases using specific keywords such as Facebook and student engagement. After both broad and targeted searches, we identified five technologies (web-conferencing software, blogs, wikis, social networking sites, and digital games) to include in our review.

We chose to focus on technologies for which there were multiple studies published, allowing us to identify areas of convergence and divergence in the literature and draw conclusions about positive and negative effects on student engagement. In total, we identified 69 articles relevant to our review, with 36 pertaining to social networking sites (21 for Facebook and 15 for Twitter ), 14 pertaining to digital games, seven pertaining to wikis, and six pertaining to blogs and web-conferencing software respectively. Articles were categorized according to their influence on specific types of student engagement, which will be described in more detail below. In some instances, one article pertained to multiple types of engagement. In the sections that follow, we will provide an overview of student engagement, including an explanation of common definitions and indicators of engagement, followed by a synthesis of how each type of technology influences student engagement. Finally, we will discuss areas for future research and make recommendations for practice.

• Student engagement

Interest in student engagement began over 70 years ago with Ralph Tyler’s research on the relationship between time spent on coursework and learning (Axelson & Flick, 2011 ; Kuh, 2009 ). Since then, the study of student engagement has evolved and expanded considerably, through the seminal works of Pace ( 1980 ; 1984 ) and Astin ( 1984 ) about how quantity and quality of student effort affect learning and many more recent studies on the environmental conditions and individual dispositions that contribute to student engagement (Bakker, Vergel, & Kuntze, 2015 ; Gilboy, Heinerichs, & Pazzaglia, 2015 ; Martin, Goldwasser, & Galentino, 2017 ; Pellas, 2014 ). Perhaps the most well-known resource on student engagement is the National Survey of Student Engagement (NSSE), an instrument designed to assess student participation in various educational activities (Kuh, 2009 ). The NSSE and other engagement instruments like it have been used in many studies that link student engagement to positive student outcomes such as higher grades, retention, persistence, and completion (Leach, 2016 ; McClenney, Marti, & Adkins, 2012 ; Trowler & Trowler, 2010 ), further convincing universities that student engagement is an important factor in the teaching and learning process. However, despite the increased interest in student engagement, its meaning is generally not well understood or agreed upon.

Student engagement is a broad and complex phenomenon for which there are many definitions grounded in psychological, social, and/or cultural perspectives (Fredricks et al., 1994; Wimpenny & Savin-Baden, 2013 ; Zepke & Leach, 2010 ). Review of definitions revealed that student engagement is defined in two ways. One set of definitions refer to student engagement as a desired outcome reflective of a student’s thoughts, feelings, and behaviors about learning. For example, Kahu ( 2013 ) defines student engagement as an “individual psychological state” that includes a student’s affect, cognition, and behavior (p. 764). Other definitions focus primarily on student behavior, suggesting that engagement is the “extent to which students are engaging in activities that higher education research has shown to be linked with high-quality learning outcomes” (Krause & Coates, 2008 , p. 493) or the “quality of effort and involvement in productive learning activities” (Kuh, 2009 , p. 6). Another set of definitions refer to student engagement as a process involving both the student and the university. For example, Trowler ( 2010 ) defined student engagement as “the interaction between the time, effort and other relevant resources invested by both students and their institutions intended to optimize the student experience and enhance the learning outcomes and development of students and the performance, and reputation of the institution” (p. 2). Similarly, the NSSE website indicates that student engagement is “the amount of time and effort students put into their studies and other educationally purposeful activities” as well as “how the institution deploys its resources and organizes the curriculum and other learning opportunities to get students to participate in activities that decades of research studies show are linked to student learning” (Center for Postsecondary Research, 2017 , para. 1).

Many existing models of student engagement reflect the latter set of definitions, depicting engagement as a complex, psychosocial process involving both student and university characteristics. Such models organize the engagement process into three areas: factors that influence student engagement (e.g., institutional culture, curriculum, and teaching practices), indicators of student engagement (e.g., interest in learning, interaction with instructors and peers, and meaningful processing of information), and outcomes of student engagement (e.g., academic achievement, retention, and personal growth) (Kahu, 2013 ; Lam et al., 2012 ; Nora et al., 2005 ). In this review, we examine the literature to determine whether technology influences student engagement. In addition, we will use Fredricks et al. ( 2004 ) typology of student engagement to organize and present research findings, which suggests that there are three types of engagement (behavioral, emotional, and cognitive). The typology is useful because it is broad in scope, encompassing different types of engagement that capture a range of student experiences, rather than narrower typologies that offer specific or prescriptive conceptualizations of student engagement. In addition, this typology is student-centered, focusing exclusively on student-focused indicators rather than combining student indicators with confounding variables, such as faculty behavior, curriculum design, and campus environment (Coates, 2008 ; Kuh, 2009 ). While such variables are important in the discussion of student engagement, perhaps as factors that may influence engagement, they are not true indicators of student engagement. Using the typology as a guide, we examined recent student engagement research, models, and measures to gain a better understanding of how behavioral, emotional, and cognitive student engagement are conceptualized and to identify specific indicators that correspond with each type of engagement, as shown in Fig. 1 .

Conceptual framework of types and indicators of student engagement

Behavioral engagement is the degree to which students are actively involved in learning activities (Fredricks et al., 2004 ; Kahu, 2013 ; Zepke, 2014 ). Indicators of behavioral engagement include time and effort spent participating in learning activities (Coates, 2008 ; Fredricks et al., 2004 ; Kahu, 2013 ; Kuh, 2009 ; Lam et al., 2012 ; Lester, 2013 ; Trowler, 2010 ) and interaction with peers, faculty, and staff (Coates, 2008 ; Kahu, 2013 ; Kuh, 2009 ; Bryson & Hand, 2007 ; Wimpenny & Savin-Baden, 2013 : Zepke & Leach, 2010 ). Indicators of behavioral engagement reflect observable student actions and most closely align with Pace ( 1980 ) and Astin’s ( 1984 ) original conceptualizations of student engagement as quantity and quality of effort towards learning. Emotional engagement is students’ affective reactions to learning (Fredricks et al., 2004 ; Lester, 2013 ; Trowler, 2010 ). Indicators of emotional engagement include attitudes, interests, and values towards learning (Fredricks et al., 2004 ; Kahu, 2013 ; Lester, 2013 ; Trowler, 2010 ; Wimpenny & Savin-Baden, 2013 ; Witkowski & Cornell, 2015 ) and a perceived sense of belonging within a learning community (Fredricks et al., 2004 ; Kahu, 2013 ; Lester, 2013 ; Trowler, 2010 ; Wimpenny & Savin-Baden, 2013 ). Emotional engagement often is assessed using self-report measures (Fredricks et al., 2004 ) and provides insight into how students feel about a particular topic, delivery method, or instructor. Finally, cognitive engagement is the degree to which students invest in learning and expend mental effort to comprehend and master content (Fredricks et al., 2004 ; Lester, 2013 ). Indicators of cognitive engagement include: motivation to learn (Lester, 2013 ; Richardson & Newby, 2006 ; Zepke & Leach, 2010 ); persistence to overcome academic challenges and meet/exceed requirements (Fredricks et al., 2004 ; Kuh, 2009 ; Trowler, 2010 ); and deep processing of information (Fredricks et al., 2004 ; Kahu, 2013 ; Lam et al., 2012 ; Richardson & Newby, 2006 ) through critical thinking (Coates, 2008 ; Witkowski & Cornell, 2015 ), self-regulation (e.g., set goals, plan, organize study effort, and monitor learning; Fredricks et al., 2004 ; Lester, 2013 ), and the active construction of knowledge (Coates, 2008 ; Kuh, 2009 ). While cognitive engagement includes motivational aspects, much of the literature focuses on how students use active learning and higher-order thinking, in some form, to achieve content mastery. For example, there is significant emphasis on the importance of deep learning, which involves analyzing new learning in relation previous knowledge, compared to surface learning, which is limited to memorization, recall, and rehearsal (Fredricks et al., 2004 ; Kahu, 2013 ; Lam et al., 2012 ).

While each type of engagement has distinct features, there is some overlap across cognitive, behavioral, and emotional domains. In instances where an indicator could correspond with more than one type of engagement, we chose to match the indicator to the type of engagement that most closely aligned, based on our review of the engagement literature and our interpretation of the indicators. Similarly, there is also some overlap among indicators. As a result, we combined and subsumed similar indicators found in the literature, where appropriate, to avoid redundancy. Achieving an in-depth understanding of student engagement and associated indicators was an important pre-cursor to our review of the technology literature. Very few articles used the term student engagement as a dependent variable given the concept is so broad and multidimensional. We found that specific indicators (e.g., interaction, sense of belonging, and knowledge construction) of student engagement were more common in the literature as dependent variables. Next, we will provide a synthesis of the findings regarding how different types of technology influence behavioral, emotional, and cognitive student engagement and associated indicators.

Influence of technology on student engagement

We identified five technologies post-literature search (i.e., web-conferencing, blogs, wikis, social networking sites , and digital games) to include in our review, based on frequency in which they appeared in the literature over the past 5 years. One commonality among these technologies is their potential value in supporting a constructivist approach to learning, characterized by the active discovery of knowledge through reflection of experiences with one’s environment, the connection of new knowledge to prior knowledge, and interaction with others (Boghossian, 2006 ; Clements, 2015 ). Another commonality is that most of the technologies, except perhaps for digital games, are designed primarily to promote interaction and collaboration with others. Our search yielded very few studies on how informational technologies, such as video lectures and podcasts, influence student engagement. Therefore, these technologies are notably absent from our review. Unlike the technologies we identified earlier, informational technologies reflect a behaviorist approach to learning in which students are passive recipients of knowledge that is transmitted from an expert (Boghossian, 2006 ). The lack of recent research on how informational technologies affect student engagement may be due to the increasing shift from instructor-centered, behaviorist approaches to student-centered, constructivist approaches within higher education (Haggis, 2009 ; Wright, 2011 ) along with the ubiquity of web 2.0 technologies.

• Web-conferencing

Web-conferencing software provides a virtual meeting space where users login simultaneously and communicate about a given topic. While each software application is unique, many share similar features such as audio, video, or instant messaging options for real-time communication; screen sharing, whiteboards, and digital pens for presentations and demonstrations; polls and quizzes for gauging comprehension or eliciting feedback; and breakout rooms for small group work (Bower, 2011 ; Hudson, Knight, & Collins, 2012 ; Martin, Parker, & Deale, 2012 ; McBrien, Jones, & Cheng, 2009 ). Of the technologies included in this literature review, web-conferencing software most closely mimics the face-to-face classroom environment, providing a space where instructors and students can hear and see each other in real-time as typical classroom activities (i.e., delivering lectures, discussing course content, asking/answering questions) are carried out (Francescucci & Foster, 2013 ; Hudson et al., 2012 ). Studies on web-conferencing software deployed Adobe Connect, Cisco WebEx, Horizon Wimba, or Blackboard Collaborate and made use of multiple features, such as screen sharing, instant messaging, polling, and break out rooms. In addition, most of the studies integrated web-conferencing software into courses on a voluntary basis to supplement traditional instructional methods (Andrew, Maslin-Prothero, & Ewens, 2015 ; Armstrong & Thornton, 2012 ; Francescucci & Foster, 2013 ; Hudson et al., 2012 ; Martin et al., 2012 ; Wdowik, 2014 ). Existing studies on web-conferencing pertain to all three types of student engagement.

Studies on web-conferencing and behavioral engagement reveal mixed findings. For example, voluntary attendance in web-conferencing sessions ranged from 54 to 57% (Andrew et al., 2015 ; Armstrong & Thornton, 2012 ) and, in a comparison between a blended course with regular web-conferencing sessions and a traditional, face-to-face course, researchers found no significant difference in student attendance in courses. However, students in the blended course reported higher levels of class participation compared to students in the face-to-face course (Francescucci & Foster, 2013 ). These findings suggest while web-conferencing may not boost attendance, especially if voluntary, it may offer more opportunities for class participation, perhaps through the use of communication channels typically not available in a traditional, face-to-face course (e.g., instant messaging, anonymous polling). Studies on web-conferencing and interaction, another behavioral indicator, support this assertion. For example, researchers found that students use various features of web-conferencing software (e.g., polling, instant message, break-out rooms) to interact with peers and the instructor by asking questions, expressing opinions and ideas, sharing resources, and discussing academic content (Andrew et al., 2015 ; Armstrong & Thornton, 2012 ; Hudson et al., 2012 ; Martin et al., 2012 ; Wdowik, 2014 ).

Studies on web-conferencing and cognitive engagement are more conclusive than those for behavioral engagement, although are fewer in number. Findings suggest that students who participated in web-conferencing demonstrated critical reflection and enhanced learning through interactions with others (Armstrong & Thornton, 2012 ), higher-order thinking (e.g., problem-solving, synthesis, evaluation) in response to challenging assignments (Wdowik, 2014 ), and motivation to learn, particularly when using polling features (Hudson et al., 2012 ). There is only one study examining how web-conferencing affects emotional engagement, although it is positive suggesting that students who participated in web-conferences had higher levels of interest in course content than those who did not (Francescucci & Foster, 2013 ). One possible reason for the positive cognitive and emotional engagement findings may be that web-conferencing software provides many features that promote active learning. For example, whiteboards and breakout rooms provide opportunities for real-time, collaborative problem-solving activities and discussions. However, additional studies are needed to isolate and compare specific web-conferencing features to determine which have the greatest effect on student engagement.

A blog, which is short for Weblog, is a collection of personal journal entries, published online and presented chronologically, to which readers (or subscribers) may respond by providing additional commentary or feedback. In order to create a blog, one must compose content for an entry, which may include text, hyperlinks, graphics, audio, or video, publish the content online using a blogging application, and alert subscribers that new content is posted. Blogs may be informal and personal in nature or may serve as formal commentary in a specific genre, such as in politics or education (Coghlan et al., 2007 ). Fortunately, many blog applications are free, and many learning management systems (LMSs) offer a blogging feature that is seamlessly integrated into the online classroom. The ease of blogging has attracted attention from educators, who currently use blogs as an instructional tool for the expression of ideas, opinions, and experiences and for promoting dialogue on a wide range of academic topics (Garrity, Jones, VanderZwan, de la Rocha, & Epstein, 2014 ; Wang, 2008 ).

Studies on blogs show consistently positive findings for many of the behavioral and emotional engagement indicators. For example, students reported that blogs promoted interaction with others, through greater communication and information sharing with peers (Chu, Chan, & Tiwari, 2012 ; Ivala & Gachago, 2012 ; Mansouri & Piki, 2016 ), and analyses of blog posts show evidence of students elaborating on one another’s ideas and sharing experiences and conceptions of course content (Sharma & Tietjen, 2016 ). Blogs also contribute to emotional engagement by providing students with opportunities to express their feelings about learning and by encouraging positive attitudes about learning (Dos & Demir, 2013 ; Chu et al., 2012 ; Yang & Chang, 2012 ). For example, Dos and Demir ( 2013 ) found that students expressed prejudices and fears about specific course topics in their blog posts. In addition, Yang and Chang ( 2012 ) found that interactive blogging, where comment features were enabled, lead to more positive attitudes about course content and peers compared to solitary blogging, where comment features were disabled.

The literature on blogs and cognitive engagement is less consistent. Some studies suggest that blogs may help students engage in active learning, problem-solving, and reflection (Chawinga, 2017 ; Chu et al., 2012 ; Ivala & Gachago, 2012 ; Mansouri & Piki, 2016 ), while other studies suggest that students’ blog posts show very little evidence of higher-order thinking (Dos & Demir, 2013 ; Sharma & Tietjen, 2016 ). The inconsistency in findings may be due to the wording of blog instructions. Students may not necessarily demonstrate or engage in deep processing of information unless explicitly instructed to do so. Unfortunately, it is difficult to determine whether the wording of blog assignments contributed to the mixed results because many of the studies did not provide assignment details. However, studies pertaining to other technologies suggest that assignment wording that lacks specificity or requires low-level thinking can have detrimental effects on student engagement outcomes (Hou, Wang, Lin, & Chang, 2015 ; Prestridge, 2014 ). Therefore, blog assignments that are vague or require only low-level thinking may have adverse effects on cognitive engagement.

A wiki is a web page that can be edited by multiple users at once (Nakamaru, 2012 ). Wikis have gained popularity in educational settings as a viable tool for group projects where group members can work collaboratively to develop content (i.e., writings, hyperlinks, images, graphics, media) and keep track of revisions through an extensive versioning system (Roussinos & Jimoyiannis, 2013 ). Most studies on wikis pertain to behavioral engagement, with far fewer studies on cognitive engagement and none on emotional engagement. Studies pertaining to behavioral engagement reveal mixed results, with some showing very little enduring participation in wikis beyond the first few weeks of the course (Nakamaru, 2012 ; Salaber, 2014 ) and another showing active participation, as seen in high numbers of posts and edits (Roussinos & Jimoyiannis, 2013 ). The most notable difference between these studies is the presence of grading, which may account for the inconsistencies in findings. For example, in studies where participation was low, wikis were ungraded, suggesting that students may need extra motivation and encouragement to use wikis (Nakamaru, 2012 ; Salaber, 2014 ). Findings regarding the use of wikis for promoting interaction are also inconsistent. In some studies, students reported that wikis were useful for interaction, teamwork, collaboration, and group networking (Camacho, Carrión, Chayah, & Campos, 2016 ; Martínez, Medina, Albalat, & Rubió, 2013 ; Morely, 2012 ; Calabretto & Rao, 2011 ) and researchers found evidence of substantial collaboration among students (e.g., sharing ideas, opinions, and points of view) in wiki activity (Hewege & Perera, 2013 ); however, Miller, Norris, and Bookstaver ( 2012 ) found that only 58% of students reported that wikis promoted collegiality among peers. The findings in the latter study were unexpected and may be due to design flaws in the wiki assignments. For example, the authors noted that wiki assignments were not explicitly referred to in face-to-face classes; therefore, this disconnect may have prevented students from building on interactive momentum achieved during out-of-class wiki assignments (Miller et al., 2012 ).

Studies regarding cognitive engagement are limited in number but more consistent than those concerning behavioral engagement, suggesting that wikis promote high levels of knowledge construction (i.e., evaluation of arguments, the integration of multiple viewpoints, new understanding of course topics; Hewege & Perera, 2013 ), and are useful for reflection, reinforcing course content, and applying academic skills (Miller et al., 2012 ). Overall, there is mixed support for the use of wikis to promote behavioral engagement, although making wiki assignments mandatory and explicitly referring to wikis in class may help bolster participation and interaction. In addition, there is some support for using wikis to promote cognitive engagement, but additional studies are needed to confirm and expand on findings as well as explore the effect of wikis on emotional engagement.

Social networking sites

Social networking is “the practice of expanding knowledge by making connections with individuals of similar interests” (Gunawardena et al., 2009 , p. 4). Social networking sites, such as Facebook, Twitter, Instagram, and LinkedIn, allow users to create and share digital content publicly or with others to whom they are connected and communicate privately through messaging features. Two of the most popular social networking sites in the educational literature are Facebook and Twitter (Camus, Hurt, Larson, & Prevost, 2016 ; Manca & Ranieri, 2013 ), which is consistent with recent statistics suggesting that both sites also are exceedingly popular among the general population (Greenwood, Perrin, & Duggan, 2016 ). In the sections that follow, we examine how both Facebook and Twitter influence different types of student engagement.

Facebook is a web-based service that allows users to create a public or private profile and invite others to connect. Users may build social, academic, and professional connections by posting messages in various media formats (i.e., text, pictures, videos) and commenting on, liking, and reacting to others’ messages (Bowman & Akcaoglu, 2014 ; Maben, Edwards, & Malone, 2014 ; Hou et al., 2015 ). Within an educational context, Facebook has often been used as a supplementary instructional tool to lectures or LMSs to support class discussions or develop, deliver, and share academic content and resources. Many instructors have opted to create private Facebook groups, offering an added layer of security and privacy because groups are not accessible to strangers (Bahati, 2015 ; Bowman & Akcaoglu, 2014 ; Clements, 2015 ; Dougherty & Andercheck, 2014 ; Esteves, 2012 ; Shraim, 2014 ; Maben et al., 2014 ; Manca & Ranieri, 2013 ; Naghdipour & Eldridge, 2016 ; Rambe, 2012 ). The majority of studies on Facebook address behavioral indicators of student engagement, with far fewer focusing on emotional or cognitive engagement.

Studies that examine the influence of Facebook on behavioral engagement focus both on participation in learning activities and interaction with peers and instructors. In most studies, Facebook activities were voluntary and participation rates ranged from 16 to 95%, with an average of rate of 47% (Bahati, 2015 ; Bowman & Akcaoglu, 2014 ; Dougherty & Andercheck, 2014 ; Fagioli, Rios-Aguilar, & Deil-Amen, 2015 ; Rambe, 2012 ; Staines & Lauchs, 2013 ). Participation was assessed by tracking how many students joined course- or university-specific Facebook groups (Bahati, 2015 ; Bowman & Akcaoglu, 2014 ; Fagioli et al., 2015 ), visited or followed course-specific Facebook pages (DiVall & Kirwin, 2012 ; Staines & Lauchs, 2013 ), or posted at least once in a course-specific Facebook page (Rambe, 2012 ). The lowest levels of participation (16%) arose from a study where community college students were invited to use the Schools App, a free application that connects students to their university’s private Facebook community. While the authors acknowledged that building an online community of college students is difficult (Fagioli et al., 2015 ), downloading the Schools App may have been a deterrent to widespread participation. In addition, use of the app was not tied to any specific courses or assignments; therefore, students may have lacked adequate incentive to use it. The highest level of participation (95%) in the literature arose from a study in which the instructor created a Facebook page where students could find or post study tips or ask questions. Followership to the page was highest around exams, when students likely had stronger motivations to access study tips and ask the instructor questions (DiVall & Kirwin, 2012 ). The wide range of participation in Facebook activities suggests that some students may be intrinsically motivated to participate, while other students may need some external encouragement. For example, Bahati ( 2015 ) found that when students assumed that a course-specific Facebook was voluntary, only 23% participated, but when the instructor confirmed that the Facebook group was, in fact, mandatory, the level of participation rose to 94%.

While voluntary participation in Facebook activities may be lower than desired or expected (Dyson, Vickers, Turtle, Cowan, & Tassone, 2015 ; Fagioli et al., 2015 ; Naghdipour & Eldridge, 2016 ; Rambe, 2012 ), students seem to have a clear preference for Facebook compared to other instructional tools (Clements, 2015 ; DiVall & Kirwin, 2012 ; Hurt et al., 2012 ; Hou et al., 2015 ; Kent, 2013 ). For example, in one study where an instructor shared course-related information in a Facebook group, in the LMS, and through email, the level of participation in the Facebook group was ten times higher than in email or the LMS (Clements, 2015 ). In other studies, class discussions held in Facebook resulted in greater levels of participation and dialogue than class discussions held in LMS discussion forums (Camus et al., 2016 ; Hurt et al., 2012 ; Kent, 2013 ). Researchers found that preference for Facebook over the university’s LMS is due to perceptions that the LMS is outdated and unorganized and reports that Facebook is more familiar, convenient, and accessible given that many students already visit the social networking site multiple times per day (Clements, 2015 ; Dougherty & Andercheck, 2014 ; Hurt et al., 2012 ; Kent, 2013 ). In addition, students report that Facebook helps them stay engaged in learning through collaboration and interaction with both peers and instructors (Bahati, 2015 ; Shraim, 2014 ), which is evident in Facebook posts where students collaborated to study for exams, consulted on technical and theoretical problem solving, discussed course content, exchanged learning resources, and expressed opinions as well as academic successes and challenges (Bowman & Akcaoglu, 2014 ; Dougherty & Andercheck, 2014 ; Esteves, 2012 Ivala & Gachago, 2012 ; Maben et al., 2014 ; Rambe, 2012 ; van Beynen & Swenson, 2016 ).

There is far less evidence in the literature about the use of Facebook for emotional and cognitive engagement. In terms of emotional engagement, studies suggest that students feel positively about being part of a course-specific Facebook group and that Facebook is useful for expressing feelings about learning and concerns for peers, through features such as the “like” button and emoticons (Bowman & Akcaoglu, 2014 ; Dougherty & Andercheck, 2014 ; Naghdipour & Eldridge, 2016 ). In addition, being involved in a course-specific Facebook group was positively related to students’ sense of belonging in the course (Dougherty & Andercheck, 2014 ). The research on cognitive engagement is less conclusive, with some studies suggesting that Facebook participation is related to academic persistence (Fagioli et al., 2015 ) and self-regulation (Dougherty & Andercheck, 2014 ) while other studies show low levels of knowledge construction in Facebook posts (Hou et al., 2015 ), particularly when compared to discussions held in the LMS. One possible reason may be because the LMS is associated with formal, academic interactions while Facebook is associated with informal, social interactions (Camus et al., 2016 ). While additional research is needed to confirm the efficacy of Facebook for promoting cognitive engagement, studies suggest that Facebook may be a viable tool for increasing specific behavioral and emotional engagement indicators, such as interactions with others and a sense of belonging within a learning community.

Twitter is a web-based service where subscribers can post short messages, called tweets, in real-time that are no longer than 140 characters in length. Tweets may contain hyperlinks to other websites, images, graphics, and/or videos and may be tagged by topic using the hashtag symbol before the designated label (e.g., #elearning). Twitter subscribers may “follow” other users and gain access to their tweets and also may “retweet” messages that have already been posted (Hennessy, Kirkpatrick, Smith, & Border, 2016 ; Osgerby & Rush, 2015 ; Prestridge, 2014 ; West, Moore, & Barry, 2015 ; Tiernan, 2014 ;). Instructors may use Twitter to post updates about the course, clarify expectations, direct students to additional learning materials, and encourage students to discuss course content (Bista, 2015 ; Williams & Whiting, 2016 ). Several of the studies on the use of Twitter included broad, all-encompassing measures of student engagement and produced mixed findings. For example, some studies suggest that Twitter increases student engagement (Evans, 2014 ; Gagnon, 2015 ; Junco, Heibergert, & Loken, 2011 ) while other studies suggest that Twitter has little to no influence on student engagement (Junco, Elavsky, & Heiberger, 2013 ; McKay, Sanko, Shekhter, & Birnbach, 2014 ). In both studies suggesting little to no influence on student engagement, Twitter use was voluntary and in one of the studies faculty involvement in Twitter was low, which may account for the negative findings (Junco et al., 2013 ; McKay et al., 2014 ). Conversely, in the studies that show positive findings, Twitter use was mandatory and often directly integrated with required assignments (Evans, 2014 ; Gagnon, 2015 ; Junco et al., 2011 ). Therefore, making Twitter use mandatory, increasing faculty involvement in Twitter, and integrating Twitter into assignments may help to increase student engagement.

Studies pertaining to specific behavioral student engagement indicators also reveal mixed findings. For example, in studies where course-related Twitter use was voluntary, 45-91% of students reported using Twitter during the term (Hennessy et al., 2016 ; Junco et al., 2013 ; Ross, Banow, & Yu, 2015 ; Tiernan, 2014 ; Williams & Whiting, 2016 ), but only 30-36% reported making contributions to the course-specific Twitter page (Hennessy et al., 2016 ; Tiernan, 2014 ; Ross et al., 2015 ; Williams & Whiting, 2016 ). The study that reported a 91% participation rate was unique because the course-specific Twitter page was accessible via a public link. Therefore, students who chose only to view the content (58%), rather than contribute to the page, did not have to create a Twitter account (Hennessy et al., 2016 ). The convenience of not having to create an account may be one reason for much higher participation rates. In terms of low participation rates, a lack of literacy, familiarity, and interest in Twitter , as well as a preference for Facebook , are cited as contributing factors (Bista, 2015 ; McKay et al., 2014 ; Mysko & Delgaty, 2015 ; Osgerby & Rush, 2015 ; Tiernan, 2014 ). However, when the use of Twitter was required and integrated into class discussions, the participation rate was 100% (Gagnon, 2015 ). Similarly, 46% of students in one study indicated that they would have been more motivated to participate in Twitter activities if they were graded (Osgerby & Rush, 2015 ), again confirming the power of extrinsic motivating factors.

Studies also show mixed results for the use of Twitter to promote interactions with peers and instructors. Researchers found that when instructors used Twitter to post updates about the course, ask and answer questions, and encourage students to tweet about course content, there was evidence of student-student and student-instructor interactions in tweets (Hennessy et al., 2016 ; Tiernan, 2014 ). Some students echoed these findings, suggesting that Twitter is useful for sharing ideas and resources, discussing course content, asking the instructor questions, and networking (Chawinga, 2017 ; Evans, 2014 ; Gagnon, 2015 ; Hennessy et al., 2016 ; Mysko & Delgaty, 2015 ; West et al., 2015 ) and is preferable over speaking aloud in class because it is more comfortable, less threatening, and more concise due to the 140 character limit (Gagnon, 2015 ; Mysko & Delgaty, 2015 ; Tiernan, 2014 ). Conversely, other students reported that Twitter was not useful for improving interaction because they viewed it predominately for social, rather than academic, interactions and they found the 140 character limit to be frustrating and restrictive. A theme among the latter studies was that a large proportion of the sample had never used Twitter before (Bista, 2015 ; McKay et al., 2014 ; Osgerby & Rush, 2015 ), which may have contributed to negative perceptions.

The literature on the use of Twitter for cognitive and emotional engagement is minimal but nonetheless promising in terms of promoting knowledge gains, the practical application of content, and a sense of belonging among users. For example, using Twitter to respond to questions that arose in lectures and tweet about course content throughout the term is associated with increased understanding of course content and application of knowledge (Kim et al., 2015 ; Tiernan, 2014 ; West et al., 2015 ). While the underlying mechanisms pertaining to why Twitter promotes an understanding of content and application of knowledge are not entirely clear, Tiernan ( 2014 ) suggests that one possible reason may be that Twitter helps to break down communication barriers, encouraging shy or timid students to participate in discussions that ultimately are richer in dialogue and debate. In terms of emotional engagement, students who participated in a large, class-specific Twitter page were more likely to feel a sense of community and belonging compared to those who did not participate because they could more easily find support from and share resources with other Twitter users (Ross et al., 2015 ). Despite the positive findings about the use of Twitter for cognitive and emotional engagement, more studies are needed to confirm existing results regarding behavioral engagement and target additional engagement indicators such as motivation, persistence, and attitudes, interests, and values about learning. In addition, given the strong negative perceptions of Twitter that still exist, additional studies are needed to confirm Twitter ’s efficacy for promoting different types of behavioral engagement among both novice and experienced Twitter users, particularly when compared to more familiar tools such as Facebook or LMS discussion forums.

• Digital games

Digital games are “applications using the characteristics of video and computer games to create engaging and immersive learning experiences for delivery of specified learning goals, outcomes and experiences” (de Freitas, 2006 , p. 9). Digital games often serve the dual purpose of promoting the achievement of learning outcomes while making learning fun by providing simulations of real-world scenarios as well as role play, problem-solving, and drill and repeat activities (Boyle et al., 2016 ; Connolly, Boyle, MacArthur, Hainey, & Boyle, 2012 ; Scarlet & Ampolos, 2013 ; Whitton, 2011 ). In addition, gamified elements, such as digital badges and leaderboards, may be integrated into instruction to provide additional motivation for completing assigned readings and other learning activities (Armier, Shepherd, & Skrabut, 2016 ; Hew, Huang, Chu, & Chiu, 2016 ). The pedagogical benefits of digital games are somewhat distinct from the other technologies addressed in this review, which are designed primarily for social interaction. While digital games may be played in teams or allow one player to compete against another, the focus of their design often is on providing opportunities for students to interact with academic content in a virtual environment through decision-making, problem-solving, and reward mechanisms. For example, a digital game may require students to adopt a role as CEO in a computer-simulated business environment, make decisions about a series of organizational issues, and respond to the consequences of those decisions. In this example and others, digital games use adaptive learning principles, where the learning environment is re-configured or modified in response to the actions and needs of students (Bower, 2016 ). Most of the studies on digital games focused on cognitive and emotional indicators of student engagement, in contrast to the previous technologies addressed in this review which primarily focused on behavioral indicators of engagement.

Existing studies provide support for the influence of digital games on cognitive engagement, through achieving a greater understanding of course content and demonstrating higher-order thinking skills (Beckem & Watkins, 2012 ; Farley, 2013 ; Ke, Xie, & Xie, 2016 ; Marriott, Tan, & Marriott, 2015 ), particularly when compared to traditional instructional methods, such as giving lectures or assigning textbook readings (Lu, Hallinger, & Showanasai, 2014 ; Siddique, Ling, Roberson, Xu, & Geng, 2013 ; Zimmermann, 2013 ). For example, in a study comparing courses that offered computer simulations of business challenges (e.g, implementing a new information technology system, managing a startup company, and managing a brand of medicine in a simulated market environment) and courses that did not, students in simulation-based courses reported higher levels of action-directed learning (i.e., connecting theory to practice in a business context) than students in traditional, non-simulation-based courses (Lu et al., 2014 ). Similarly, engineering students who participated in a car simulator game, which was designed to help students apply and reinforce the knowledge gained from lectures, demonstrated higher levels of critical thinking (i.e., analysis, evaluation) on a quiz than students who only attended lectures (Siddique et al., 2013 ).

Motivation is another cognitive engagement indicator that is linked to digital games (Armier et al., 2016 ; Chang & Wei, 2016 ; Dichev & Dicheva, 2017 ; Grimley, Green, Nilsen, & Thompson, 2012 ; Hew et al., 2016 ; Ibáñez, Di-Serio, & Delgado-Kloos, 2014 ; Ke et al., 2016 ; Liu, Cheng, & Huang, 2011 ; Nadolny & Halabi, 2016 ). Researchers found that incorporating gamified elements into courses, such as giving students digital rewards (e.g., redeemable points, trophies, and badges) for participating in learning activities or creating competition through the use of leaderboards where students can see how they rank against other students positively affects student motivation to complete learning tasks (Armier et al., 2016 ; Chang & Wei, 2016 ; Hew et al., 2016 ; Nadolny & Halabi, 2016 ). In addition, students who participated in gamified elements, such as trying to earn digital badges, were more motivated to complete particularly difficult learning activities (Hew et al., 2016 ) and showed persistence in exceeding learning requirements (Ibáñez et al., 2014 ). Research on emotional engagement may help to explain these findings. Studies suggest that digital games positively affect student attitudes about learning, evident in student reports that games are fun, interesting, and enjoyable (Beckem & Watkins, 2012 ; Farley, 2013 ; Grimley et al., 2012 ; Hew et al., 2016 ; Liu et al., 2011 ; Zimmermann, 2013 ), which may account for higher levels of student motivation in courses that offered digital games.

Research on digital games and behavioral engagement is more limited, with only one study suggesting that games lead to greater participation in educational activities (Hew et al., 2016 ). Therefore, more research is needed to explore how digital games may influence behavioral engagement. In addition, research is needed to determine whether the underlying technology associated with digital games (e.g., computer-based simulations and virtual realities) produce positive engagement outcomes or whether common mechanisms associated with both digital and non-digital games (e.g., role play, rewards, and competition) account for those outcomes. For example, studies in which non-digital, face-to-face games were used also showed positive effects on student engagement (Antunes, Pacheco, & Giovanela, 2012 ; Auman, 2011 ; Coffey, Miller, & Feuerstein, 2011 ; Crocco, Offenholley, & Hernandez, 2016 ; Poole, Kemp, Williams, & Patterson, 2014 ; Scarlet & Ampolos, 2013 ); therefore, it is unclear if and how digitizing games contributes to student engagement.

Discussion and implications

Student engagement is linked to a number of academic outcomes, such as retention, grade point average, and graduation rates (Carini et al., 2006 ; Center for Postsecondary Research, 2016 ; Hu & McCormick, 2012 ). As a result, universities have shown a strong interest in how to increase student engagement, particularly given rising external pressures to improve learning outcomes and prepare students for academic success (Axelson & Flick, 2011 ; Kuh, 2009 ). There are various models of student engagement that identify factors that influence student engagement (Kahu, 2013 ; Lam et al., 2012 ; Nora et al., 2005 ; Wimpenny & Savin-Baden, 2013 ; Zepke & Leach, 2010 ); however, none include the overt role of technology despite the growing trend and student demands to integrate technology into the learning experience (Amirault, 2012 ; Cook & Sonnenberg, 2014 ; Revere & Kovach, 2011 ; Sun & Chen, 2016 ; Westera, 2015 ). Therefore, the primary purpose of our literature review was to explore whether technology influences student engagement. The secondary purpose was to address skepticism and uncertainty about pedagogical benefits of technology (Ashrafzadeh & Sayadian, 2015 ; Kopcha et al., 2016 ; Reid, 2014 ) by reviewing the literature regarding the efficacy of specific technologies (i.e., web-conferencing software, blogs, wikis, social networking sites, and digital games) for promoting student engagement and offering recommendations for effective implementation, which are included at the end of this paper. In the sections that follow, we provide an overview of the findings, an explanation of existing methodological limitations and areas for future research, and a list of best practices for integrating the technologies we reviewed into the teaching and learning process.

Summary of findings

Findings from our literature review provide preliminary support for including technology as a factor that influences student engagement in existing models (Table 1 ). One overarching theme is that most of the technologies we reviewed had a positive influence on multiple indicators of student engagement, which may lead to a larger return on investment in terms of learning outcomes. For example, digital games influence all three types of student engagement and six of the seven indicators we identified, surpassing the other technologies in this review. There were several key differences in the design and pedagogical use between digital games and other technologies that may explain these findings. First, digital games were designed to provide authentic learning contexts in which students could practice skills and apply learning (Beckem & Watkins, 2012 ; Farley, 2013 ; Grimley et al., 2012 ; Ke et al., 2016 ; Liu et al., 2011 ; Lu et al., 2014 ; Marriott et al., 2015 ; Siddique et al., 2013 ), which is consistent with experiential learning and adult learning theories. Experiential learning theory suggests that learning occurs through interaction with one’s environment (Kolb, 2014 ) while adult learning theory suggests that adult learners want to be actively involved in the learning process and be able apply learning to real life situations and problems (Cercone, 2008 ). Second, students reported that digital games (and gamified elements) are fun, enjoyable, and interesting (Beckem & Watkins, 2012 ; Farley, 2013 ; Grimley et al., 2012 ; Hew et al., 2016 ; Liu et al., 2011 ; Zimmermann, 2013 ), feelings that are associated with a flow-like state where one is completely immersed in and engaged with the activity (Csikszentmihalyi, 1988 ; Weibel, Wissmath, Habegger, Steiner, & Groner, 2008 ). Third, digital games were closely integrated into the curriculum as required activities (Farley, 2013 ; Grimley et al., 2012 , Ke et al., 2016 ; Liu et al., 2011 ; Marriott et al., 2015 ; Siddique et al., 2013 ) as opposed to wikis, Facebook , and Twitter , which were often voluntary and used to supplement lectures (Dougherty & Andercheck, 2014 Nakamaru, 2012 ; Prestridge, 2014 ; Rambe, 2012 ).

Web-conferencing software and Facebook also yielded the most positive findings, influencing four of the seven indicators of student engagement, compared to other collaborative technologies, such as blogs, wikis, and Twitter . Web-conferencing software was unique due to the sheer number of collaborative features it offers, providing multiple ways for students to actively engage with course content (screen sharing, whiteboards, digital pens) and interact with peers and the instructor (audio, video, text chats, breakout rooms) (Bower, 2011 ; Hudson et al., 2012 ; Martin et al., 2012 ; McBrien et al., 2009 ); this may account for the effects on multiple indicators of student engagement. Positive findings regarding Facebook ’s influence on student engagement could be explained by a strong familiarity and preference for the social networking site (Clements, 2015 ; DiVall & Kirwin, 2012 ; Hurt et al., 2012 ; Hou et al., 2015 ; Kent, 2013 ; Manca & Ranieri, 2013 ), compared to Twitter which was less familiar or interesting to students (Bista, 2015 ; McKay et al., 2014 ; Mysko & Delgaty, 2015 ; Osgerby & Rush, 2015 ; Tiernan, 2014 ). Wikis had the lowest influence on student engagement, with mixed findings regarding behavioral engagement, limited, but conclusive findings, regarding one indicator of cognitive engagement (deep processing of information), and no studies pertaining to other indicators of cognitive engagement (motivation, persistence) or emotional engagement.

Another theme that arose was the prevalence of mixed findings across multiple technologies regarding behavioral engagement. Overall, the vast majority of studies addressed behavioral engagement, and we expected that technologies designed specifically for social interaction, such as web-conferencing, wikis, and social networking sites, would yield more conclusive findings. However, one possible reason for the mixed findings may be that the technologies were voluntary in many studies, resulting in lower than desired participation rates and missed opportunities for interaction (Armstrong & Thornton, 2012 ; Fagioli et al., 2015 ; Nakamaru, 2012 ; Rambe, 2012 ; Ross et al., 2015 ; Williams & Whiting, 2016 ), and mandatory in a few studies, yielding higher levels of participation and interaction (Bahati, 2015 ; Gagnon, 2015 ; Roussinos & Jimoyiannis, 2013 ). Another possible reason for the mixed findings is that measures of variables differed across studies. For example, in some studies participation meant that a student signed up for a Twitter account (Tiernan, 2014 ), used the Twitter account for class (Williams & Whiting, 2016 ), or viewed the course-specific Twitter page (Hennessy et al., 2016 ). The pedagogical uses of the technologies also varied considerably across studies, making it difficult to make comparisons. For example, Facebook was used in studies to share learning materials (Clements, 2015 ; Dyson et al., 2015 ), answer student questions about academic content or administrative issues (Rambe, 2012 ), prepare for upcoming exams and share study tips (Bowman & Akcaoglu, 2014 ; DiVall & Kirwin, 2012 ), complete group work (Hou et al., 2015 ; Staines & Lauchs, 2013 ), and discuss course content (Camus et al., 2016 ; Kent, 2013 ; Hurt et al., 2012 ). Finally, cognitive indicators (motivation and persistence) drew the fewest amount of studies, which suggests that research is needed to determine whether technologies affect these indicators.

Methodological limitations

While there appears to be preliminary support for the use of many of the technologies to promote student engagement, there are significant methodological limitations in the literature and, as a result, findings should be interpreted with caution. First, many studies used small sample sizes and were limited to one course, one degree level, and one university. Therefore, generalizability is limited. Second, very few studies used experimental or quasi-experimental designs; therefore, very little evidence exists to substantiate a cause and effect relationship between technologies and student engagement indicators. In addition, in many studies that did use experimental or quasi-experimental designs, participants were not randomized; rather, participants who volunteered to use a specific technology were compared to those who chose not to use the technology. As a result, there is a possibility that fundamental differences between users and non-users could have affected the engagement results. Furthermore, many of the studies did not isolate specific technological features (e.g, using only the breakout rooms for group work in web-conferencing software, rather than using the chat feature, screen sharing, and breakout rooms for group work). Using multiple features at once could have conflated student engagement results. Third, many studies relied on one source to measure technological and engagement variables (single source bias), such as self-report data (i.e., reported usage of technology and perceptions of student engagement), which may have affected the validity of the results. Fourth, many studies were conducted during a very brief timeframe, such as one academic term. As a result, positive student engagement findings may be attributed to a “novelty effect” (Dichev & Dicheva, 2017 ) associated with using a new technology. Finally, many studies lack adequate details about learning activities, raising questions about whether poor instructional design may have adversely affected results. For example, an instructor may intend to elicit higher-order thinking from students, but if learning activity instructions are written using low-level verbs, such as identify, describe, and summarize, students will be less likely to engage in higher-order thinking.

Areas for future research

The findings of our literature review suggest that the influence of technology on student engagement is still a developing area of knowledge that requires additional research to build on promising, but limited, evidence, clarify mixed findings, and address several gaps in the literature. As such, our recommendations for future areas of research are as follows:

Examine the effect of collaborative technologies (i.e., web-conferencing, blogs, wikis, social networking sites ) on emotional and cognitive student engagement. There are significant gaps in the literature regarding whether these technologies affect attitudes, interests, and values about learning; a sense of belonging within a learning community; motivation to learn; and persistence to overcome academic challenges and meet or exceed requirements.

Clarify mixed findings, particularly regarding how web-conferencing software, wikis, and Facebook and Twitter affect participation in learning activities. Researchers should make considerable efforts to gain consensus or increase consistency on how participation is measured (e.g., visited Facebook group or contributed one post a week) in order to make meaningful comparisons and draw conclusions about the efficacy of various technologies for promoting behavioral engagement. In addition, further research is needed to clarify findings regarding how wikis and Twitter influence interaction and how blogs and Facebook influence deep processing of information. Future research studies should include justifications for the pedagogical use of specific technologies and detailed instructions for learning activities to minimize adverse findings from poor instructional design and to encourage replication.

Conduct longitudinal studies over several academic terms and across multiple academic disciplines, degree levels, and institutions to determine long-term effects of specific technologies on student engagement and to increase generalizability of findings. Also, future studies should take individual factors into account, such as gender, age, and prior experience with the technology. Studies suggest that a lack of prior experience or familiarity with Twitter was a barrier to Twitter use in educational settings (Bista, 2015 , Mysko & Delgaty, 2015 , Tiernan, 2014 ); therefore, future studies should take prior experience into account.

Compare student engagement outcomes between and among different technologies and non-technologies. For example, studies suggest that students prefer Facebook over Twitter (Bista, 2015 ; Osgerby & Rush, 2015 ), but there were no studies that compared these technologies for promoting student engagement. Also, studies are needed to isolate and compare different features within the same technology to determine which might be most effective for increasing engagement. Finally, studies on digital games (Beckem & Watkins, 2012 ; Grimley et al., 2012 ; Ke et al., 2016 ; Lu et al., 2014 ; Marriott et al., 2015 ; Siddique et al., 2013 ) and face-to-face games (Antunes et al., 2012 ; Auman, 2011 ; Coffey et al., 2011 ; Crocco et al., 2016 ; Poole et al., 2014 ; Scarlet & Ampolos, 2013 ) show similar, positive effects on student engagement, therefore, additional research is needed to determine the degree to which the delivery method (i.e.., digital versus face-to-face) accounts for positive gains in student engagement.

Determine whether other technologies not included in this review influence student engagement. Facebook and Twitter regularly appear in the literature regarding social networking, but it is unclear how other popular social networking sites, such as LinkedIn, Instagram, and Flickr, influence student engagement. Future research should focus on the efficacy of these and other popular social networking sites for promoting student engagement. In addition, there were very few studies about whether informational technologies, which involve the one-way transmission of information to students, affect different types of student engagement. Future research should examine whether informational technologies, such as video lectures, podcasts, and pre-recorded narrated Power Point presentations or screen casts, affect student engagement. Finally, studies should examine the influence of mobile software and technologies, such as educational apps or smartphones, on student engagement.

Achieve greater consensus on the meaning of student engagement and its distinction from similar concepts in the literature, such as social and cognitive presence (Garrison & Arbaugh, 2007 )

Recommendations for practice

Despite the existing gaps and mixed findings in the literature, we were able to compile a list of recommendations for when and how to use technology to increase the likelihood of promoting student engagement. What follows is not an exhaustive list; rather, it is a synthesis of both research findings and lessons learned from the studies we reviewed. There may be other recommendations to add to this list; however, our intent is to provide some useful information to help address barriers to technology integration among faculty who feel uncertain or unprepared to use technology (Ashrafzadeh & Sayadian, 2015 ; Hauptman, 2015 ; Kidd et al., 2016 ; Reid, 2014 ) and to add to the body of practical knowledge in instructional design and delivery. Our recommendations for practice are as follows:

Consider context before selecting technologies. Contextual factors such as existing technological infrastructure and requirements, program and course characteristics, and the intended audience will help determine which technologies, if any, are most appropriate (Bullen & Morgan, 2011 ; Bullen, Morgan, & Qayyum, 2011 ). For example, requiring students to use a blog that is not well integrated with the existing LMS may prove too frustrating for both the instructor and students. Similarly, integrating Facebook- and Twitter- based learning activities throughout a marketing program may be more appropriate, given the subject matter, compared to doing so in an engineering or accounting program where social media is less integral to the profession. Finally, do not assume that students appreciate or are familiar with all technologies. For example, students who did not already have Facebook or Twitter accounts were less likely to use either for learning purposes and perceived setting up an account to be an increase in workload (Bista, 2015 , Clements, 2015 ; DiVall & Kirwin, 2012 ; Hennessy et al., 2016 ; Mysko & Delgaty, 2015 , Tiernan, 2014 ). Therefore, prior to using any technology, instructors may want to determine how many students already have accounts and/or are familiar with the technology.

Carefully select technologies based on their strengths and limitations and the intended learning outcome. For example, Twitter is limited to 140 characters, making it a viable tool for learning activities that require brevity. In one study, an instructor used Twitter for short pop quizzes during lectures, where the first few students to tweet the correct answer received additional points (Kim et al., 2015 ), which helped students practice applying knowledge. In addition, studies show that students perceive Twitter and Facebook to be primarily for social interactions (Camus et al., 2016 ; Ross et al., 2015 ), which may make these technologies viable tools for sharing resources, giving brief opinions about news stories pertaining to course content, or having casual conversations with classmates rather than full-fledged scholarly discourse.

Incentivize students to use technology, either by assigning regular grades or giving extra credit. The average participation rates in voluntary web-conferencing, Facebook , and Twitter learning activities in studies we reviewed was 52% (Andrew et al., 2015 ; Armstrong & Thornton, 2012 ; Bahati, 2015 ; Bowman & Akcaoglu, 2014 ; Divall & Kirwin, 2012 ; Dougherty & Andercheck, 2014 ; Fagioli et al., 2015 ; Hennessy et al., 2016 ; Junco et al., 2013 ; Rambe, 2012 ; Ross et al., 2015 ; Staines & Lauchs, 2013 ; Tiernan, 2014 ; Williams & Whiting, 2016 ). While there were far fewer studies on the use of technology for graded or mandatory learning activities, the average participation rate reported in those studies was 97% (Bahati2015; Gagnon, 2015 ), suggesting that grading may be a key factor in ensuring students participate.

Communicate clear guidelines for technology use. Prior to the implementation of technology in a course, students may benefit from an overview the technology, including its navigational features, privacy settings, and security (Andrew et al., 2015 ; Hurt et al., 2012 ; Martin et al., 2012 ) and a set of guidelines for how to use the technology effectively and professionally within an educational setting (Miller et al., 2012 ; Prestridge, 2014 ; Staines & Lauchs, 2013 ; West et al., 2015 ). In addition, giving students examples of exemplary and poor entries and posts may also help to clarify how they are expected to use the technology (Shraim, 2014 ; Roussinos & Jimoyiannis, 2013 ). Also, if instructors expect students to use technology to demonstrate higher-order thinking or to interact with peers, there should be explicit instructions to do so. For example, Prestridge ( 2014 ) found that students used Twitter to ask the instructor questions but very few interacted with peers because they were not explicitly asked to do so. Similarly, Hou et al., 2015 reported low levels of knowledge construction in Facebook , admitting that the wording of the learning activity (e.g., explore and present applications of computer networking) and the lack of probing questions in the instructions may have been to blame.

Use technology to provide authentic and integrated learning experiences. In many studies, instructors used digital games to simulate authentic environments in which students could apply new knowledge and skills, which ultimately lead to a greater understanding of content and evidence of higher-order thinking (Beckem & Watkins, 2012 ; Liu et al., 2011 ; Lu et al., 2014 ; Marriott et al., 2015 ; Siddique et al., 2013 ). For example, in one study, students were required to play the role of a stock trader in a simulated trading environment and they reported that the simulation helped them engage in critical reflection, enabling them to identify their mistakes and weaknesses in their trading approaches and strategies (Marriott et al., 2015 ). In addition, integrating technology into regularly-scheduled classroom activities, such as lectures, may help to promote student engagement. For example, in one study, the instructor posed a question in class, asked students to respond aloud or tweet their response, and projected the Twitter page so that everyone could see the tweets in class, which lead to favorable comments about the usefulness of Twitter to promote engagement (Tiernan, 2014 ).

Actively participate in using the technologies assigned to students during the first few weeks of the course to generate interest (Dougherty & Andercheck, 2014 ; West et al., 2015 ) and, preferably, throughout the course to answer questions, encourage dialogue, correct misconceptions, and address inappropriate behavior (Bowman & Akcaoglu, 2014 ; Hennessy et al., 2016 ; Junco et al., 2013 ; Roussinos & Jimoyiannis, 2013 ). Miller et al. ( 2012 ) found that faculty encouragement and prompting was associated with increases in students’ expression of ideas and the degree to which they edited and elaborated on their peers’ work in a course-specific wiki.

Be mindful of privacy, security, and accessibility issues. In many studies, instructors took necessary steps to help ensure privacy and security by creating closed Facebook groups and private Twitter pages, accessible only to students in the course (Bahati, 2015 ; Bista, 2015 ; Bowman & Akcaoglu, 2014 ; Esteves, 2012 ; Rambe, 2012 ; Tiernan, 2014 ; Williams & Whiting, 2016 ) and by offering training to students on how to use privacy and security settings (Hurt et al., 2012 ). Instructors also made efforts to increase accessibility of web-conferencing software by including a phone number for students unable to access audio or video through their computer and by recording and archiving sessions for students unable to attend due to pre-existing conflicts (Andrew et al., 2015 ; Martin et al., 2012 ). In the future, instructors should also keep in mind that some technologies, like Facebook and Twitter , are not accessible to students living in China; therefore, alternative arrangements may need to be made.

In 1985, Steve Jobs predicted that computers and software would revolutionize the way we learn. Over 30 years later, his prediction has yet to be fully confirmed in the student engagement literature; however, our findings offer preliminary evidence that the potential is there. Of the technologies we reviewed, digital games, web-conferencing software, and Facebook had the most far-reaching effects across multiple types and indicators of student engagement, suggesting that technology should be considered a factor that influences student engagement in existing models. Findings regarding blogs, wikis, and Twitter, however, are less convincing, given a lack of studies in relation to engagement indicators or mixed findings. Significant methodological limitations may account for the wide range of findings in the literature. For example, small sample sizes, inconsistent measurement of variables, lack of comparison groups, and missing details about specific, pedagogical uses of technologies threaten the validity and reliability of findings. Therefore, more rigorous and robust research is needed to confirm and build upon limited but positive findings, clarify mixed findings, and address gaps particularly regarding how different technologies influence emotional and cognitive indicators of engagement.

Abbreviations

Learning management system

Amirault, R. J. (2012). Distance learning in the 21 st century university. Quarterly Review of Distance Education, 13 (4), 253–265.

Google Scholar  

Anderson, M. (2016). More Americans using smartphones for getting directions, streaming TV . Washington, D.C.: Pew Research Center Retrieved from http://www.pewresearch.org/fact-tank/2016/01/29/us-smartphone-use/ .

Anderson, M., & Horrigan, J. B. (2016). Smartphones help those without broadband get online, but don’t necessary bridge the digital divide . Washington, D.C.: Pew Research Center Retrieved from http://www.pewresearch.org/fact-tank/2016/10/03/smartphones-help-those-without-broadband-get-online-but-dont-necessarily-bridge-the-digital-divide/ .

Andrew, L., Maslin-Prothero, S., & Ewens, B. (2015). Enhancing the online learning experience using virtual interactive classrooms. Australian Journal of Advanced Nursing, 32 (4), 22–31.

Antunes, M., Pacheco, M. R., & Giovanela, M. (2012). Design and implementation of an educational game for teaching chemistry in higher education. Journal of Chemical Education, 89 (4), 517–521. doi: 10.1021/ed2003077 .

Article   Google Scholar  

Armier, D. J., Shepherd, C. E., & Skrabut, S. (2016). Using game elements to increase student engagement in course assignments. College Teaching, 64 (2), 64–72 https://doi.org/10.1080/87567555.2015.1094439 .

Armstrong, A., & Thornton, N. (2012). Incorporating Brookfield’s discussion techniques synchronously into asynchronous online courses. Quarterly Review of Distance Education, 13 (1), 1–9.

Ashrafzadeh, A., & Sayadian, S. (2015). University instructors’ concerns and perceptions of technology integration. Computers in Human Behavior, 49 , 62–73. doi: 10.1016/j.chb.2015.01.071 .

Astin, A. W. (1984). Student involvement: A developmental theory for higher education. Journal of College Student Personnel, 25 (4), 297–308.

Auman, C. (2011). Using simulation games to increase student and instructor engagement. College Teaching, 59 (4), 154–161. doi: 10.1080/87567555 .

Axelson, R. D., & Flick, A. (2011). Defining student engagement. Change: The magazine of higher learning, 43 (1), 38–43.

Bahati, B. (2015). Extending student discussions beyond lecture room walls via Facebook. Journal of Education and Practice, 6 (15), 160–171.

Bakker, A. B., Vergel, A. I. S., & Kuntze, J. (2015). Student engagement and performance: A weekly diary study on the role of openness. Motivation and Emotion, 39 (1), 49–62. doi: 10.1007/s11031-014-9422-5 .

Beckem, J. I., & Watkins, M. (2012). Bringing life to learning: Immersive experiential learning simulations for online and blended courses. Journal if Asynchronous Learning Networks, 16 (5), 61–70 https://doi.org/10.24059/olj.v16i5.287 .

Bista, K. (2015). Is Twitter an effective pedagogical tool in higher education? Perspectives of education graduate students. Journal of the Scholarship Of Teaching And Learning, 15 (2), 83–102 https://doi.org/10.14434/josotl.v15i2.12825 .

Boghossian, P. (2006). Behaviorism, constructivism, and Socratic pedagogy. Educational Philosophy and Theory, 38 (6), 713–722 https://doi.org/10.1111/j.1469-5812.2006.00226.x .

Bower, M. (2011). Redesigning a web-conferencing environment to scaffold computing students’ creative design processes. Journal of Educational Technology & Society, 14 (1), 27–42.

MathSciNet   Google Scholar  

Bower, M. (2016). A framework for adaptive learning design in a Web-conferencing environment. Journal of Interactive Media in Education, 2016 (1), 11 http://doi.org/10.5334/jime.406 .

Article   MathSciNet   Google Scholar  

Bowman, N. D., & Akcaoglu, M. (2014). “I see smart people!”: Using Facebook to supplement cognitive and affective learning in the university mass lecture. The Internet and Higher Education, 23 , 1–8. doi: 10.1016/j.iheduc.2014.05.003 .

Boyle, E. A., Hainey, T., Connolly, T. M., Gray, G., Earp, J., Ott, M., et al. (2016). An update to the systematic literature review of empirical evidence of the impacts and outcomes of computer games and serious games. Computers & Education, 94 , 178–192. doi: 10.1016/j.compedu.2015.11.003 .

Bryson, C., & Hand, L. (2007). The role of engagement in inspiring teaching and learning. Innovations in Education and Teaching International, 44 (4), 349–362. doi: 10.1080/14703290701602748 .

Buchanan, T., Sainter, P., & Saunders, G. (2013). Factors affecting faculty use of learning technologies: Implications for models of technology adoption. Journal of Computer in Higher Education, 25 (1), 1–11.

Bullen, M., & Morgan, T. (2011). Digital learners not digital natives. La Cuestión Universitaria, 7 , 60–68.

Bullen, M., Morgan, T., & Qayyum, A. (2011). Digital learners in higher education: Generation is not the issue. Canadian Journal of Learning and Technology, 37 (1), 1–24.

Calabretto, J., & Rao, D. (2011). Wikis to support collaboration of pharmacy students in medication management workshops -- a pilot project. International Journal of Pharmacy Education & Practice, 8 (2), 1–12.

Camacho, M. E., Carrión, M. D., Chayah, M., & Campos, J. M. (2016). The use of wiki to promote students’ learning in higher education (Degree in Pharmacy). International Journal of Educational Technology in Higher Education, 13 (1), 1–8 https://doi.org/10.1186/s41239-016-0025-y .

Camus, M., Hurt, N. E., Larson, L. R., & Prevost, L. (2016). Facebook as an online teaching tool: Effects on student participation, learning, and overall course performance. College Teaching, 64 (2), 84–94 https://doi.org/10.1080/87567555.2015.1099093 .

Carini, R. M., Kuh, G. D., & Klein, S. P. (2006). Student engagement and student learning: Testing the linkages. Research in Higher Education, 47 (1), 1–32. doi: 10.1007/s11162-005-8150-9 .

Cassidy, E. D., Colmenares, A., Jones, G., Manolovitz, T., Shen, L., & Vieira, S. (2014). Higher Education and Emerging Technologies: Shifting Trends in Student Usage. The Journal of Academic Librarianship, 40 , 124–133. doi: 10.1016/j.acalib.2014.02.003 .

Center for Postsecondary Research (2016). Engagement insights: Survey findings on the quality of undergraduate education . Retrieved from http://nsse.indiana.edu/NSSE_2016_Results/pdf/NSSE_2016_Annual_Results.pdf .

Center for Postsecondary Research (2017). About NSSE. Retrieved on February 15, 2017 from http://nsse.indiana.edu/html/about.cfm

Cercone, K. (2008). Characteristics of adult learners with implications for online learning design. AACE Journal, 16 (2), 137–159.

Chang, J. W., & Wei, H. Y. (2016). Exploring Engaging Gamification Mechanics in Massive Online Open Courses. Educational Technology & Society, 19 (2), 177–203.

Chawinga, W. D. (2017). Taking social media to a university classroom: teaching and learning using Twitter and blogs. International Journal of Educational Technology in Higher Education, 14 (1), 3 https://doi.org/10.1186/s41239-017-0041-6 .

Chen, B., Seilhamer, R., Bennett, L., & Bauer, S. (2015). Students’ mobile learning practices in higher education: A multi-year study. In EDUCAUSE Review Retrieved from http://er.educause.edu/articles/2015/6/students-mobile-learning-practices-in-higher-education-a-multiyear-study .

Chu, S. K., Chan, C. K., & Tiwari, A. F. (2012). Using blogs to support learning during internship. Computers & Education, 58 (3), 989–1000. doi: 10.1016/j.compedu.2011.08.027 .

Clements, J. C. (2015). Using Facebook to enhance independent student engagement: A case study of first-year undergraduates. Higher Education Studies, 5 (4), 131–146 https://doi.org/10.5539/hes.v5n4p131 .

Coates, H. (2008). Attracting, engaging and retaining: New conversations about learning . Camberwell: Australian Council for Educational Research Retrieved from http://research.acer.edu.au/cgi/viewcontent.cgi?article=1015&context=ausse .

Coffey, D. J., Miller, W. J., & Feuerstein, D. (2011). Classroom as reality: Demonstrating campaign effects through live simulation. Journal of Political Science Education, 7 (1), 14–33.

Coghlan, E., Crawford, J. Little, J., Lomas, C., Lombardi, M., Oblinger, D., & Windham, C. (2007). ELI Discovery Tool: Guide to Blogging . Retrieved from https://net.educause.edu/ir/library/pdf/ELI8006.pdf .

Connolly, T. M., Boyle, E. A., MacArthur, E., Hainey, T., & Boyle, J. M. (2012). A systematic literature review of empirical evidence on computer games and serious games. Computers & Education, 59 , 661–686. doi: 10.1016/j.compedu.2012.03.004 .

Cook, C. W., & Sonnenberg, C. (2014). Technology and online education: Models for change. ASBBS E-Journal, 10 (1), 43–59.

Crocco, F., Offenholley, K., & Hernandez, C. (2016). A proof-of-concept study of game-based learning in higher education. Simulation & Gaming, 47 (4), 403–422. doi: 10.1177/1046878116632484 .

Csikszentmihalyi, M. (1988). The flow experience and its significance for human psychology. In M. Csikszentmihalyi & I. Csikszentmihalyi (Eds.), Optimal experience: Psychological studies of flow in consciousness (pp. 15–13). Cambridge, UK: Cambridge University Press.

Chapter   Google Scholar  

Dahlstrom, E. (2012). ECAR study of undergraduate students and information technology, 2012 (Research Report). Retrieved from http://net.educause.edu/ir/library/pdf/ERS1208/ERS1208.pdf

de Freitas, S. (2006). Learning in immersive worlds: A review of game-based learning . Retrieved from https://curve.coventry.ac.uk/open/file/aeedcd86-bc4c-40fe-bfdf-df22ee53a495/1/learning%20in%20immersive%20worlds.pdf .

Dichev, C., & Dicheva, D. (2017). Gamifying education: What is known, what is believed and what remains uncertain: A critical review. International Journal of Educational Technology in Higher Education, 14 (9), 1–36. doi: 10.1186/s41239-017-0042-5 .

DiVall, M. V., & Kirwin, J. L. (2012). Using Facebook to facilitate course-related discussion between students and faculty members. American Journal of Pharmaceutical Education, 76 (2), 1–5 https://doi.org/10.5688/ajpe76232 .

Dos, B., & Demir, S. (2013). The analysis of the blogs created in a blended course through the reflective thinking perspective. Educational Sciences: Theory & Practice, 13 (2), 1335–1344.

Dougherty, K., & Andercheck, B. (2014). Using Facebook to engage learners in a large introductory course. Teaching Sociology, 42 (2), 95–104 https://doi.org/10.1177/0092055x14521022 .

Dyson, B., Vickers, K., Turtle, J., Cowan, S., & Tassone, A. (2015). Evaluating the use of Facebook to increase student engagement and understanding in lecture-based classes. Higher Education: The International Journal of Higher Education and Educational Planning, 69 (2), 303–313 https://doi.org/10.1007/s10734-014-9776-3.

Esteves, K. K. (2012). Exploring Facebook to enhance learning and student engagement: A case from the University of Philippines (UP) Open University. Malaysian Journal of Distance Education, 14 (1), 1–15.

Evans, C. (2014). Twitter for teaching: Can social media be used to enhance the process of learning? British Journal of Educational Technology, 45 (5), 902–915 https://doi.org/10.1111/bjet.12099 .

Fagioli, L., Rios-Aguilar, C., & Deil-Amen, R. (2015). Changing the context of student engagement: Using Facebook to increase community college student persistence and success. Teachers College Record, 17 , 1–42.

Farley, P. C. (2013). Using the computer game “FoldIt” to entice students to explore external representations of protein structure in a biochemistry course for nonmajors. Biochemistry and Molecular Biology Education, 41 (1), 56–57 https://doi.org/10.1002/bmb.20655 .

Francescucci, A., & Foster, M. (2013). The VIRI classroom: The impact of blended synchronous online courses on student performance, engagement, and satisfaction. Canadian Journal of Higher Education, 43 (3), 78–91.

Fredricks, J., Blumenfeld, P., & Paris, A. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74 (1), 59–109. doi: 10.3102/00346543074001059 .

Gagnon, K. (2015). Using twitter in health professional education: A case study. Journal of Allied Health, 44 (1), 25–33.

Gandhi, P., Khanna, S., & Ramaswamy, S. (2016). Which industries are the most digital (and why?) . Retrieved from https://hbr.org/2016/04/a-chart-that-shows-which-industries-are-the-most-digital-and-why .

Garrison, D. R., & Arbaugh, J. B. (2007). Researching the community of inquiry framework: Review, issues, and future directions. The Internet and Higher Education, 10 (3), 157–172 http://dx.doi.org/10.1016/j.iheduc.2007.04.001 .

Garrity, M. K., Jones, K., VanderZwan, K. J., de la Rocha, A. R., & Epstein, I. (2014). Integrative review of blogging: Implications for nursing education. Journal of Nursing Education, 53 (7), 395–401. doi: 10.3928/01484834-20140620-01 .

Gikas, J., & Grant, M. M. (2013). Mobile computing devices in higher education: Student perspectives on learning with cellphones, smartphones & social media. The Internet and Higher Education, 19 , 18–26 http://dx.doi.org/10.1016/j.iheduc.2013.06.002 .

Gilboy, M. B., Heinerichs, S., & Pazzaglia, G. (2015). Enhancing student engagement using the flipped classroom. Journal of Nutrition Education and Behavior, 47 (1), 109–114 http://dx.doi.org/10.1016/j.jneb.2014.08.008 .

Greenwood, S., Perrin, A., & Duggan, M. (2016). Social media update 2016 . Washington.: Pew Research Center Retrieved from http://www.pewinternet.org/2016/11/11/social-media-update-2016/ .

Grimley, M., Green, R., Nilsen, T., & Thompson, D. (2012). Comparing computer game and traditional lecture using experience ratings from high and low achieving students. Australasian Journal of Educational Technology, 28 (4), 619–638 https://doi.org/10.14742/ajet.831 .

Gunawardena, C. N., Hermans, M. B., Sanchez, D., Richmond, C., Bohley, M., & Tuttle, R. (2009). A theoretical framework for building online communities of practice with social networking tools. Educational Media International, 46 (1), 3–16 https://doi.org/10.1080/09523980802588626 .

Haggis, T. (2009). What have we been thinking of? A critical overview of 40 years of student learning research in higher education. Studies in Higher Education, 34 (4), 377–390. doi: 10.1080/03075070902771903 .

Hauptman, P.H. (2015). Mobile technology in college instruction. Faculty perceptions and barriers to adoption (Doctoral dissertation). Retrieved from ProQuest. (AAI3712404).

Hennessy, C. M., Kirkpatrick, E., Smith, C. F., & Border, S. (2016). Social media and anatomy education: Using twitter to enhance the student learning experience in anatomy. Anatomical Sciences Education, 9 (6), 505–515 https://doi.org/10.1002/ase.1610 .

Hew, K. F., Huang, B., Chu, K. S., & Chiu, D. K. (2016). Engaging Asian students through game mechanics: Findings from two experiment studies. Computers & Education, 93 , 221–236. doi: 10.1016/j.compedu.2015.10.010 .

Hewege, C. R., & Perera, L. R. (2013). Pedagogical significance of wikis: Towards gaining effective learning outcomes. Journal of International Education in Business, 6 (1), 51–70 https://doi.org/10.1108/18363261311314953 .

Hou, H., Wang, S., Lin, P., & Chang, K. (2015). Exploring the learner’s knowledge construction and cognitive patterns of different asynchronous platforms: comparison of an online discussion forum and Facebook. Innovations in Education and Teaching International, 52 (6), 610–620. doi: 10.1080/14703297.2013.847381 .

Hu, S., & McCormick, A. C. (2012). An engagement-based student typology and its relationship to college outcomes. Research in Higher Education, 53 , 738–754. doi: 10.1007/s11162-012-9254-7 .

Hudson, T. M., Knight, V., & Collins, B. C. (2012). Perceived effectiveness of web conferencing software in the digital environment to deliver a graduate course in applied behavior analysis. Rural Special Education Quarterly, 31 (2), 27–39.

Hurt, N. E., Moss, G. S., Bradley, C. L., Larson, L. R., Lovelace, M. D., & Prevost, L. B. (2012). The ‘Facebook’ effect: College students’ perceptions of online discussions in the age of social networking. International Journal for the Scholarship of Teaching & Learning, 6 (2), 1–24 https://doi.org/10.20429/ijsotl.2012.060210 .

Ibáñez, M. B., Di-Serio, A., & Delgado-Kloos, C. (2014). Gamification for engaging computer science students in learning activities: A case study. IEEE Transactions on Learning Technologies, 7 (3), 291–301 https://doi.org/10.1109/tlt.2014.2329293 .

Ivala, E., & Gachago, D. (2012). Social media for enhancing student engagement: The use of facebook and blogs at a university of technology. South African Journal of Higher Education, 26 (1), 152–167.

Johnson, D. R. (2013). Technological change and professional control in the professoriate. Science, Technology & Human Values, 38 (1), 126–149. doi: 10.1177/0162243911430236 .

Junco, R., Elavsky, C. M., & Heiberger, G. (2013). Putting Twitter to the test: Assessing outcomes for student collaboration, engagement and success. British Journal of Educational Technology, 44 (2), 273–287. doi: 10.1111/j.1467-8535.2012.01284.x .

Junco, R., Heibergert, G., & Loken, E. (2011). The effect of Twitter on college student engagement and grades. Journal of Computer Assisted Learning, 27 (2), 119–132. doi: 10.1111/j.1365-2729.2010.00387.x .

Kahu, E. R. (2013). Framing student engagement in higher education. Studies in Higher Education, 38 (5), 758–773. doi: 10.1080/03075079.2011.598505 .

Kaware, S. S., & Sain, S. K. (2015). ICT Application in Education: An Overview. International Journal of Multidisciplinary Approach & Studies, 2 (1), 25–32.

Ke, F., Xie, K., & Xie, Y. (2016). Game-based learning engagement: A theory- and data-driven exploration. British Journal of Educational Technology, 47 (6), 1183–1201 https://doi.org/10.1111/bjet.12314 .

Kent, M. (2013). Changing the conversation: Facebook as a venue for online class discussion in higher education. Journal of Online Learning & Teaching, 9 (4), 546–565 https://doi.org/10.1353/rhe.2015.0000 .

Kidd, T., Davis, T., & Larke, P. (2016). Experience, adoption, and technology: Exploring the phenomenological experiences of faculty involved in online teaching at once school of public health. International Journal of E-Learning, 15 (1), 71–99.

Kim, Y., Jeong, S., Ji, Y., Lee, S., Kwon, K. H., & Jeon, J. W. (2015). Smartphone response system using twitter to enable effective interaction and improve engagement in large classrooms. IEEE Transactions on Education, 58 (2), 98–103 https://doi.org/10.1109/te.2014.2329651 .

Kinchin. (2012). Avoiding technology-enhanced non-learning. British Journal of Educational Technology, 43 (2), E43–E48.

Kolb, D. A. (2014). Experiential learning: Experience as the source of learning and development (2nd ed.). Upper Saddle River: Pearson Education, Inc..

Kopcha, T. J., Rieber, L. P., & Walker, B. B. (2016). Understanding university faculty perceptions about innovation in teaching and technology. British Journal of Educational Technology, 47 (5), 945–957. doi: 10.1111/bjet.12361 .

Krause, K., & Coates, H. (2008). Students’ engagement in first-year university. Assessment and Evaluation in Higher Education, 33 (5), 493–505. doi: 10.1080/02602930701698892 .

Kuh, G. D. (2009). The National Survey of Student Engagement: Conceptual and empirical foundations. New Directions for Institutional Research, 141 , 5–20.

Lam, S., Wong, B., Yang, H., & Yi, L. (2012). Understanding student engagement with a contextual model. In S. L. Christenson, A. L. Reschly, & C. Wylie (Eds.), Handbook of Research on Student Engagement (pp. 403–419). New York: Springer.

Lawrence, B., & Lentle-Keenan, S. (2013). Teaching beliefs and practice, institutional context, and the uptake of Web-based technology. Distance Education, 34 (1), 4–20.

Leach, L. (2016). Enhancing student engagement in one institution. Journal of Further and Higher Education, 40 (1), 23–47.

Lester, D. (2013). A review of the student engagement literature. Focus on Colleges, Universities, and Schools, 7 (1), 1–8.

Lewis, C. C., Fretwell, C. E., Ryan, J., & Parham, J. B. (2013). Faculty use of established and emerging technologies in higher education: A unified theory of acceptance and use of technology perspective. International Journal of Higher Education, 2 (2), 22–34 http://dx.doi.org/10.5430/ijhe.v2n2p22 .

Lin, C., Singer, R., & Ha, L. (2010). Why university members use and resist technology? A structure enactment perspective. Journal of Computing in Higher Education, 22 (1), 38–59. doi: 10.1007/s12528-010-9028-1 .

Linder-VanBerschot, J. A., & Summers, L. L. (2015). Designing instruction in the face of technology transience. Quarterly Review of Distance Education, 16 (2), 107–118.

Liu, C., Cheng, Y., & Huang, C. (2011). The effect of simulation games on the learning of computational problem solving. Computers & Education, 57 (3), 1907–1918 https://doi.org/10.1016/j.compedu.2011.04.002 .

Lu, J., Hallinger, P., & Showanasai, P. (2014). Simulation-based learning in management education: A longitudinal quasi-experimental evaluation of instructional effectiveness. Journal of Management Development, 33 (3), 218–244. doi: 10.1108/JMD-11-2011-0115 .

Maben, S., Edwards, J., & Malone, D. (2014). Online engagement through Facebook groups in face-to-face undergraduate communication courses: A case study. Southwestern Mass Communication Journal, 29 (2), 1–27.

Manca, S., & Ranieri, M. (2013). Is it a tool suitable for learning? A critical review of the literature on Facebook as a technology-enhanced learning environment. Journal of Computer Assisted Learning, 29 (6), 487–504. doi: 10.1111/jcal.12007 .

Mansouri, S. A., & Piki, A. (2016). An exploration into the impact of blogs on students’ learning: Case studies in postgraduate business education. Innovations in Education And Teaching International, 53 (3), 260–273 http://dx.doi.org/10.1080/14703297.2014.997777 .

Marriott, P., Tan, S. W., & Marriot, N. (2015). Experiential learning – A case study of the use of computerized stock market trading simulation in finance education. Accounting Education, 24 (6), 480–497 http://dx.doi.org/10.1080/09639284.2015.1072728 .

Martin, F., Parker, M. A., & Deale, D. F. (2012). Examining interactivity in synchronous virtual classrooms. International Review of Research in Open and Distance Learning, 13 (3), 227–261.

Martin, K., Goldwasser, M., & Galentino, R. (2017). Impact of Cohort Bonds on Student Satisfaction and Engagement. Current Issues in Education, 19 (3), 1–14.

Martínez, A. A., Medina, F. X., Albalat, J. A. P., & Rubió, F. S. (2013). Challenges and opportunities of 2.0 tools for the interdisciplinary study of nutrition: The case of the Mediterranean Diet wiki. International Journal of Educational Technology in Higher Education, 10 (1), 210–225 https://doi.org/10.7238/rusc.v10i1.1341 .

McBrien, J. L., Jones, P., & Cheng, R. (2009). Virtual spaces: Employing a synchronous online classroom to facilitate student engagement in online learning. International Review of Research in Open and Distance Learning, 10 (3), 1–17 https://doi.org/10.19173/irrodl.v10i3.605 .

McClenney, K., Marti, C. N., & Adkins, C. (2012). Student engagement and student outcomes: Key findings from “CCSSE” validation research . Austin: Community College Survey of Student Engagement.

McKay, M., Sanko, J., Shekhter, I., & Birnbach, D. (2014). Twitter as a tool to enhance student engagement during an interprofessional patient safety course. Journal of Interprofessional Care, 28 (6), 565–567 https://doi.org/10.3109/13561820.2014.912618 .

Miller, A. D., Norris, L. B., & Bookstaver, P. B. (2012). Use of wikis in pharmacy hybrid elective courses. Currents in Pharmacy Teaching & Learning, 4 (4), 256–261. doi: 10.1016/j.cptl.2012.05.004 .

Morley, D. A. (2012). Enhancing networking and proactive learning skills in the first year university experience through the use of wikis. Nurse Education Today, 32 (3), 261–266.

Mysko, C., & Delgaty, L. (2015). How and why are students using Twitter for #meded? Integrating Twitter into undergraduate medical education to promote active learning. Annual Review of Education, Communication & Language Sciences, 12 , 24–52.

Nadolny, L., & Halabi, A. (2016). Student participation and achievement in a large lecture course with game-based learning. Simulation and Gaming, 47 (1), 51–72. doi: 10.1177/1046878115620388 .

Naghdipour, B., & Eldridge, N. H. (2016). Incorporating social networking sites into traditional pedagogy: A case of facebook. TechTrends, 60 (6), 591–597 http://dx.doi.org/10.1007/s11528-016-0118-4 .

Nakamaru, S. (2012). Investment and return: Wiki engagement in a “remedial” ESL writing course. Journal of Research on Technology in Education, 44 (4), 273–291.

Nelson, R. (2016). Apple’s app store will hit 5 million apps by 2020, more than doubling its current size . Retrieved from https://sensortower.com/blog/app-store-growth-forecast-2020 .

Nora, A., Barlow, E., & Crisp, G. (2005). Student persistence and degree attainment beyond the first year in college. In A. Seidman (Ed.), College Student Retention (pp. 129–154). Westport: Praeger Publishers.

Osgerby, J., & Rush, D. (2015). An exploratory case study examining undergraduate accounting students’ perceptions of using Twitter as a learning support tool. International Journal of Management Education, 13 (3), 337–348. doi: 10.1016/j.ijme.2015.10.002 .

Pace, C. R. (1980). Measuring the quality of student effort. Current Issues in Higher Education, 2 , 10–16.

Pace, C. R. (1984). Student effort: A new key to assessing quality . Los Angeles: University of California, Higher Education Research Institute.

Paul, J. A., & Cochran, J. D. (2013). Key interactions for online programs between faculty, students, technologies, and educational institutions: A holistic framework. Quarterly Review of Distance Education, 14 (1), 49–62.

Pellas, N. (2014). The influence of computer self-efficacy, metacognitive self-regulation, and self-esteem on student engagement in online learning programs: Evidence from the virtual world of Second Life. Computers in Human Behavior, 35 , 157–170. doi: 10.1016/j.chb.2014.02.048 .

Poole, S. M., Kemp, E., Williams, K. H., & Patterson, L. (2014). Get your head in the game: Using gamification in business education to connect with Generation Y. Journal for Excellence in Business Education, 3 (2), 1–9.

Poushter, J. (2016). Smartphone ownership and internet usage continues to climb in emerging economies . Washington, D.C.: Pew Research Center Retrieved from http://www.pewglobal.org/2016/02/22/smartphone-ownership-and-internet-usage-continues-to-climb-in-emerging-economies/ .

Prestridge, S. (2014). A focus on students’ use of Twitter - their interactions with each other, content and interface. Active Learning in Higher Education, 15 (2), 101–115.

Rambe, P. (2012). Activity theory and technology mediated interaction: Cognitive scaffolding using question-based consultation on “Facebook”. Australasian Journal of Educational Technology, 28 (8), 1333–1361 https://doi.org/10.14742/ajet.775 .

Reid, P. (2014). Categories for barriers to adoption of instructional technologies. Education and Information Technologies, 19 (2), 383–407.

Revere, L., & Kovach, J. V. (2011). Online technologies for engagement learning: A meaningful synthesis for educators. Quarterly Review of Distance Education, 12 (2), 113–124.

Richardson, J. C., & Newby, T. (2006). The role of students’ cognitive engagement in online learning. American Journal of Distance Education, 20 (1), 23–37 http://dx.doi.org/10.1207/s15389286ajde2001_3 .

Ross, H. M., Banow, R., & Yu, S. (2015). The use of Twitter in large lecture courses: Do the students see a benefit? Contemporary Educational Technology, 6 (2), 126–139.

Roussinos, D., & Jimoyiannis, A. (2013). Analysis of students’ participation patterns and learning presence in a wiki-based project. Educational Media International, 50 (4), 306–324 https://doi.org/10.1080/09523987.2013.863471 .

Salaber, J. (2014). Facilitating student engagement and collaboration in a large postgraduate course using wiki-based activities. International Journal of Management Education, 12 (2), 115–126. doi: 10.1016/j.ijme.2014.03.006 .

Scarlet, J., & Ampolos, L. (2013). Using game-based learning to teach psychopharmacology. Psychology Learning and Teaching, 12 (1), 64–70 https://doi.org/10.2304/plat.2013.12.1.64 .

Sharma, P., & Tietjen, P. (2016). Examining patterns of participation and meaning making in student blogs: A case study in higher education. American Journal of Distance Education, 30 (1), 2–13 http://dx.doi.org/10.1080/08923647.2016.1119605 .

Shraim, K. Y. (2014). Pedagogical innovation within Facebook: A case study in tertiary education in Palestine. International Journal of Emerging Technologies in Learning, 9 (8), 25–31. doi: 10.3991/ijet.v9i8.3805 .

Siddique, Z., Ling, C., Roberson, P., Xu, Y., & Geng, X. (2013). Facilitating higher-order learning through computer games. Journal of Mechanical Design, 135 (12), 121004–121010.

Smith, A., & Anderson, M. (2016). Online Shopping and E-Commerce . Washington, D.C.: Pew Research Center Retrieved from http://www.pewinternet.org/2016/12/19/online-shopping-and-e-commerce/ .

Staines, Z., & Lauchs, M. (2013). Students’ engagement with Facebook in a university undergraduate policing unit. Australasian Journal of Educational Technology, 29 (6), 792–805 https://doi.org/10.14742/ajet.270 .

Sun, A., & Chen, X. (2016). Online education and its effective practice: A research review. Journal of Information Technology Education: Research, 15 , 157–190.

Tiernan, P. (2014). A study of the use of Twitter by students for lecture engagement and discussion. Education and Information Technologies, 19 (4), 673–690 https://doi.org/10.1007/s10639-012-9246-4 .

Trowler, V. (2010). Student engagement literature review . Lancaster: Lancaster University Retrieved from http://www.lancaster.ac.uk/staff/trowler/StudentEngagementLiteratureReview.pdf .

Trowler, V., & Trowler, P. (2010). Student engagement evidence summary . Lancaster: Lancaster University Retrieved from http://eprints.lancs.ac.uk/61680/1/Deliverable_2._Evidence_Summary._Nov_2010.pdf .

van Beynen, K., & Swenson, C. (2016). Exploring peer-to-peer library content and engagement on a student-run Facebook group. College & Research Libraries, 77 (1), 34–50 https://doi.org/10.5860/crl.77.1.34 .

Wang, S. (2008). Blogs in education. In M. Pagani (Ed.), Encyclopedia of Multimedia Technology and Networking (2nd ed., pp. 134–139). Hershey: Information Sciences Reference.

Wdowik, S. (2014). Using a synchronous online learning environment to promote and enhance transactional engagement beyond the classroom. Campus — Wide Information Systems, 31 (4), 264–275. doi: 10.1108/CWIS-10-2013-0057 .

Weibel, D., Wissmath, B., Habegger, S., Steiner, Y., & Groner, R. (2008). Playing online games against computer-vs. human-controlled opponents: Effects on presence, flow, and enjoyment. Computers in Human Behavior, 24 (5), 2274–2291 https://doi.org/10.1016/j.chb.2007.11.002 .

West, B., Moore, H., & Barry, B. (2015). Beyond the tweet: Using Twitter to enhance engagement, learning, and success among first-year students. Journal of Marketing Education, 37 (3), 160–170. doi: 10.1177/0273475315586061 .

Westera, W. (2015). Reframing the role of educational media technologies. Quarterly Review of Distance Education, 16 (2), 19–32.

Whitton, N. (2011). Game engagement theory and adult learning. Simulation & Gaming, 42 (5), 596–609.

Williams, D., & Whiting, A. (2016). Exploring the relationship between student engagement, Twitter, and a learning management system: A study of undergraduate marketing students. International Journal of Teaching & Learning in Higher Education, 28 (3), 302–313.

Wimpenny, K., & Savin-Baden, M. (2013). Alienation, agency, and authenticity: A synthesis of the literature on student engagement. Teaching in Higher Education, 18 (3), 311–326. doi: 10.1080/13562517.2012.725223 .

Witkowski, P., & Cornell, T. (2015). An Investigation into Student Engagement in Higher Education Classrooms. InSight: A Journal of Scholarly Teaching, 10 , 56–67.

Wright, G. B. (2011). Student-centered learning in higher education. International Journal of Teaching and Learning in Higher Education, 23 (3), 92–97.

Yang, C., & Chang, Y. (2012). Assessing the effects of interactive blogging on student attitudes towards peer interaction, learning motivation, and academic achievements. Journal of Computer Assisted Learning, 28 (2), 126–135 https://doi.org/10.1111/j.1365-2729.2011.00423.x .

Zepke, N. (2014). Student engagement research in higher education: questioning an academic orthodoxy. Teaching in Higher Education, 19 (6), 697–708 http://dx.doi.org/10.1080/13562517.2014.901956 .

Zepke, N., & Leach, L. (2010). Improving student engagement: Ten proposals for action. Active Learning in Higher Education, 11 (3), 167–177. doi: 10.1177/1469787410379680 .

Zickuhr, K., & Raine, L. (2014). E-reading rises as device ownership jumps . Washington, D.C.: Pew Research Center Retrieved from http://www.pewinternet.org/2014/01/16/e-reading-rises-as-device-ownership-jumps/ .

Zimmermann, L. K. (2013). Using a virtual simulation program to teach child development. College Teaching, 61 (4), 138–142. doi: 10.1080/87567555.2013.817377 .

Download references

Acknowledgements

Not applicable.

This research was supported in part by a Laureate Education, Incl. David A. Wilson research grant study awarded to the second author, “A Comparative Analysis of Student Engagement and Critical Thinking in Two Approaches to the Online Classroom”.

Availability of data and materials

Authors’ contributions.

The first and second authors contributed significantly to the writing, review, and conceptual thinking of the manuscript. The third author provided a first detailed outline of what the paper could address, and the fourth offer provided input and feedback through critical review. All authors read and approved the final manuscript.

Competing interests

The authors declare that they have no competing interests.

Consent for publication

Ethics approval and consent to participate.

The parent study was approved by the University of Liverpool Online International Online Ethics Review Committee, approval number 04-24-2015-01.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Authors and affiliations.

University of Liverpool Online, Liverpool, UK

Laura A. Schindler & Osama A. Morad

Laureate Education, Inc., Baltimore, USA

Gary J. Burkholder

Walden University, Minneapolis, USA

University of Lincoln, Lincoln, UK

Craig Marsh

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Laura A. Schindler .

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Cite this article.

Schindler, L.A., Burkholder, G.J., Morad, O.A. et al. Computer-based technology and student engagement: a critical review of the literature. Int J Educ Technol High Educ 14 , 25 (2017). https://doi.org/10.1186/s41239-017-0063-0

Download citation

Received : 31 March 2017

Accepted : 06 June 2017

Published : 02 October 2017

DOI : https://doi.org/10.1186/s41239-017-0063-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Social networking

Software as a Service Challenges: A Systematic Literature Review

• Conference paper
• First Online: 14 October 2022
• Cite this conference paper

• Ahmed Mamdouh Abdelfatah Ibrahim 10 ,
• Norris Syed Abdullah 11 &
• Mahadi Bahari 11  

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 561))

Included in the following conference series:

• Proceedings of the Future Technologies Conference

600 Accesses

1 Citations

Software as a Service (SaaS) as a type of Cloud computing is a type of computing that become a contentious topic among enterprise information technology (IT) professionals. In recent decades, rapid advances and breakthroughs in SaaS have enabled many businesses in numerous industries to consider it as a promising technology towards using. Nevertheless, such a new approach is up against a number of obstacles, a systematic literature review of SaaS cloud computing’s possible antecedents has been conducted to describe the significant challenges SaaS cloud computing adoptions are facing. Articles describing the difficulties of SaaS Adoption were compiled. For a concentrated discourse on solutions, we organized the key challenges in the ontology. As a result, out of more than 68 factors which has been addressed, 16 factors has been identified and deeply discussed as a significant factors that may affects CC SaaS adoption. A comprehensive framework of SaaS Adoption obstacles will be required to expedite the adoption of this Innovation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Ratten, V.: Cloud computing technology innovation advances: a set of research propositions. Int. J. Cloud Appl. Comput. 5 (1), 69–76 (2015). https://doi.org/10.4018/ijcac.2015010106

Article   Google Scholar  

Mell, P., Grance, T.: The NIST definition of cloud computing. National Institute of Standards and Technology, Gaithersburg, USA, September 2011

Google Scholar  

QSS: Cloud Computing – Delivery and Deployment Models (2019). https://www.qsstechnosoft.com/cloud-computing-delivery-and-deployment-models/

Pillai, S.: Cloud computing delivery models explained (2014). https://www.ibm.com/blogs/cloud-computing/2014/03/17/cloud-computing-delivery-models-explained/

www.binaryinformatics.com (2019). Cloud Service Model – Understand the Types, Characteristics, & Advantages. http://blog.binaryinformatics.com/technology/what-is-the-cloud-service-model/. Accessed 03 Apr 2019

Jing, X., Jian-Jun, Z.: A brief survey on the security model of cloud computing. Distrib. Comput. Appl. 34 (19), 475–478 (2010)

Amazon AWS: What is cloud computing? (2019). https://aws.amazon.com/what-is-cloud-computing/

Mitchell Grant: Software-as-a-Service (SaaS) (2020). https://www.investopedia.com/terms/s/software-as-a-service-saas.asp

Mell, P., Grance, T.: The NIST definition of cloud computing version 15. National Institute of Standards and Technology (NIST). Information Technology, Laboratory (2009). www.csrc.nist.gov

Avram, M.G.: Advantages and challenges of adopting cloud computing from an enterprise perspective. Procedia Technol. 12 , 529–534 (2014)

Sasikala, P.: Cloud computing in higher education. Int. J. Cloud Appl. Comput. 1 (2), 1–13 (2011)

CtrIs 2015: Top 5 Benefits of Cloud Adoption (2016). http://www.ctrls.in/blog/benefits-of-cloud-adoption/ . Accessed 14 Mar 2020

Kumar, D., Samalia, H.V., Verma, P.: Exploring suitability of cloud computing for small and medium-sized enterprises in India. J. Small Bus. Enterp. Dev. 24 (4), 814–832 (2017). https://doi.org/10.1108/JSBED-01-2017-0002

Rath, A., Kumar, S.: Decision points for adoption cloud computing in small, medium enterprises (SMEs). Internet Technol. Commun. 34 (4), 688–691 (2012)

Haider, Y., Selvan, S.: Confidentiality issues in cloud computing and countermeasures: a survey (2016)

Armstrong, R., Hall, B.J., Doyle, J., Waters, E.: Cochrane update. ‘Scoping the scope’ of a cochrane review. J. Public Health 33 (1), 147–150 (2011). https://doi.org/10.1093/pubmed/fdr015.PMID21345890

Pittway, L.: Systematic literature reviews. In: Thorpe, R., Holt, R. (eds.) The SAGE Dictionary of Qualitative Management Research. SAGE Publications Ltd. (2008). https://doi.org/10.4135/9780857020109

Eden, J., Levit, L., Berg, A., Morton, S., et al.: Institute of medicine (US) committee on standards for systematic reviews of comparative effectiveness research. In: Finding What Works in Health Care: Standards for Systematic Reviews (2011). https://doi.org/10.17226/13059 . ISBN 978-0-309-16425-2. PMID 24983062

Kitchenham, B., Charters, S.: Guidelines for performing systematic literature reviews in software engineering. Keele University and Durham University joint report ((2007))

Anwer, F., Aftab, S.: Latest customizations of XP: a systematic literature review. Int. J. Mod. Educ. Comput. Sci. (IJMECS) 9 (12), 26–37 (2017). https://doi.org/10.5815/ijmecs.2017.12.04

Brereton, P., Kitchenham, B.A., Budgen, D., Turner, M., Khalil, M.: Lessons from applying the systematic literature review process within the software engineering domain. J. Syst. Softw. 80 (4), 571–583 (2007)

Kitchenham, B.A., et al.: Preliminary guidelines for empirical research in software engineering. IEEE Trans. Softw. Eng. 28 (8), 721–734 (2002)

Palos-sanchez, P.R., Arenas-marquez, F.J., Aguayo-camacho, M.: Cloud computing (SaaS) adoption as a strategic technology: results of an empirical study (2017)

Oliveira, T., Martins, R., Sarker, S., Thomas, M., Popovič, A.: Understanding SaaS adoption: the moderating impact of the environment context. Int. J. Inf. Manag. 49 , 1–12 (2019)

Salum, K., Rozan, A., Zaidi, M.: Exploring the challenge impacted SMEs to adopt cloud ERP. Indian J. Sci. Technol. 9 , 1–8 (2016). https://doi.org/10.17485/ijst/2016/v9i45/100452

El-Haddadeh, R.: Digital innovation dynamics influence on organisational adoption: the case of cloud computing services. Inf. Syst. Front. 22 (4), 985–999 (2019). https://doi.org/10.1007/s10796-019-09912-2

Hemlata, G., Date, H., Ramaswamy, R.: Understanding determinants of cloud computing adoption using an integrated TAM-TOE model (2017)

Senyo, P.K., Effah, J., Addae, E.: Preliminary insight into cloud computing adoption in a developing country (2016). https://doi.org/10.1108/JEIM-09-2014-0094

Yang, Z., Sun, J., Zhang, Y., Wang, Y.: Understanding SaaS adoption from the perspective of organizational users: a tripod readiness model. Comput. Hum. Behav. 45 , 254–264 (2015)

Stieninger, M., et al.: Factors influencing the organizational adoption of cloud computing: a survey among cloud workers. Int. J. Inf. Syst. Proj. Manag. 6 (1), 5–23 (2018). https://doi.org/10.12821/ijispm060101

Yadegaridehkordi, E., Nilashi, M., Shuib, L., Samad, S.: A behavioral intention model for SaaS-based collaboration services in higher education. Educ. Inf. Technol. 25 (2), 791–816 (2019). https://doi.org/10.1007/s10639-019-09993-1

Safari, F., Safari, N., Hasanzadeh, A.: The adoption ofsoftware-as-a-service (SaaS): ranking the determinants (2017)

Ming, C.F., et al.: The determinant factors affecting cloud computing adoption by small and medium enterprises ( SMEs) in Sabah, Malaysia. J. Telecommun. Electron. Comput. Eng. (JTEC) 10 (3), 83–88 (2018)

Gutierrez, A., Boukrami, E., Lumsden, R.: Technological, organisational and environmental factors influencing managers’ decision to adopt cloud computing in the UK. J. Enterp. Inf. Manag. 28 (6), 788–807 (2015). https://doi.org/10.1108/JEIM-01-2015-0001

Hasheela Miss, V.T., Mufeti Dr, T.K.: An investigation of factors leading to the reluctance of SaaS ERP adoption in Namibian SMEs. Afr. J. Inf. Syst. 8 (4), 1 (2016)

Lian, J.W., Yen, D.C., Wang, Y.T.: An exploratory study to understand the critical factors affecting the decision to adopt cloud computing in Taiwan hospital. Int. J. Inf. Manag. 34 (1), 28–36 (2015)

Alkhater, N., et al.: organisations an empirical study of factors influencing cloud adoption among private sector organisations. Telemat. Inform. (2017). https://doi.org/10.1016/j.tele.2017.09.017

Senarathna, I., Wilkin, C., Warren, M., Yeoh, W., Salzman, S.: Factors that influence adoption of cloud computing: an empirical study of Australian SMEs. Australas. J. Inf. Syst. 22 (2018)

Awodele, O., Adebayo, A.O., Tayo, O.O.: Security and privacy issues in cloud computing. Commun. Appl. Electron. 7 (3), 14–17 (2017)

Hsu, C.-L., Lin, J.C.-C.: Exploring factors affecting the adoption of Internet of Things services. J. Comput. Inf. Syst. 58 (1), 49–57 (2016). https://doi.org/10.1080/08874417.2016.1186524

Sakr, S., Zomaya, A.: Encyclopedia of Big Data Technologies, 1st edn. Springer, Switzerland (2019). https://doi.org/10.1007/978-3-319-77525-8 . eReference ISBN 978-3-319-77525-8

Sun, Y., Zhang, J., Xiong, Y., Zhu, G.: Data security and privacy in cloud computing. Int. J. Distrib. Sens. Netw. 10 (7), 190903 (2014)

Gholami, A., Laure, E.: Security and privacy of sensitive data in cloud computing: a survey of recent developments. arXiv preprint arXiv:1601.01498 (2016)

Makkaoui, K.E., Ezzati, A., Beni-Hssane, A., Motamed, C.: Data confidentiality in the world of cloud. J. Theor. Appl. Inf. Technol. 84 (3) (2016)

Aloraini, A., Hammoudeh, M.: A survey on data confidentiality and privacy in cloud computing. In: Proceedings of the International Conference on Future Networks and Distributed Systems - ICFNDS 2017 (2017). https://doi.org/10.1145/3102304.3102314

Vasiljeva, T., Kreslins, K., Novik, D.: Challenge of cloud computing for SMEs: a case of baltic countries. J. Innov. Manag. Small Medium Enterp. 1–10 (2018). https://doi.org/10.5171/2018.238581

Arvanitis, S., Kyriakou, N., Loukis, E.N.: Why do firms adopt cloud computing? A comparative analysisbased on South and North Europe firm data. Telemat. Inform. 34 (7), 1322–1332 (2017). https://doi.org/10.1016/j.tele.2016.05.013

Narwal, R., Sangwan, S.: Benefits, dimensions and issues of software as a service (SAAS). Int. J. New Innov. Eng. Technol. (IJNIET), 36–40 (2013)

Caldeira, M.M., Ward, J.M.: Using resource-based theory to interpret the successful adoption and use of information systems and technology in manufacturing small and medium-sized enterprises. Eur. J. Inf. Syst. 12 (2), 127–141 (2003)

Gashami, J.P.G., Chang, Y., Rho, J.J., Park, M.-C.: Privacy concerns and benefits in SaaS adoption by individual users: a trade-off approach. Inf. Dev. 32 (4), 837–852 (2016). https://doi.org/10.1177/0266666915571428

Matias, J.B., Hernandez, A.A.: Cloud computing adoption intention by MSMEs in the Philippines. Glob. Bus. Rev. (2019). https://doi.org/10.1177/0972150918818262

Singh, A., Sharma, S., Kumar, S.R., Yadav, S.A.: Overview of PaaS and SaaS and its application in cloud computing. Paper presented at the 2016 International Conference on Innovation and Challenges in Cyber Security (ICICCS-INBUSH) (2016)

Opara-Martins, J., Sahandi, R., Tian, F.: A holistic decision framework to avoid vendor lock-in for cloud SaaS migration (2017). https://doi.org/10.5539/cis.v10n3p29

Chou, D.C.: Cloud computing: a value creation model. Comput. Stand. Interfaces 38 ,. 72–77 (2015). https://doi.org/10.1016/j.csi.2014.10.001

Tehrani, S.R., Shirazi, F.: Factors influencing the adoption of cloud computing by small and medium size enterprises (SMEs). Paper presented at the International Conference on Human Interface and the Management of Information (2014)

Gangwar, H., Date, H., Ramaswamy, R.: Understanding determinants of cloud computing adoption using an integrated TAM-TOE model. J. Enterp. Inf. Manag. 28 (1), 107–130 (2015). https://doi.org/10.1108/JEIM-08-2013-0065

AlBar, A.M., Hoque, M.R.: Factors affecting cloud ERP adoption in Saudi Arabia: an empirical study (2017). https://doi.org/10.1177/0266666917735677

Hsu, C., Lin, J.C.-C.: Factors affecting the adoption of cloud services in enterprises. Inf. Syst. e-Bus. Manag. (321) (2015). https://doi.org/10.1007/s10257-015-0300-9

Priyadarshinee, P., et al.: Understanding and predicting the determinants of cloud computing adoption: a two staged hybrid SEM - neural networks approach. Comput. Hum. Behav. 76 , 341–362 (2017). https://doi.org/10.1016/j.chb.2017.07.027

Lal, P., Bharadwaj, S.S.: Understanding the impact of cloud-based services adoption on organizational flexibility an exploratory study (2017)

Dincă, V.M., Dima, A.M., Rozsa, Z.: Determinants of cloud computing adoption by Romanian SMEs in the digital economy. J. Bus. Econ. Manag. 20 (4), 798–820 (2019)

Download references

Author information

Authors and affiliations.

Faculty of Computing, Universiti Teknologi Malaysia, 81310, Skudai, Johor, Malaysia

Ahmed Mamdouh Abdelfatah Ibrahim

Azman Hashim International Business School, Universiti Teknologi Malaysia, 81310, Skudai, Johor, Malaysia

Norris Syed Abdullah & Mahadi Bahari

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Ahmed Mamdouh Abdelfatah Ibrahim .

Editor information

Editors and affiliations.

Faculty of Science and Engineering, Saga University, Saga, Japan

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Cite this paper.

Ibrahim, A.M.A., Abdullah, N.S., Bahari, M. (2023). Software as a Service Challenges: A Systematic Literature Review. In: Arai, K. (eds) Proceedings of the Future Technologies Conference (FTC) 2022, Volume 3. FTC 2022 2022. Lecture Notes in Networks and Systems, vol 561. Springer, Cham. https://doi.org/10.1007/978-3-031-18344-7_17

Download citation

DOI : https://doi.org/10.1007/978-3-031-18344-7_17

Published : 14 October 2022

Publisher Name : Springer, Cham

Print ISBN : 978-3-031-18343-0

Online ISBN : 978-3-031-18344-7

eBook Packages : Intelligent Technologies and Robotics Intelligent Technologies and Robotics (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Research on product-service systems: topic landscape and future trends

Journal of Manufacturing Technology Management

ISSN : 1741-038X

Article publication date: 1 June 2021

Issue publication date: 17 December 2021

The paper attempts to address the following research questions (RQs): RQ1: What are the main research topics within PSS research? RQ2: What are future trends for PSS research?

Design/methodology/approach

Twenty years of research (1999–2018) on product-service systems (PSS) produced a significant amount of scientific literature on the topic. As the PSS field is relatively new and fragmented across different disciplines, a review of the prior and relevant literature is important in order to provide the necessary framework for understanding current developments and future perspectives. This paper aims to review and organize research contributions regarding PSS. A machine-learning algorithm, namely Latent Dirichlet Allocation, has been applied to the whole literature corpus on PSS in order to understand its structure.

The adopted approach resulted in the definition of eight distinct and representative topics able to deal adequately with the multidisciplinarity of the PSS. Furthermore, a systematic review of the literature is proposed to summarize the state-of-the-art and limitations in the identified PSS research topics. Based on this critical analysis, major gaps and future research challenges are presented and discussed.

Originality/value

On the basis of the results of the topic landscape, the paper presents some potential research opportunities on PSSs. In particular, challenges, transversal to the eight research topics and related to recent technology trends and digital transformation, have been discussed.

• Product-service systems
• Text mining
• Latent dirichlet allocation
• Topic landscape
• Literature review

Barravecchia, F. , Franceschini, F. , Mastrogiacomo, L. and Zaki, M. (2021), "Research on product-service systems: topic landscape and future trends", Journal of Manufacturing Technology Management , Vol. 32 No. 9, pp. 208-238. https://doi.org/10.1108/JMTM-04-2020-0164

Emerald Publishing Limited

Copyright © 2021, Federico Barravecchia, Fiorenzo Franceschini, Luca Mastrogiacomo and Mohamed Zaki

Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode .

1. Introduction

The last two decades have seen a growing trend toward research on product-service systems (PSSs). One of the first definitions of the concept was provided by Goedkoop et al. (1999) who stated that PSSs are “systems of products, services, networks of “players” and supporting infrastructure that continuously strives to be competitive, satisfy customer needs and have a lower environmental impact than traditional business models”. The following year, Roy (2000) published one of the early works on this topic, highlighting the positive environmental impacts that arise from the cohesive delivery of products and services.

PSSs are value offering composed of a mix of tangible and non-tangible elements, that can be labeled as products, services and infrastructures ( Goedkoop et al. , 1999 ; Barravecchia et al. , 2020a , b );

the elements of a PSS interact to fulfill customer needs ( Manzini and Vezzoli, 2003 );

PSS-based strategies have positive environmental effects compared to traditional business models ( Roy, 2000 );

Interactions between provider and customers are extended to different phases of PSS lifecycle ( Cavalieri and Pezzotta, 2012 ).

Even if relatively young, this field of research has produced a remarkable amount of scientific literature that analyzed the PSS concept from a plurality of points of view. Despite this, the literature still remains sparse in terms of guidance on how to categorize and delineate studies on PSSs ( Li et al. , 2020 ). Research on PSS is becoming increasingly fine-grained and some fundamental aspects of PSS remain dispersed ( Annarelli et al. , 2016 ). Multiple disciplines are interested in the concept of PSS ( Mont and Tukker, 2006 ). Each of these analyzes the PSS from its specific point of view, sometimes losing the overall view of the phenomenon. This conclusion is proven by the wide diffusion of literature reviews related to specific domains such as design ( Qu et al. , 2016 ), business models ( Reim et al. , 2015 ), or service engineering ( Cavalieri and Pezzotta, 2012 ), while, there is a lack of updated reviews concerning the whole PSS corpus literature ( Moro et al. , 2020 ). Heterogeneity in dealing with problems related to PSS enriches the field of research, but at the same time it is likely to lose a comprehensive view of the phenomenon.

A field of research that is becoming increasingly mature and specialized needs a general framework to collocate the different scientific contributions. Considering this, a structured and objective analysis allowing us to understand the main pillars of the PSS literature could be helpful.

The presented research aims at applying a computational and replicable approach to build on existing literature a theoretical framework capable of framing past, present and future works in PSS research. The methodological approach taken in this study is based on topic modeling analysis so as to produce a clustering of the entire body of literature on PSS. Specifically, the Latent Dirichlet Allocation (LDA) algorithm was implemented. This approach provides different advantages, including (1) the capability of analyzing a vast number of papers time, (2) the possibility of automatically grouping scientific articles in homogeneous clusters, thus identifying a framework of the main topics in a specific research field, (3) the outputs are not influenced by bias or prior knowledge of the researchers being produced solely by data analysis ( Antons and Breidbach, 2018 ; Asmussen and Møller, 2019 ).

What are the main topics within PSS research? Specific research goals are the recognition and characterization of the main streams of research within the heterogeneous scientific literature on PSS to establish a framework capable of describing the hidden structure of the PSS research corpus.

What are future trends for PSS research? A specific research goal is the recognition of current and future research priorities to address the academic community's efforts.

Trying to answer these questions, this paper provides the following three main contributions: (1) it proposes a conceptual framework – i.e. a topic landscape – to support identifying and understanding topics and trends in PSS research over the last 20 years, (2) it provides an updated overview of issues discussed in PSS literature and (3) it identifies and analyses emerging trends and possible research directions related to PSS.

Moreover, this study represents a first attempt to leverage machine learning and text mining techniques to cluster scientific contributions on PSS and identify the main research topics.

The rest of the paper is organized as follows. Section 2 presents the methodology applied to produce the topic landscape. Section 3 illustrates the results of the topic landscape of the literature on PSSs. The description of each PSS research topic, including the related literature, is presented in section 4 . Section 5 proposes potential opportunities and future trends in PSS research. The concluding section summarizes the implications, limitations and original contributions of the paper.

2. Mapping the structure of the PSS literature

In order to interpret a corpus as vast as that concerning PSS, the first step is to understand its structure. To this end, one of the challenges to be faced when attempting to analyze the scientific production of a large research field is the identification of the most discussed topic. Online databases make available a massive amount of journal articles and conference proceedings. In managing such a large number of documents, traditional methods, based on a systematic reading and classification of the documents, show some limitations, mainly related to the huge need of time and resources (see Table 1 ).

Trying to overcome these limitations, this study proposes the use of a topic modeling algorithm to identify the hidden structure of PSS literature.

Topic modeling is unsupervised machine-learning algorithms that can discover topics running through a collection of documents and annotating individual documents with topic labels ( Blei et al. , 2003 ). In this research, a specific topic modeling algorithm, i.e. Latent Dirichlet Allocation (LDA), has been applied. Given a big set of documents, LDA handles with the problems of: (1) identifying a set of topics that describe a text corpus (i.e. a collection of text document from a variety of sources); (2) associate a set of keywords to each topic and (3) define a specific mixture of these topics for each document ( Blei et al. , 2003 ).

The main difference between topic modeling techniques and traditional literature analysis lies in the cluster identification process ( Asmussen and Møller, 2019 ). Text mining techniques use the information related to the analyzed documents as a whole and cluster the various documents according to their content. From this perspective, this methodology can be defined as a bottom-up approach since the clustering is exclusively based on data analysis without any influence from the expert perceptions and prior knowledge. On the contrary, traditional literature survey methodologies can be considered as top-down approaches since the classification of papers is based either on pre-existing frameworks or on logical and deductive processes driven by an expert.

Table 1 offers a summary of the differences in terms of assumptions and costs between traditional literature review methodologies based on article reading and the topic modeling approach. The main differences relate to the automation of the process and the required to instruct and perform the analysis. In addition, there is evidence that the topic modeling approach allows even researchers with low substantive knowledge of the field under analysis to address the pre-analysis and analysis phases.

In detail, the LDA methodology can be structured in four steps: (1) identification of the text corpus; (2) text-corpus pre-processing; (3) topic extraction and (4) topic labeling and validation (see Figure 1 ). The following sections describe the four aforementioned steps in the specific application case related to the review of the PSS literature.

2.1 PHASE I: identification of the text corpus

The first step in the analysis was the selection of the data source. According to a plurality of authors, Scopus proved to be the bibliometric databases with the highest coverage in engineering and management literature in terms of indexed journals and conference proceeding ( Harzing and Alakangas, 2016 ). Data used for our analysis were retrieved from Scopus in January 2020 ( Scopus Elsevier, 2020 ).

This study focuses its analysis on the literature related to the concept of PSS. Similar concepts such as “Solutions” or “Hybrid offerings” were not explicitly included. The primary difference between PSS and similar concepts lies in their origins and in the different research communities investigating them. PSS is highly related to manufacturing and engineering research (see Table 2 ), while other concepts are mainly associated with marketing or service science ( Ulaga and Reinartz, 2011 ). Moreover, the PSS concept has been associated with the positive impact that it can have on the environmental sustainability of production systems. This aspect is marginal in the scientific literature concerning “hybrid offerings” or “solutions” concepts.

In this view, the main inclusion criterion for article selection was the presence in the title, abstract or keywords of the words “Product-Service Systems”. Analogous notations were also included in the query. The selection of search keywords is consistent with the objective of the paper, which, as mentioned above, is limited to surveying the literature regarding the specific concept of PSS. The resulting query string was: “PSS” OR “PSSs” OR “Industrial product service system” OR “Industrial product service systems” OR “Product Service Systems” OR “Industrial Product Service System” OR “Product-Service Systems” OR “Product/Service-Systems” OR “IPS2” OR “IPSS”. The differences in PSS notation are due to the different origin and academic backgrounds of the authors. In addition, a reference dictionary and standardized definitions have yet to be defined.

Only articles published in scientific peer-reviewed journals and indexed conference proceedings were included in the analysis. This selection resulted in 2028 documents published from 1999 to December 2019, of which 843 were published in journals and the rest on conference proceedings.

Table 2 reports the ten most influential journals in PSS research in terms of the number of published articles. From the table, it can be seen that PSS research covers several domains. In particular, the most critical subject categories in PSS publications are “Engineering”, “Business, Management and Accounting”, “Computer Science”, “Environmental Science” “Energy” and “Decision Sciences”.

Given the limited number of articles published, some strongly influential journals on PSS research are not included in the list reported in Table 2 . Among these, the most prominent are: International Journal of Operations and Production Management; Proceedings of the Institution of Mechanical Engineers, Part B: Journal of Engineering Manufacture; CIRP Annals - Manufacturing Technology; Industrial Marketing Management .

As for most of the applications of LDA in the analysis of scientific literature, the collection of the abstract of the documents was used as text corpus for the analysis ( Fang et al. , 2018 ). This phase resulted in the definition of the initial dataset of 534 pages of raw text composed of 286,145 words.

2.2 PHASE II: text corpus pre-processing

Removal of stop words (e.g. “the”, “and”, “when”), punctuation, numbers, words with a low frequency, words generally not related to topical content (e.g. “paper”, “present”);

Text lemmatization, i.e. all the words with similar meaning but with different inflected forms were replaced with a unique lemma (e.g. “manufacturing”, “manufacturer”);

Replacing all the n-grams – i.e. contiguous sequences of n items from a given sequence of text – with a single term;

2.3 PHASE III: topic extraction

The LDA algorithm was used to infer the major topics addressed within the text corpus. To this end, KNIME Analytics Platform, an open-source software, was used. The algorithm requires the definition of three parameters: the Dirichlet hyperparameters ( α , β ) and the number of topics ( T ) ( Blei et al. , 2003 ). Following the indications of Griffiths and Steyvers (2004) , we used β  = 0.1, α  = 50/ T . The optimal number of topics was determined to minimize the value of perplexity, i.e. a statistical measure of how well a probability model predicts a sample ( Clarkson and Robinson, 1999 ). A lower perplexity value suggests a better fit. As shown in Figure 2 , an optimal value of 8 topics can be defined. Based on the statistical distribution of the words in the text corpus, the LDA algorithm generated the list of keywords characterizing each topic (see Table 3 ) and, for each document, its probability of belonging to each of the defined topics.

2.4 PHASE IV: topic labeling and validation

The LDA algorithm identifies sets of keywords associated with each topic without generating a semantic label to describe each topic ( Blei, 2012 ). The authors of this paper used the sets of the principal 15 keywords associated with each topic to assign descriptive labels to each topic. To test their reliability, the defined topic labels were submitted to an external panel of scholars familiar with the PSSs literature, which confirmed the identified labels. The LDA algorithm results were validated by comparing the assigned topic of a randomly selected sample composed of one hundred articles with a manual topic assignment performed by a team of four researchers on PSS-related topics. The validation showed that the LDA algorithm produced a topic assignment that generally did not differ significantly from the results obtained by a manual categorization (estimated accuracy: 93%).

3. Topic landscape of product-service systems research

around 21% of the scientific production on PSSs concerns “PSS design”, followed by “PSS environmental and social impact” (18%) and “PSS & Servitization Process” (13%);

sustainability, including both the design and environmental and social aspects, represents more than a quarter of the scientific production on PSSs (the two topics may seem overlapping, but they deal with different aspects of sustainability: the former concerns the sustainability assessment, while the latter deals with the design of sustainable PSS).

Figure 3 reports the publication trend over time of the eight identified topics. Griffiths and Steyvers (2004) proposed a distinction between hot and cold topics: while the former are topics for which the number of published articles grows over time or remains constant, the latter are topics with a decreasing number of publications. According to this view, none of the topics concerning PSSs can be defined as “cold” (see Figure 3 ). In particular, two topics (“PSS business models” and “PSS environmental and social impact”) show a steady increase in the number of publications. Furthermore, it can be observed that some topics show a leveling in the number of publications, see for example: “Industrial PSS”, “PSS requirement analysis” and “PSS & Servitization Process”.

4. Product-service systems research topics

Results reported in Table 3 provide a framework to understand and analyze the vast literature corpus concerning the concept of PSSs. Since the first definition of the PSS concept, several literature reviews concerning the field of research have been published ( Moro et al. , 2020 ). Most of these cover specific topics (see Table 4 ), whereas only a handful focused on an overall analysis of the PSS literature corpus ( Tukker and Tischner, 2006 ; Baines et al. , 2007 ; Wang et al. , 2011 ; Beuren et al. , 2013 ). Given the highly evaluative context, most of these reviews can be considered dated, being the most recent more than seven years old ( Moro et al. , 2020 ).

The following sections provide an updated overview of PSS research taking into account the theoretical framework presented in the previous sections and recent advances in PSS research.

4.1 PSS design

PSS design is the most influential topic in terms of the number of documents (21%). Studies discussing concepts, methodologies and tools aiming at supporting the design of innovative PSSs are considered part of this topic. The integration between the design process of products and services is intended as the basis for the development of novel PSS ( Aurich et al. , 2006 ; Barravecchia et al. , 2020a , b ). Due to the heterogeneous configuration of PSSs, their design is often characterized by systemic approaches useful to integrate product and service perspective ( Morelli, 2006 ). Several studies agree that the research topic on PSS design is not yet fully mature, a variety of dedicated methodologies have been developed; however, there are neither reference standards nor established models yet ( Vasantha et al. , 2012 ).

Table 5 reports some tools specifically developed to support PSS design during the different phases of the PSS life cycle. To face this aspect, recent contributions on the topic are focusing on integrating digital technologies within PSS.

4.2 PSS environmental and social impact

The offering of PSSs may provide new opportunities in increasing sustainability ( Tukker, 2004 ). Existing research recognizes the critical role of the combined offering of products and services to effectively decrease the consumption of natural resources, reduce environmental impacts and improve social equity and cohesion ( Vezzoli et al. , 2015 ). Moreover, as Tukker (2015) argued, PSSs are one of the most effective instruments for moving society toward a circular economy.

The creation of PSSs long-term relationships with customers provides incentives to extend the useful life of the product and to optimize resource utilization ( Evans et al. , 2007 ). The implementation of service-oriented strategies encourages advances in product life cycle management and facilitates the transition toward more sustainable consumption patterns ( Salazar et al. , 2015 ).

Table 6 introduces and summarizes articles concerning the management and the evaluation of PSS environmental and social impact. Common elements of these studies are: the multidimensional analysis of sustainability, the comparison with traditional business models and the analysis of impacts throughout the PSS life cycle.

4.3 PSS and servitization process

The concepts of servitization and PSSs are strongly related. While the former refers to the progressive transformation process, the latter refers to the output of the servitized manufacturing companies ( Beuren et al. , 2013 ).

The process of servitization has been defined as “the innovation of organization's capabilities and processes to better create mutual value through a shift from selling products to selling Product-Service Systems” ( Neely, 2009 ). It is possible to recognize a variety of forms of servitization ( Baines et al. , 2009 ; Mastrogiacomo et al. , 2020a ), defining the so-called “product-service continuum” ( Kowalkowski et al. , 2015 ), i.e. a continuum from traditional manufacturing companies to product-service providers able to manage all the PSS lifecycle and to offer complex solutions. Interaction between PSS delivery and the servitization process has been analyzed from a variety of points of view. Most characterizing works of the topic are reported in Table 7 . Issues addressed are wide-ranging, covering analysis of the diffusion of the servitization process, impact on the company's performance and challenges to be faced during the transformation process.

4.4 Sustainable PSS

LDA application resulted in the definition of a specific topic on the development of sustainable PSS, with a particular focus on increasing the environmental, social and economic sustainability of complex solutions composed of tangible elements (products) and intangible elements (services). This topic straddles two other topics identified by this analysis: “PSS design” and “PSS environmental and social impact”.

PSSs developed and designed with the explicit purpose of reducing environmental impact are labeled as sustainable PSS (SPSS) ( Roy, 2000 ; Vezzoli et al. , 2015 ) or eco-efficient PSS (EPSS) ( Ceschin, 2013 ). Several practical methodologies have been developed over the past years to address issues related to the development of sustainable PSS. Intending to summarize the main research works, Table 8 provides an overview of articles on the subject.

4.5 PSS business models

PSSs can be seen as new opportunities to create new business models. A growing number of manufacturing companies are shifting their focus to services as a new way of creating and capturing added value ( Adrodegari et al. , 2017b ). The emerging of complex offerings composed of products and services implies a transition from revenue streams mainly determined by selling products to revenue mechanisms based on customer relationship and service provisions ( Mathieu, 2001 ).

Several studies noted that business models are critical for the success of PSS implementations ( Kindström, 2010 ). This is confirmed by the results reported in section 3 , according to which a significant number of papers on PSS deal directly with business model issues, and the topic is also closely related to many other research topics.

There is a general consensus on categorizing the PSS business model into three types: product-oriented, use-oriented and result-oriented business models ( Tukker, 2004 ). These three types of business models can be applied in both consumer-oriented and B2B contexts (e.g. industrial PSS). Besides this, several business model classifications of PSS have been defined ( Barquet et al. , 2013 ; Adrodegari et al. , 2017a ).

Most of the studies related to PSS business models are highly related to a particular application or to the implementation of sustainable strategies integrating both the economic and the environmental aspects ( Boons et al. , 2013 ; Yang et al. , 2017 ). Table 9 shows the most characterizing papers on this topic.

4.6 PSS performance analysis

The performance analysis of PSS is a fundamental step in the implementation of innovative service-oriented strategies ( Qu et al. , 2016 ). There is a broad agreement among experts about the critical role played by the in-depth evaluation of PSSs performance in their management and productivity ( Baines et al. , 2007 ).

Additional challenges for the performance analysis are posed by the PSS heterogeneity being composed of tangible products, intangible services, infrastructures. Traditional tools developed for the analysis of products and services are not capable of capturing the complexity of PSS ( Rondini et al. , 2020 ) . To face this issue, several authors proposed practical approaches and methodologies for evaluating the performance of a PSS. Some of these are adaptations of pre-existing approaches ( Geng and Chu, 2012 ), while others have been explicitly developed for PSS ( Rondini et al. , 2020 ). Table 10 shows the most influential papers on this topic.

4.7 PSS requirements analysis

Given the complexity of value offerings composed of a mix of tangible and intangible elements, the requirement analysis carved out a niche in the scientific literature on PSS. The definition of PSS structure and requirements is a necessary preliminary stage for their design ( Wiesner et al. , 2017 ). The definition of the requirements of a PSS cannot disregard some essential elements: the consideration of the entire PSS life cycle, the demand for customized solutions, the involvement of different stakeholders in the value creation process and the additional services associated with the primary offering ( Berkovich et al. , 2011 ). Several attempts have been made to define novel methods or to adapt existing approaches to the problem of defining PSS requirements ( Cavalieri and Pezzotta, 2012 ). Table 11 shows the state-of-the-art on this topic.

4.8 Industrial PSS

According to Meier et al. (2010) , “an industrial product-service system (IPSS) is characterized by the integrated and mutually determined planning, development, provision and use of product and service shares including its immanent software components in business-to-business applications and represents a knowledge-intensive socio-technical system”.

A shift from product and technology-centered strategies toward the offering of complex solutions comprising both product and services is taking place also in contexts characterized by business-to-business relationships ( Meier et al. , 2011 ).

This topic has been analyzed from different perspectives (see Table 12 ). Apart from a few articles that have defined the concept of Industrial PSS, most of the articles are contaminated by other research topics, in particular by the requirements analysis, design and performance evaluation.

4.9 Relationship between topics

The results of the LDA algorithm allow the construction of a topic network graph, i.e. a directed graph in which nodes represent topics and directed arcs indicate relationships between topics. In detail, the graph shows an arc between two topics, say from “Topic A” to “Topic B”, when documents belonging to “Topic A” have an average probability of belonging to “Topic B” above a certain threshold. For the purpose of this analysis, the selected threshold was set to 5%. The resulting graph is shown in Figure 4 .

“PSS business models” and “PSS design” nodes show the most incoming arcs. This evidence suggests that these two topics are cross-cutting and intertwined with most of the others. The literature review confirms this evidence. The concept of business model and the process of developing and designing novel PSS are also addressed in many of the pivotal papers in other topics. For example, the provision of industrial PSS cannot be achieved without the definition of new business models for the specific industry context or the application of PSS design approaches.

Newer, less developed topics such as “Industrial PSS” have no incoming arcs, meaning it has limited relevance within the documents belonging to any other topics.

5. Future trends in product-service systems research

While previous sections provided structure and interpretation to PSS research, looking at past and present scientific production as well as the relationships between topics, the purpose of this section is to outline potential research trends and challenges yet to be tackled.

5.1 Smart PSS and digital servitization

In parallel with the spread of the concept of PSS and service-oriented business models, there has been a growing diffusion of digital technologies. In recent years, these two once distinct fields of research converged ( Zheng et al. , 2019 ). Digitalization supports companies in the development of new services, business models and innovative products. This process is generally defined as digital servitization ( Kohtamäki et al. , 2019 ). The outputs of the companies that faced the process of digital servitization are PSS that exploit digital technologies, commonly labeled as Smart-PSS, digital PSS, cyber-physical PSS, IoT-enabled PSS, digital-driven PSS ( Zheng et al. , 2019 ).

The LDA algorithm (see section 2 ) did not recognize a specific topic related to these issues, but assigned the articles dealing with these issues to the eight PSS research topics. This result tells us that technology and Smart PSS does not yet represent a pillar of PSS research. Despite this, given the growing importance of digital servitization in research and practice, this area will represent a major challenge for PSS research in all the eight identified topics.

In recent years, interest in smart PSS has grown exponentially (see Figure 5 ). According to data reported in Figure 5 , it can be inferred that Smart PSS is rapidly becoming “hot topic” in the coming years. This fact is also supported by the results of the LDA algorithm. From 2014, most topics have seen the emergence of prominent articles addressing issues related to the deployment of Smart PSS (see Table 13 ).

The analysis of the literature and debate with researchers and practitioners revealed several possible RQs still to be addressed. These potential RQs are reported in Table 14 by dividing them into the eight PSS research topics and the key phases of the PSS life cycle.

Current literature on traditional PSS guided the definition of these RQs. Problems already faced in the study of traditional PSS require new considerations to successfully address the new challenges posed by the implementation of Smart PSS. For instance, consider RQ 1.4, 1.6 and 1.10. These issues were considered critical ten years ago for the development of traditional PSS. The same issues became critical again with the emergence of Smart PSS. Similar considerations can be made for the other phases of the PSS life cycle. See for example RQ 2.4, 2.5, 2.11 for the Delivery phase and RQ 3.2, 3.4, 3.8 for the end-of-life phase of the PSS lifecycle.

Taken together, these challenges provide us with an overview of what is needed for full implementation of smart PSS and what are the main open issues requiring further investigation.

5.2 PSS and emerging technological issues

The previous section highlighted the challenges that need to be addressed to link PSS research with currently diffused digital technologies. However, researchers and companies must also prepare to face new challenges arising from the emergence of new technological paradigms that will invest the digital world and beyond. In this respect, an analysis of the relationships between the top technological trends and PSS research topics deserves more attention. Every year Gartner, a world's leading research and advisory company, highlights strategic trends, including the most promising emerging technologies and business innovations ( Panetta, 2019 ).

Table 15 relates each technological trend with the identified eight PSS research topics and reports the number of articles published in each area. Data were retrieved from Scopus database ( Scopus Elsevier, 2020 ). This table reveals that research on PSS related to emerging technological trends is still limited and particularly recent.

In detail, few studies investigated the relationship between hyperautomation and PSS, mainly focusing on the application of data mining algorithms for the evaluation and management of PSS ( Wiesner et al. , 2017 ; Shimomura et al. , 2018 ). Issues related to multiexperience were partially covered by analyzing the use of augmented reality equipment and smart devices for the maintenance and monitoring of industrial plants with a Product-Service approach ( Mourtzis et al. , 2017 ).

Transparency and traceability, empowered edge, distributed cloud and autonomous things received more, albeit limited, attention in various PSS research topics.

Recent studies associated the study of empowered edge to the topic of industrial PSS ( Liu et al. , 2019a ) and to the design of IoT-based PSSs ( Shao et al. , 2019 ). Few works have explored opportunities resulting from edge-computing and the implementation of IoT technologies in the servitization process ( Heinis et al. , 2018 ).

Issues related to transparency and traceability were mainly addressed in the study of sustainable consumption and production practices of PSS ( Pialot et al. , 2017 ; Sakao, 2019 ). The impact of distributed cloud on PSS strategies is an area partially covered. So far, the study of the distributed cloud paradigm related to the PSS concept resulted in the definition of smart PSSs design tools ( Zheng et al. , 2018 ). Finally, concerning autonomous things, some studies have analyzed their design in specific PSS application, such as autonomous vehicles ( Wang et al. , 2018 ) or collaborative robots ( Cordeiro, 2018 ).

Many technological trends (democratization, human augmentation, practical blockchain and AI security) resulted in having no connection with current PSS research.

Table 15 shows several uncovered areas and emphasizes potential issues that would be matter of research in the PSS field. If the debate on PSS is to be moved forward, a better understanding of the influence of emerging technologies on PSS research topics needs to be developed.

In the past twenty years, the PSS concept advanced in different contexts: product, service and software sector. Nowadays, as aforementioned, the concept of PSS and that of digital transformation are inevitably approaching each other. Business models based on the delivery of tangible products and intangible services contaminate a wide range of sectors and new emerging trends will not be unaffected. It is therefore crucial starting to understand how to leverage these technologies to develop more effective, innovative and sustainable PSS.

In addition to core topics presented in previous sections, further research could usefully attempt to shed light on how these new technologies could impact on the design, management and provision of PSS.

6. Discussion and conclusions

The analysis reported in this article provides insight into what PSS research has been over the past 20 years and offers a key to understanding how it might evolve in the near future.

Given the vastness of the body of literature on PSSs, machine learning and text mining techniques were applied to understand its structure.

Eight topics were identified as pillars of research related to PSSs: (1) PSS design; (2) PSS environmental and social impact; (3) PSS and servitization process; (4) sustainable PSS; (5) PSS business models; (6) PSS performance analysis; (7) PSS requirements analysis and (viii) industrial PSS. “PSS design” and “PSS environmental and social impact” are the two most discussed topics in terms of the number of articles published.

This research confirms the vital link between research on PSS and research on the development of environmentally and socially sustainable production systems. It also reinforces the notion that the advent of the PSS paradigm has necessitated the development of new business models capable of capturing the value generated by the joint offering of products and services. The topic related to PSS business models was found to be central to the literature on PSS and closely related to the other research topics.

Twenty years of research may seem like a long time, but it is very short compared to other fields of research. Therefore, we should expect continued research on the topics identified, delving into particular aspects and providing operational tools to implement, evaluate and improve PSS over time. Alongside the eight identified research topics, however, we can expect others to emerge. Section 5 attempted to identify some of them. The PSS technological aspect is likely to play a key role in the development of the field. Digitization and emerging technologies will contaminate the PSS concept and make it relevant for coming years.

The result of this study strengthens the idea that PSSs often pose significant challenges to a broad range of discipline, ranging from engineering to design and management. For this reason, a map for navigating the literature on PSS can be useful for a variety of subjects. Researchers and practitioners approaching this field can understand how PSS research is structured and can identify key topics and articles, open issues and research gaps that need to be addressed. Conversely, for researchers who are already addressing PSS-related issues, the results of this study may be useful in positioning their research within a framework that may allow them to identify new research opportunities. Finally, research funding organizations and institutions could use the proposed topic landscape to focus and target support and funding.

Further research will be addressed to deepen the analysis between strategic technology trends and PSS research topics.

Methodology of analysis of the PSS literature corpus

Perplexity of the topic model varying the number of topics

Publication trend in PSS research topics

Topic network graph of product-service systems research

Trend of publications on PSS and digital-related topics

Comparison between alternative literature survey methodologies

PSS topic landscape, description and top 15 keywords associated to each topic (keywords ordered by their relative importance)

Literature reviews related to specific PSS research topics

Tools and methodologies for PSS design

Tools and methodologies to manage and assess PSS sustainability

Articles on PSS and servitization process

Tools and methodologies to develop sustainable PSS

Articles on PSS business models

Articles on PSS performance analysis

Articles on PSS requirement analysis

Articles on industrial PSS

Articles concerning Smart PSS in the eight PSS research topics

Potential research questions (RQs) and challenges for Smart PSS research

Relationship between the top 10 Strategic Technology Trends in 2020 (Panetta, 2019) and PSS research topics. The symbol (*) highlights articles published from 2017 to 2019

Adrodegari , F. , Saccani , N. , Kowalkowski , C. and Vilo , J. ( 2017a ), “ PSS business model conceptualization and application ”, Production Planning and Control , Vol. 28 No. 15 , pp. 1251 - 1263 .

Adrodegari , F. , Pashou , T. and Saccani , N. ( 2017b ), “ Business model innovation: process and tools for service transformation of industrial firms ”, Procedia CIRP , Vol. 64 , pp. 103 - 108 .

Andriankaja , H. , Boucher , X. and Medini , K. ( 2018 ), “ A method to design integrated product-service systems based on the extended functional analysis approach ”, CIRP Journal of Manufacturing Science and Technology , Vol. 21 , pp. 120 - 139 .

Annarelli , A. , Battistella , C. and Nonino , F. ( 2016 ), “ Product service system: a conceptual framework from a systematic review ”, Journal of Cleaner Production , Vol. 139 , pp. 1011 - 1032 .

Antons , D. and Breidbach , C.F. ( 2018 ), “ Big data, big insights? Advancing service innovation and design with machine learning ”, Journal of Service Research , Vol. 21 No. 1 , pp. 17 - 39 .

Asmussen , C.B. and Møller , C. ( 2019 ), “ Smart literature review: a practical topic modelling approach to exploratory literature review ”, Journal of Big Data , Vol. 6 No. 1 , p. 93 .

Aurich , J.C. , Fuchs , C. and Wagenknecht , C. ( 2006 ), “ Life cycle oriented design of technical Product-Service Systems ”, Journal of Cleaner Production , Vol. 14 No. 17 , pp. 1480 - 1494 .

Azarenko , A. , Roy , R. , Shehab , E. and Tiwari , A. ( 2009 ), “ Technical product-service systems: some implications for the machine tool industry ”, Journal of Manufacturing Technology Management , Vol. 20 No. 5 , pp. 700 - 722 .

Baines , T.S. , Lightfoot , H.W. , Evans , S. , Neely , A. , Greenough , R. , Peppard , J. , Roy , R. , Shehab , E. , Braganza , A. , Tiwari , A. , Alcock , J.R. , Angus , J.P. , Basti , M. , Cousens , A. , Irving , P. , Johnson , M. , Kingston , J. , Lockett , H. , Martinez , V. , Michele , P. , Tranfield , D. , Walton , I.M. and Wilson , H. ( 2007 ), “ State-of-the-art in product-service systems ”, Proceedings of the Institution of Mechanical Engineers, Part B: Journal of Engineering Manufacture , Vol. 221 No. 10 , pp. 1543 - 1552 .

Baines , T.S. , Lightfoot , H.W. , Benedettini , O. and Kay , J.M. ( 2009 ), “ The servitization of manufacturing: a review of literature and reflection on future challenges ”, Journal of Manufacturing Technology Management , Vol. 20 No. 5 , pp. 547 - 567 .

Baines , T. , Lightfoot , H. , Smart , P. and Fletcher , S. ( 2013 ), “ Servitization of manufacture: exploring the deployment and skills of people critical to the delivery of advanced services ”, Journal of Manufacturing Technology Management , Vol. 24 No. 4 , pp. 637 - 646 .

Barquet , A.P.B. , de Oliveira , M.G. , Amigo , C.R. , Cunha , V.P. and Rozenfeld , H. ( 2013 ), “ Employing the business model concept to support the adoption of product-service systems (PSS) ”, Industrial Marketing Management , Vol. 42 No. 5 , pp. 693 - 704 .

Barravecchia , F. , Mastrogiacomo , L. and Franceschini , F. ( 2020a ), “ Categorizing quality determinants mining user-generated contents ”, Sustainability , Vol. 12 No. 23 , p. 9944 .

Barravecchia , Mastrogiacomo , L. and Franceschini , F. ( 2020b ), “ The Player-Interface method: an approach to support Product-Service Systems concept generation and prioritization ”, Journal of Engineering Design , Vol. 31 No. 5 , pp. 331 - 348 .

Berkovich , M. , Leimeister , J.M. and Krcmar , H. ( 2011 ), “ Requirements engineering for product service systems: a state of the art analysis ”, Business and Information Systems Engineering , Vol. 3 No. 6 , pp. 369 - 380 .

Berkovich , M. , Leimeister , J.M. , Hoffmann , A. and Krcmar , H. ( 2014 ), “ A requirements data model for product service systems ”, Requirements Engineering , Vol. 19 No. 2 , pp. 161 - 186 .

Besch , K. ( 2005 ), “ Product-service systems for office furniture: barriers and opportunities on the European market ”, Journal of Cleaner Production , Vol. 13 Nos 10–11 , pp. 1083 - 1094 .

Beuren , F.H. , Gomes Ferreira , M.G. and Cauchick Miguel , P.A. ( 2013 ), “ Product-service systems: a literature review on integrated products and services ”, Journal of Cleaner Production , Vol. 47 , pp. 222 - 231 .

Blei , D.M. ( 2012 ), “ Probabilistic topic models ”, Communications of the ACM , Vol. 55 No 4 , pp. 77 - 84 .

Blei , D.M. , Ng , A.Y. and Jordan , M.I. ( 2003 ), “ Latent Dirichlet allocation ”, Journal of machine Learning research , Vol. 3 , pp. 993 - 1022 .

Bo , Z. , Xiaohua , S. and Binhui , Z. ( 2019 ), “ SDIV: Service-Defined Intelligent Vehicle towards the 2020 urban mobility ”, Advanced Engineering Informatics , Vol. 39 , pp. 203 - 213 .

Boons , F. , Montalvo , C. , Quist , J. and Wagner , M. ( 2013 ), “ Sustainable innovation, business models and economic performance: an overview ”, Journal of Cleaner Production , Vol. 45 , pp. 1 - 8 .

Brambila-Macias , S.A. , Sakao , T. and Kowalkowski , C. ( 2018 ), “ Bridging the gap between engineering design and marketing: insights for research and practice in product/service system design ”, Design Science , Vol. 4 , pp. 1 - 61 .

Bressanelli , G. , Adrodegari , F. , Perona , M. and Saccani , N. ( 2018 ), “ Exploring how usage-focused business models enable circular economy through digital technologies ”, Sustainability (Switzerland) , Vol. 10 No. 3 , pp. 1 - 21 .

Bustinza , O.F. , Bigdeli , A.Z. , Baines , T. and Elliot , C. ( 2015 ), “ Servitization and competitive advantage : the importance of organizational structure and value chain position ”, Research Technology Management , Vol. 58 No. 5 , pp. 53 - 60 .

Carreira , R. , Patrício , L. , Jorge , R.N. and Magee , C.L. ( 2013 ), “ Development of an extended Kansei engineering method to incorporate experience requirements in product-service system design ”, Journal of Engineering Design , Vol. 24 No. 10 , pp. 738 - 764 .

Cavalieri , S. and Pezzotta , G. ( 2012 ), “ Product-service systems engineering: State of the art and research challenges ”, Computers in Industry , Vol. 63 No. 4 , pp. 278 - 288 .

Ceschin , F. ( 2013 ), “ Critical factors for implementing and diffusing sustainable product-service systems: insights from innovation studies and companies' experiences ”, Journal of Cleaner Production , Vol. 45 , pp. 74 - 88 .

Chierici , E. and Copani , G. ( 2018 ), “ Remanufacturing with upgrade PSS for new sustainable business models ”, CIRP Journal of Manufacturing Science and Technology , Vol. 29 , Part B , pp. 245 - 256 .

Chiu , M.-C. , Chu , C.-Y. and Chen , C.-C. ( 2018 ), “ An integrated product service system modelling methodology with a case study of clothing industry ”, International Journal of Production Research , Vol. 56 No. 6 , pp. 2388 - 2409 .

Chou , C.-J. , Chen , C.-W. and Conley , C. ( 2015 ), “ An approach to assessing sustainable product-service systems ”, Journal of Cleaner Production , Vol. 86 , pp. 277 - 284 .

Clarkson , P. and Robinson , T. ( 1999 ), ‘ Towards improved language model evaluation measures ’, in Sixth European Conference on Speech Communication and Technology .

Cordeiro , C.M. ( 2018 ), “ Manifestation of intent in product-service systems. A study of type of sensing in collaborative robots ”, Revue Europeenne d' Economie et Management des Services , Vol. 1 No. 5 , pp. 97 - 131 .

Coreynen , W. , Matthyssens , P. , De Rijck , R. and Dewit , I. ( 2017 ), “ Internal levers for servitization: how product-oriented manufacturers can upscale product-service systems ”, International Journal of Production Research , Vol. 56 No. 6 , pp. 2184 - 2198 .

Dachs , B. , Biege , S. , Borowiecki , M. , Lay , G. , Jäger , A. and Schartinger , D. ( 2014 ), “ Servitisation of European manufacturing: evidence from a large scale database ”, The Service Industries Journal , Vol. 34 No. 1 , pp. 5 - 23 .

Dahmani , S. , Boucher , X. , Peillon , S. and Besombes , B. ( 2016 ), “ A reliability diagnosis to support servitization decision-making process ”, Journal of Manufacturing Technology Management , Vol. 27 No. 4 , pp. 502 - 534 .

Datta , P.P. and Roy , R. ( 2011 ), “ Operations strategy for the effective delivery of integrated industrial product-service offerings: two exploratory defence industry case studies ”, International Journal of Operations and Production Management , Vol. 31 No. 5 , pp. 579 - 603 .

Ding , K. , Jiang , P. and Zheng , M. ( 2017 ), “ Environmental and economic sustainability-aware resource service scheduling for industrial product service systems ”, Journal of Intelligent Manufacturing , Vol. 28 No. 6 , pp. 1303 - 1316 .

Durugbo , C. ( 2014 ), “ Strategic framework for industrial product-service co-design: findings from the microsystems industry ”, International Journal of Production Research , Vol. 52 No. 10 , pp. 2881 - 2900 .

Erkoyuncu , J.A. , Roy , R. , Shehab , E. and Cheruvu , K. ( 2011 ), “ Understanding service uncertainties in industrial product-service system cost estimation ”, International Journal of Advanced Manufacturing Technology , Vol. 52 Nos 9–12 , pp. 1223 - 1238 .

Erkoyuncu , J.A. , Roy , R. , Shehab , E. , Durugbo , C. , Khan , S. and Datta , P. ( 2019 ), “ An effective uncertainty based framework for sustainable industrial product-service system transformation ”, Journal of Cleaner Production , Vol. 208 , pp. 160 - 177 .

Evans , S. , Partidário , P.J. and Lambert , J. ( 2007 ), “ Industrialization as a key element of sustainable product-service solutions ”, International Journal of Production Research , Vol. 45 Nos 18–19 , pp. 4225 - 4246 .

Fang , D. , Yang , H. , Gao , B. and Li , X. ( 2018 ), “ Discovering research topics from library electronic references using latent Dirichlet allocation ”, Library Hi Tech , Vol. 36 No. 3 , pp. 400 - 410 .

Fargnoli , M. and Haber , N. ( 2019 ), “ A practical ANP-QFD methodology for dealing with requirements' inner dependency in PSS development ”, Computers and Industrial Engineering , Vol. 127 , pp. 536 - 548 .

Geng , X. and Chu , X. ( 2012 ), “ A new importance-performance analysis approach for customer satisfaction evaluation supporting PSS design ”, Expert Systems with Applications , Vol. 39 No. 1 , pp. 1492 - 1502 .

Geng , X. , Chu , X. , Xue , D. and Zhang , Z. ( 2010 ), “ An integrated approach for rating engineering characteristics' final importance in product-service system development ”, Computers and Industrial Engineering , Vol. 59 No. 4 , pp. 585 - 594 .

Geum , Y. and Park , Y. ( 2011 ), “ Designing the sustainable product-service integration: a product-service blueprint approach ”, Journal of Cleaner Production , Vol. 19 No. 14 , pp. 1601 - 1614 .

Goedkoop , M.J. , van Halen , C.J.G. , te Riele , H.R.M. and Rommens , P.J.M. ( 1999 ), Product Service Systems, Ecological and Economic Basics , Report for Dutch Ministries of Environment and Economic Affairs .

Griffiths , T.L. and Steyvers , M. ( 2004 ), “ Finding scientific topics ”, Proceedings of the National Academy of Sciences , Vol. 101 suppl 1 , pp. 5228 - 5235 .

Hansch Beuren , F. and Cauchick Miguel , P.A. ( 2013 ), “ First steps to construct a life cycle framework for PSS ”, in 22nd International Conference on Production Research, ICPR 2013 .

Harzing , A.-W. and Alakangas , S. ( 2016 ), “ Google scholar, Scopus and the web of science: a longitudinal and cross-disciplinary comparison ”, Scientometrics , Vol. 106 No. 2 , pp. 787 - 804 .

Heinis , T.B. , Loy , C.L. and Meboldt , M. ( 2018 ), “ Improving usage metrics for pay-per-use pricing with IoT technology and machine learning: IoT technology and machine learning can identify and capture advanced metrics that make pay-per-use servitization models viable for a wider range of applications ”, Research Technology Management , Vol. 61 No. 5 , pp. 32 - 40 .

Joore , P. and Brezet , H. ( 2015 ), “ A multilevel design model: the mutual relationship between product-service system development and societal change processes ”, Journal of Cleaner Production , Vol. 97 , pp. 92 - 105 .

Juehling , E. , Torney , M. , Herrmann , C. and Droeder , K. ( 2010 ), “ Integration of automotive service and technology strategies ”, CIRP Journal of Manufacturing Science and Technology , Vol. 3 No. 2 , pp. 98 - 106 .

Karlsson , A. , Larsson , L. and Öhrwall Rönnbäck , A. ( 2017 ), “ Product-service system innovation capabilities: linkages between the fuzzy front end and subsequent development phases ”, International Journal of Production Research , Vol. 56 No. 6 , pp. 2218 - 2232 .

Kim , K.-J. , Lim , C.-H. , Heo , J.-Y. , Lee , D.-H.H. , Hong , Y.-S. and Park , K. ( 2013 ), “ An evaluation scheme for product-service system models with a lifecycle consideration from customer's perspective ”, in Re-Engineering Manufacturing for Sustainability , pp. 69 - 74 .

Kindström , D. ( 2010 ), “ Towards a service-based business model – key aspects for future competitive advantage ”, European Management Journal , Vol. 28 No. 6 , pp. 479 - 490 .

Kjaer , L.L. , Pagoropoulos , A. , Schmidt , J.H. and McAloone , T.C. ( 2016 ), “ Challenges when evaluating product/service-systems through life cycle assessment ”, Journal of Cleaner Production , Vol. 120 , pp. 95 - 104 .

Kohtamäki , M. , Parida , V. , Oghazi , P. , Gebauer , H. and Baines , T. ( 2019 ), “ Digital servitization business models in ecosystems: a theory of the firm ”, Journal of Business Research , Vol. 104 , pp. 380 - 392 .

Kowalkowski , C. , Windahl , C. , Kindström , D. and Gebauer , H. ( 2015 ), “ What service transition? Rethinking established assumptions about manufacturers' service-led growth strategies ”, Industrial Marketing Management , Vol. 45 , pp. 59 - 69 .

Krucken , L. and Meroni , A. ( 2006 ), “ Building stakeholder networks to develop and deliver product-service-systems: practical experiences on elaborating pro-active materials for communication ”, Journal of Cleaner Production , Vol. 14 No. 17 , pp. 1502 - 1508 .

Kuo , T.C. ( 2011 ), “ Simulation of purchase or rental decision-making based on product service system ”, International Journal of Advanced Manufacturing Technology , Vol. 52 Nos 9–12 , pp. 1239 - 1249 .

Lee , S. , Geum , Y. , Lee , H. and Park , Y. ( 2012 ), “ Dynamic and multidimensional measurement of product-service system (PSS) sustainability: a triple bottom line (TBL)-based system dynamics approach ”, Journal of Cleaner Production , Vol. 32 , pp. 173 - 182 .

Lee , H. , Seo , H. and Geum , Y. ( 2018 ), “ Uncovering the topic landscape of product-service system research: from sustainability to value creation ”, Sustainability , Vol. 10 No. 4 , p. 911 .

Lee , C.-H. , Chen , C.-H. and Trappey , A.J.C. ( 2019 ), “ A structural service innovation approach for designing smart product service systems: case study of smart beauty service ”, Advanced Engineering Informatics , Vol. 40 , pp. 154 - 167 .

Léo , P.-Y. and Philippe , J. ( 2001 ), “ Offer of services by goods exporters: strategic and marketing dimensions ”, The Service Industries Journal , Vol. 21 No. 2 , pp. 91 - 116 .

Lerch , C. and Gotsch , M. ( 2015 ), “ Digitalized product-service systems in manufacturing firms : a case study analysis ”, Research Technology Management , Vol. 58 No. 5 , pp. 45 - 52 .

Li , A.Q. , Kumar , M. , Claes , B. and Found , P. ( 2020 ), “ The state-of-the-art of the theory on Product-Service Systems ”, International Journal of Production Economics , Vol. 222 , p. 107491 .

Lightfoot , H. , Baines , T. and Smart , P. ( 2013 ), “ The servitization of manufacturing: a systematic literature review of interdependent trends ”, International Journal of Operations and Production Management , Vol. 33 No. 11 , pp. 1408 - 1434 .

Lim , C.-H. , Kim , K.-J. , Hong , Y.-S. and Park , K. ( 2012 ), “ PSS Board: a structured tool for product-service system process visualization ”, Journal of Cleaner Production , Vol. 37 , pp. 42 - 53 .

Lindahl , M. , Sundin , E. and Sakao , T. ( 2014 ), “ Environmental and economic benefits of Integrated Product Service Offerings quantified with real business cases ”, Journal of Cleaner Production , Vol. 64 , pp. 288 - 296 .

Lingegård , S. and Lindahl , M. ( 2015 ), “ Integrated product service offerings for rail infrastructure–benefits and challenges regarding knowledge transfer and cultural change in a Swedish case ”, Journal of Cleaner Production , Vol. 98 , pp. 166 - 174 .

Liu , B. , Zhang , Y. , Zhang , G. and Zheng , P. ( 2019a ), “ Edge-cloud orchestration driven industrial smart product-service systems solution design based on CPS and IIoT ”, Advanced Engineering Informatics , Vol. 42 , p. 100984 .

Liu , Z. , Ming , X. and Song , W. ( 2019b ), “ A framework integrating interval-valued hesitant fuzzy DEMATEL method to capture and evaluate co-creative value propositions for smart PSS ”, Journal of Cleaner Production , Vol. 215 , pp. 611 - 625 .

Lu , D. , Lai , I.K.W. and Liu , Y. ( 2019 ), “ The consumer acceptance of smart product-service systems in sharing economy: the effects of perceived interactivity and particularity ”, Sustainability (Switzerland) , Vol. 11 No. 3 , p. 928 .

Mahut , F. , Daaboul , J. , Bricogne , M. and Eynard , B. ( 2017 ), “ Product-Service Systems for servitization of the automotive industry: a literature review ”, International Journal of Production Research , Vol. 55 No. 7 , pp. 2102 - 2120 .

Manzini , E. and Vezzoli , C. ( 2003 ), “ A strategic design approach to develop sustainable product service systems: examples taken from the ‘environmentally friendly innovation’ Italian prize ”, Journal of Cleaner Production , Vol. 11 No. 8 , pp. 851 - 857 .

Martinez , V. , Bastl , M. , Kingston , J. and Evans , S. ( 2010 ), “ Challenges in transforming manufacturing organisations into product-service providers ”, Journal of Manufacturing Technology Management , Vol. 21 No. 4 , pp. 449 - 469 .

Mastrogiacomo , Barravecchia , F. and Franceschini , F. ( 2019 ), “ A worldwide survey on manufacturing servitization ”, The International Journal of Advanced Manufacturing Technology , Vol. 103 , pp. 3927 - 3942 .

Mastrogiacomo , Barravecchia , F. and Franceschini , F. ( 2020a ), “ Definition of a conceptual scale of servitization: proposal and preliminary results ”, CIRP Journal of Manufacturing Science and Technology , Vol. 29 Part B , pp. 141 - 156 .

Mastrogiacomo , Barravecchia , F. and Franceschini , F. ( 2020b ), “ Enabling factors of manufacturing servitization: empirical analysis and implications for strategic positioning ”, Proceedings of the Institution of Mechanical Engineers, Part B: Journal of Engineering Manufacture , Vol. 234 No. 9 , pp. 1258 - 1270 .

Mastrogiacomo , L. , Barravecchia , F. , Franceschini , F. and Marimon , F. ( 2021 ), “ Mining quality determinants of product-service systems from unstructured user-generated contents: the case of car-sharing ”, Quality Engineering , In press , doi: 10.1080/08982112.2021.1877305 .

Mathieu , V. ( 2001 ), “ Service strategies within the manufacturing sector: benefits, costs and partnership ”, International Journal of Service Industry Management , Vol. 12 No. 5 , pp. 451 - 475 .

Matschewsky , J. , Lindahl , M. and Sakao , T. ( 2020 ), “ Capturing and enhancing provider value in product-service systems throughout the lifecycle: a systematic approach ”, CIRP Journal of Manufacturing Science and Technology , Vol. 29 Part B , pp. 191 - 204 .

Maussang , N. , Zwolinski , P. and Brissaud , D. ( 2009 ), “ Product-service system design methodology: from the PSS architecture design to the products specifications ”, Journal of Engineering Design , Vol. 20 No. 4 , pp. 349 - 366 .

Maxwell , D. , Sheate , W. and van der Vorst , R. ( 2006 ), “ Functional and systems aspects of the sustainable product and service development approach for industry ”, Journal of Cleaner Production , Vol. 14 No. 17 , pp. 1466 - 1479 .

Meier , H. , Roy , R. and Seliger , G. ( 2010 ), “ Industrial Product-Service Systems-IPS2 ”, CIRP Annals - Manufacturing Technology , Vol. 59 No. 2 , pp. 607 - 627 .

Meier , H. , Völker , O. and Funke , B. ( 2011 ), “ Industrial Product-Service Systems (IPS2) : paradigm shift by mutually determined products and services ”, International Journal of Advanced Manufacturing Technology , Vol. 52 Nos 9–12 , pp. 1175 - 1191 .

Meyer , D. , Hornik , K. and Feinerer , I. ( 2008 ), “ Text mining infrastructure in R ”, Journal of Statistical Software , Vol. 25 No. 5 , pp. 1 - 54 .

Mont , O. and Tukker , A. ( 2006 ), “ Product-Service Systems: reviewing achievements and refining the research agenda ”, Journal of Cleaner Production , Vol. 14 No. 17 , pp. 1451 - 1454 .

Morelli , N. ( 2006 ), “ Developing new product service systems (PSS): methodologies and operational tools ”, Journal of Cleaner Production , Vol. 14 No. 17 , pp. 1495 - 1501 .

Morelli , N. ( 2009 ), “ Service as value co-production: reframing the service design process ”, Journal of Manufacturing Technology Management , Vol. 20 No. 5 , pp. 568 - 590 .

Moro , S. , Cauchick-Migue , P.A. and de Sousa Mendes , G.H. ( 2020 ), “ Product-service systems benefits and barriers: an overview of literature review papers ”, International Journal of Industrial Engineering and Management , Vol. 11 No. 1 , pp. 61 - 70 .

Mourtzis , D. , Fotia , S. and Vlachou , E. ( 2017 ), “ Lean rules extraction methodology for lean PSS design via key performance indicators monitoring ”, Journal of Manufacturing Systems , Vol. 42 , pp. 233 - 243 .

Mourtzis , D. , Fotia , S. , Boli , N. and Vlachou , E. ( 2018 ), “ An approach for the modelling and quantification of PSS customisation ”, International Journal of Production Research , Vol. 56 No. 3 , pp. 1137 - 1153 .

Neely , A. ( 2009 ), “ Exploring the financial consequences of the servitization of manufacturing ”, Operations Management Research , Vol. 1 No. 2 , pp. 103 - 118 .

Ng , I.C.L. , Maull , R. and Yip , N. ( 2009 ), “ Outcome-based contracts as a driver for systems thinking and service-dominant logic in service science: evidence from the defence industry ”, European Management Journal , Vol. 27 No. 6 , pp. 377 - 387 .

Opresnik , D. and Taisch , M. ( 2015 ), “ The value of big data in servitization ”, International Journal of Production Economics , Vol. 165 , pp. 174 - 184 .

Pan , J.-N. and Nguyen , H.T.N. ( 2015 ), “ Achieving customer satisfaction through product-service systems ”, European Journal of Operational Research , Vol. 247 No. 1 , pp. 179 - 190 .

Panetta , K. ( 2019 ), Gartner Top 10 Strategic Technology Trends for 2020 , available at: https://www.gartner.com/smarterwithgartner/gartner-top-10-strategic-technology-trends-for-2020/ .

Parida , V. , Sjödin , D.R. , Wincent , J. and Kohtamäki , M. ( 2014 ), “ Mastering the transition to product-service provision: insights into business models, Learning activities, and capabilities ”, Research Technology Management , Vol. 57 No. 3 , pp. 44 - 52 .

Pereira , V.R. , Kreye , M.E. and Carvalho , M.M. ( 2019 ), “ Customer-pulled and provider-pushed pathways for product-service system: the contingent effect of the business ecosystems ”, Journal of Manufacturing Technology Management , Vol. 30 No. 4 , pp. 729 - 747 .

Pezzotta , G. , Cavalieri , S. and Gaiardelli , P. ( 2012 ), “ A spiral process model to engineer a product service system: an explorative analysis through case studies ”, CIRP Journal of Manufacturing Science and Technology , Vol. 5 No. 3 , pp. 214 - 225 .

Pezzotta , G. , Pirola , F. , Rondini , A. , Pinto , R. and Ouertani , M.-Z. ( 2016 ), “ Towards a methodology to engineer industrial product-service system – evidence from power and automation industry ”, CIRP Journal of Manufacturing Science and Technology , Vol. 15 , pp. 19 - 32 .

Pezzotta , G. , Sassanelli , C. , Pirola , F. , Sala , R. , Rossi , M. , Fotia , S. , Koutoupes , A. , Terzi , S. and Mourtzis , D. ( 2018 ), “ The product service system lean design methodology (PSSLDM) integrating product and service components along the whole PSS lifecycle ”, Journal of Manufacturing Technology Management , Vol. 29 No. 8 , pp. 1270 - 1295 .

Pialot , O. , Millet , D. and Bisiaux , J. ( 2017 ), “ ‘Upgradable PSS’: clarifying a new concept of sustainable consumption/production based on upgradablility ”, Journal of Cleaner Production , Vol. 141 , pp. 538 - 550 .

Qu , M. , Yu , S. , Chen , D. , Chu , J. and Tian , B. ( 2016 ), “ State-of-the-art of design, evaluation, and operation methodologies in product service systems ”, Computers in Industry , Vol. 77 , pp. 1 - 14 .

Reim , W. , Parida , V. and Örtqvist , D. ( 2015 ), “ Product-Service Systems (PSS) business models and tactics - a systematic literature review ”, Journal of Cleaner Production , Vol. 97 , pp. 61 - 75 .

Rodríguez , A.E. , Pezzotta , G. , Pinto , R. and Romero , D. ( 2020 ), “ A comprehensive description of the product-service systems' cost estimation process: an integrative review ”, International Journal of Production Economics , Vol. 221 , 107481 .

Rondini , A. , Tornese , F. , Gnoni , M.G. , Pezzotta , G. and Pinto , R. ( 2017 ), “ Hybrid simulation modelling as a supporting tool for sustainable product service systems: a critical analysis ”, International Journal of Production Research , Vol. 55 No. 23 , pp. 6932 - 6945 .

Rondini , A. , Bertoni , M. and Pezzotta , G. ( 2020 ), “ At the origins of product service systems: supporting the concept assessment with the engineering value assessment method ”, CIRP Journal of Manufacturing Science and Technology , Vol. 29 Part B , pp. 157 - 175 .

Roy , R. ( 2000 ), “ Sustainable product-service systems ”, Futures , Vol. 32 Nos 3–4 , pp. 289 - 299 .

Roy , R. and Cheruvu , K.S. ( 2009 ), “ A competitive framework for industrial product-service systems ”, International Journal of Internet Manufacturing and Services , Vol. 2 No. 1 , pp. 4 - 29 .

Sakao , T. ( 2019 ), “ Research series review for transdisciplinarity assessment-validation with sustainable consumption and production research ”, Sustainability (Switzerland) , Vol. 11 No. 19 , p. 5250 .

Sakao , T. and Lindahl , M. ( 2012 ), “ A value based evaluation method for product/service system using design information ”, CIRP Annals - Manufacturing Technology , Vol. 61 No. 1 , pp. 51 - 54 .

Salazar , C. , Lelah , A. and Brissaud , D. ( 2015 ), “ Eco-designing product service systems by degrading functions while maintaining user satisfaction ”, Journal of Cleaner Production , Vol. 87 , pp. 452 - 462 .

Schweitzer , E. and Aurich , J.C. ( 2010 ), “ Continuous improvement of industrial product-service systems ”, CIRP Journal of Manufacturing Science and Technology , Vol. 3 No. 2 , pp. 158 - 164 .

Scopus Elsevier ( 2020 ), Scopus Content Converage , available at: http://www.scopus.com ( accessed 15 January 2020 ).

Shao , S. , Xu , G. and Li , M. ( 2019 ), “ The design of an IoT-based route optimization system: a smart product-service system (SPSS) approach ”, Advanced Engineering Informatics , Vol. 42 , pp. 1 - 9 .

Shimomura , Y. , Hara , T. and Arai , T. ( 2009 ), “ A unified representation scheme for effective PSS development ”, CIRP Annals - Manufacturing Technology , Vol. 58 No. 1 , pp. 379 - 382 .

Shimomura , Y. , Nemoto , Y. and Kimita , K. ( 2015 ), “ A method for analysing conceptual design process of product-service systems ”, CIRP Annals - Manufacturing Technology , Vol. 64 No. 1 , pp. 145 - 148 .

Shimomura , Y. , Nemoto , Y. , Ishii , T. and Nakamura , T. ( 2018 ), “ A method for identifying customer orientations and requirements for product–service systems design ”, International Journal of Production Research , Vol. 56 No. 7 , pp. 2585 - 2595 .

Sholihah , M. , Maezono , T. , Mitake , Y. and Shimomura , Y. ( 2019 ), “ PSS strategic alignment: linking service transition strategy with PSS business model ”, Sustainability , Vol. 11 No. 22 , p. 6245 .

Song , W. ( 2017 ), “ Requirement management for product-service systems: status review and future trends ”, Computers in Industry , Vol. 85 , pp. 11 - 22 .

Song , W. and Sakao , T. ( 2017 ), “ A customization-oriented framework for design of sustainable product/service system ”, Journal of Cleaner Production , Vol. 140 Part 3 , pp. 1672 - 1685 .

Song , W. , Ming , X. , Han , Y. and Wu , Z. ( 2013 ), “ A rough set approach for evaluating vague customer requirement of industrial product-service system ”, International Journal of Production Research , Vol. 51 No. 22 , pp. 6681 - 6701 .

Sousa-Zomer , T.T. and Miguel , P.A.C. ( 2018 ), “ The main challenges for social life cycle assessment (SLCA) to support the social impacts analysis of product-service systems ”, The International Journal of Life Cycle Assessment , Vol. 23 No. 3 , pp. 607 - 616 .

Takenaka , T. , Yamamoto , Y. , Fukuda , K. , Kimura , A. and Ueda , K. ( 2016 ), “ Enhancing products and services using smart appliance networks ”, CIRP Annals - Manufacturing Technology , Vol. 65 No. 1 , pp. 397 - 400 .

Tukker , A. ( 2004 ), “ Eight types of product-service system: eight ways to sustainability? Experiences from suspronet ”, Business Strategy and the Environment , Vol. 13 No. 4 , pp. 246 - 260 .

Tukker , A. ( 2015 ), “ Product services for a resource-efficient and circular economy - a review ”, Journal of Cleaner Production , Vol. 97 , pp. 76 - 91 .

Tukker , A. and Tischner , U. ( 2006 ), “ Product-services as a research field: past, present and future. Reflections from a decade of research ”, Journal of Cleaner Production , Vol. 14 No. 17 , pp. 1552 - 1556 .

Ulaga , W. and Reinartz , W.J. ( 2011 ), “ Hybrid offerings: how manufacturing firms combine goods and services successfully ”, Journal of Marketing , Vol. 75 No. 6 , pp. 5 - 23 .

Valencia , A. , Mugge , R. , Schoormans , J.P.L. and Schifferstein , H.N.J. ( 2015 ), “ The design of smart product-service systems (PSSs): an exploration of design characteristics ”, International Journal of Design , Vol. 9 No. 1 , pp. 13 - 28 .

Vasantha , G.V.A. , Roy , R. , Lelah , A. and Brissaud , D. ( 2012 ), “ A review of product-service systems design methodologies ”, Journal of Engineering Design , Vol. 23 No. 9 , pp. 635 - 659 .

Vezzoli , C. , Ceschin , F. , Diehl , J.C. and Kohtala , C. ( 2015 ), “ New design challenges to widely implement ‘Sustainable Product-Service Systems’ ”, Journal of Cleaner Production , Vol. 97 , pp. 1 - 12 .

Visnjic Kastalli , I. and Van Looy , B. ( 2013 ), “ Servitization: disentangling the impact of service business model innovation on manufacturing firm performance ”, Journal of Operations Management , Vol. 31 No. 4 , pp. 169 - 180 .

Wallin , J. , Parida , V. and Isaksson , O. ( 2015 ), “ Understanding product-service system innovation capabilities development for manufacturing companies ”, Journal of Manufacturing Technology Management , Vol. 26 No. 5 , pp. 763 - 787 .

Wang , P.P. , Ming , X.G. , Li , D. , Kong , F.B. , Wang , L. and Wu , Z.Y. ( 2011 ), “ Status review and research strategies on product-service systems ”, International Journal of Production Research , Vol. 49 No. 22 , pp. 6863 - 6883 .

Wang , W. , Zhou , F. , Li , W. and Budd , J. ( 2018 ), “ Designing the product-service system for autonomous vehicles ”, IT Professional , Vol. 20 No. 6 , pp. 62 - 69 .

Wang , Z. , Chen , C.-H. , Zheng , P. , Li , X. and Khoo , L.P. ( 2019 ), “ A graph-based context-aware requirement elicitation approach in smart product-service systems ”, International Journal of Production Research , pp. 1 - 17 , doi: 10.1080/00207543.2019.1702227 .

Wiesner , S. , Marilungo , E. and Thoben , K.-D. ( 2017 ), “ Cyber-physical product-service systems – challenges for requirements engineering ”, International Journal of Automation Technology , Vol. 11 No. 1 , pp. 17 - 28 .

Yang , M. and Evans , S. ( 2019 ), “ Product-service system business model archetypes and sustainability ”, Journal of Cleaner Production , Vol. 220 , pp. 1156 - 1166 .

Yang , M. , Evans , S. , Vladimirova , D. and Rana , P. ( 2017 ), “ Value uncaptured perspective for sustainable business model innovation ”, Journal of Cleaner Production , Vol. 140 Part 3 , pp. 1794 - 1804 .

Zancul , E.D.S. , Takey , S.M. , Barquet , A.P.B. , Kuwabara , L.H. , Cauchick Miguel , P.A. and Rozenfeld , H. ( 2016 ), “ Business process support for IoT based product-service systems (PSS) ”, Business Process Management Journal , Vol. 22 No. 2 , pp. 305 - 323 .

Zheng , P. , Lin , T.-J. , Chen , C.-H. and Xu , X. ( 2018 ), “ A systematic design approach for service innovation of smart product-service systems ”, Journal of Cleaner Production , Vol. 201 , pp. 657 - 667 .

Zheng , P. , Wang , Z. , Chen , C.-H. and Khoo , L.P. ( 2019 ), “ A survey of smart product-service systems: key aspects, challenges and future perspectives ”, Advanced Engineering Informatics , Vol. 42 , p. 100973 .

Zhu , Q.Q. , Jiang , P.Y. , Huang , G.Q. and Qu , T. ( 2011 ), “ Implementing an industrial product-service system for CNC machine tool ”, International Journal of Advanced Manufacturing Technology , Vol. 52 Nos 9–12 , pp. 1133 - 1147 .

Ziaee Bigdeli , A. , Baines , T. , Schroeder , A. , Brown , S. , Musson , E. , Guang Shi , V. and Calabrese , A. ( 2018 ), “ Measuring servitization progress and outcome: the case of ‘advanced services ”, Production Planning and Control , Vol. 29 No. 4 , pp. 315 - 332 .

Acknowledgements

Funding : This work has been partially supported by “Ministero dell'Istruzione, dell'Università e della Ricerca” Award “TESUN-83486178370409 finanziamento dipartimenti di eccellenza CAP. 1694 TIT. 232 ART. 6”.

Corresponding author

About the authors.

Federico Barravecchia is a Postdoctoral Researcher at the Department of Management and Production Engineering at Politecnico di Torino. His main scientific interests currently concern the areas of Product-Servic Systems, Service Quality and Quality Engineering.

Fiorenzo Franceschini is a Professor of Quality Engineering at Politecnico di Torino (Italy) – Department of Management and Production Engineering. He is author or coauthor of 9 books and more than 270 many published papers in prestigious scientific journals and international conference proceedings. His current research interests are in the areas of Product-Service Systems, Quality Engineering and Performance Measurements Systems.

Luca Mastrogiacomo is an Associate Professor at the Department of Management and Production Engineering at Politecnico di Torino. His main scientific interests currently concern the areas of Product-Service Systems and Quality Engineering.

Mohamed Zaki is the Deputy Director of the Cambridge Service Alliance at the University of Cambridge (UK), a research center that brings together the world's leading firms and academics to address service challenges. Mohamed's research interests lie in the field of machine learning and its application on Digital Manufacturing and services. He uses an interdisciplinary approach of data science techniques to address a range of real organizations' problems such as measuring and managing customer experience and customer loyalty. Other research interests include digital service transformation strategy and data-driven business models.

Related articles

We’re listening — tell us what you think, something didn’t work….

Report bugs here

All feedback is valuable

Please share your general feedback

Join us on our journey

Platform update page.

Visit emeraldpublishing.com/platformupdate to discover the latest news and updates

Questions & More Information

Answers to the most commonly asked questions here

• Call For Papers 2023
• Online Submission
• Authors GuideLines
• Publication Charge
• Registration
• Certificate Request
• Thesis Publication
• Journal Indexing
• How to publish research paper
• Research Paper Topics
• Editorial Board
• Peer Review & Publication Policy
• Ethics Policy
• Editorial Scope
• 2023- List of Thesis
• Online Thesis Submission
• Thesis Publication Detail
• January 2024 Edition
• February 2024 Edition
• March 2024 Edition
• April 2024 Edition
• January 2023 Edition
• February 2023 Edition
• March 2023 Edition
• April 2023 Edition
• May 2023 Edition
• June 2023 Edition
• July 2023 Edition
• August 2023 Edition
• September 2023 Edition
• October 2023 Edition
• November 2023 Edition
• December 2023 Edition
• January 2022 Edition
• February 2022 Edition
• March 2022 Edition
• April 2022 Edition
• May 2022 Edition
• June 2022 Edition
• July 2022 Edition
• August 2022 Edition
• September 2022 Edition
• October 2022 Edition
• November 2022 Edition
• December 2022 Edition
• January 2020 Edition
• February 2020 Edition
• March 2020 Edition
• April 2020 Edition
• May 2020 Edition
• June 2020 Edition
• July 2020 Edition
• August 2020 Edition
• September 2020 Edition
• October 2020 Edition
• November 2020 Edition
• December 2020 Edition
• January 2019 Edition
• February 2019 Edition
• March 2019 Edition
• April 2019 Edition
• May 2019 Edition
• June 2019 Edition
• July 2019 Edition
• August 2019 Edition
• September 2019 Edition
• October 2019 Edition
• November 2019 Edition
• December 2019 Edition
• January 2018 Edition
• February 2018 Edition
• March 2018 Edition
• April 2018 Edition
• May 2018 Edition
• June 2018 Edition
• July 2018 Edition
• August 2018 Edition
• September 2018 Edition
• October 2018 Edition
• November 2018 Edition
• December 2018 Edition
• January 2017 Edition
• February 2017 Edition
• March 2017 Edition
• April 2017 Edition
• May 2017 Edition
• June 2017 Edition
• July 2017 Edition
• August 2017 Edition
• September 2017 Edition
• October 2017 Edition
• November 2017 Edition
• December 2017 Edition
• Archives- All Editions
• Registration Form
• Copyright Transfer

JOURNAL CALL FOR PAPER

• Call for Paper 2023
• Publication Indexing

AUTHORHS & EDITORS

• Author Guidelines

THESIS PUBLICATION

• Submit Your Thesis
• Published Thesis List
• Thesis Guidelines

CONFERENCE SPONSERSHIP

• Conference Papers
• Sponsorship Guidelines

PUBLICATIONS

• January 2016 Edition
• February 2016 Edition
• March 2016 Edition
• April 2016 Edition
• May 2016 Edition
• June 2016 Edition
• July 2016 Edition
• August 2016 Edition

• MyU : For Students, Faculty, and Staff

Fall 2024 CSCI Special Topics Courses

Cloud computing.

Meeting Time: 09:45 AM‑11:00 AM TTh  Instructor: Ali Anwar Course Description: Cloud computing serves many large-scale applications ranging from search engines like Google to social networking websites like Facebook to online stores like Amazon. More recently, cloud computing has emerged as an essential technology to enable emerging fields such as Artificial Intelligence (AI), the Internet of Things (IoT), and Machine Learning. The exponential growth of data availability and demands for security and speed has made the cloud computing paradigm necessary for reliable, financially economical, and scalable computation. The dynamicity and flexibility of Cloud computing have opened up many new forms of deploying applications on infrastructure that cloud service providers offer, such as renting of computation resources and serverless computing.    This course will cover the fundamentals of cloud services management and cloud software development, including but not limited to design patterns, application programming interfaces, and underlying middleware technologies. More specifically, we will cover the topics of cloud computing service models, data centers resource management, task scheduling, resource virtualization, SLAs, cloud security, software defined networks and storage, cloud storage, and programming models. We will also discuss data center design and management strategies, which enable the economic and technological benefits of cloud computing. Lastly, we will study cloud storage concepts like data distribution, durability, consistency, and redundancy. Registration Prerequisites: CS upper div, CompE upper div., EE upper div., EE grad, ITI upper div., Univ. honors student, or dept. permission; no cr for grads in CSci. Complete the following Google form to request a permission number from the instructor ( https://forms.gle/6BvbUwEkBK41tPJ17 ).

CSCI 5980/8980 

Machine learning for healthcare: concepts and applications.

Meeting Time: 11:15 AM‑12:30 PM TTh  Instructor: Yogatheesan Varatharajah Course Description: Machine Learning is transforming healthcare. This course will introduce students to a range of healthcare problems that can be tackled using machine learning, different health data modalities, relevant machine learning paradigms, and the unique challenges presented by healthcare applications. Applications we will cover include risk stratification, disease progression modeling, precision medicine, diagnosis, prognosis, subtype discovery, and improving clinical workflows. We will also cover research topics such as explainability, causality, trust, robustness, and fairness.

Registration Prerequisites: CSCI 5521 or equivalent. Complete the following Google form to request a permission number from the instructor ( https://forms.gle/z8X9pVZfCWMpQQ6o6  ).

Visualization with AI

Meeting Time: 04:00 PM‑05:15 PM TTh  Instructor: Qianwen Wang Course Description: This course aims to investigate how visualization techniques and AI technologies work together to enhance understanding, insights, or outcomes.

This is a seminar style course consisting of lectures, paper presentation, and interactive discussion of the selected papers. Students will also work on a group project where they propose a research idea, survey related studies, and present initial results.

This course will cover the application of visualization to better understand AI models and data, and the use of AI to improve visualization processes. Readings for the course cover papers from the top venues of AI, Visualization, and HCI, topics including AI explainability, reliability, and Human-AI collaboration.    This course is designed for PhD students, Masters students, and advanced undergraduates who want to dig into research.

Registration Prerequisites: Complete the following Google form to request a permission number from the instructor ( https://forms.gle/YTF5EZFUbQRJhHBYA  ). Although the class is primarily intended for PhD students, motivated juniors/seniors and MS students who are interested in this topic are welcome to apply, ensuring they detail their qualifications for the course.

Visualizations for Intelligent AR Systems

Meeting Time: 04:00 PM‑05:15 PM MW  Instructor: Zhu-Tian Chen Course Description: This course aims to explore the role of Data Visualization as a pivotal interface for enhancing human-data and human-AI interactions within Augmented Reality (AR) systems, thereby transforming a broad spectrum of activities in both professional and daily contexts. Structured as a seminar, the course consists of two main components: the theoretical and conceptual foundations delivered through lectures, paper readings, and discussions; and the hands-on experience gained through small assignments and group projects. This class is designed to be highly interactive, and AR devices will be provided to facilitate hands-on learning.    Participants will have the opportunity to experience AR systems, develop cutting-edge AR interfaces, explore AI integration, and apply human-centric design principles. The course is designed to advance students' technical skills in AR and AI, as well as their understanding of how these technologies can be leveraged to enrich human experiences across various domains. Students will be encouraged to create innovative projects with the potential for submission to research conferences.

Registration Prerequisites: Complete the following Google form to request a permission number from the instructor ( https://forms.gle/Y81FGaJivoqMQYtq5 ). Students are expected to have a solid foundation in either data visualization, computer graphics, computer vision, or HCI. Having expertise in all would be perfect! However, a robust interest and eagerness to delve into these subjects can be equally valuable, even though it means you need to learn some basic concepts independently.

Sustainable Computing: A Systems View

Meeting Time: 09:45 AM‑11:00 AM  Instructor: Abhishek Chandra Course Description: In recent years, there has been a dramatic increase in the pervasiveness, scale, and distribution of computing infrastructure: ranging from cloud, HPC systems, and data centers to edge computing and pervasive computing in the form of micro-data centers, mobile phones, sensors, and IoT devices embedded in the environment around us. The growing amount of computing, storage, and networking demand leads to increased energy usage, carbon emissions, and natural resource consumption. To reduce their environmental impact, there is a growing need to make computing systems sustainable. In this course, we will examine sustainable computing from a systems perspective. We will examine a number of questions:   • How can we design and build sustainable computing systems?   • How can we manage resources efficiently?   • What system software and algorithms can reduce computational needs?    Topics of interest would include:   • Sustainable system design and architectures   • Sustainability-aware systems software and management   • Sustainability in large-scale distributed computing (clouds, data centers, HPC)   • Sustainability in dispersed computing (edge, mobile computing, sensors/IoT)

Registration Prerequisites: This course is targeted towards students with a strong interest in computer systems (Operating Systems, Distributed Systems, Networking, Databases, etc.). Background in Operating Systems (Equivalent of CSCI 5103) and basic understanding of Computer Networking (Equivalent of CSCI 4211) is required.

• Future undergraduate students
• Future transfer students
• Future graduate students
• Future international students
• Diversity and Inclusion Opportunities
• Learn abroad
• Living Learning Communities
• Mentor programs
• Programs for women
• Student groups
• Visit, Apply & Next Steps
• Information for current students
• Departments and majors overview
• Departments
• Undergraduate majors
• Graduate programs
• Integrated Degree Programs
• Additional degree-granting programs
• Online learning
• Academic Advising overview
• Academic Advising FAQ
• Academic Advising Blog
• Appointments and drop-ins
• Academic support
• Commencement
• Four-year plans
• Honors advising
• Policies, procedures, and forms
• Career Services overview
• Resumes and cover letters
• Jobs and internships
• Interviews and job offers
• CSE Career Fair
• Major and career exploration
• Graduate school
• Collegiate Life overview
• Scholarships
• Diversity & Inclusivity Alliance
• Anderson Student Innovation Labs
• Information for alumni
• Get engaged with CSE
• Upcoming events
• CSE Alumni Society Board
• Alumni volunteer interest form
• Golden Medallion Society Reunion
• 50-Year Reunion
• Alumni honors and awards
• Outstanding Achievement
• Alumni Service
• Distinguished Leadership
• Honorary Doctorate Degrees
• Nobel Laureates
• Alumni resources
• Alumni career resources
• Alumni news outlets
• CSE branded clothing
• International alumni resources
• Inventing Tomorrow magazine
• Update your info
• CSE giving overview
• Why give to CSE?
• College priorities
• Give online now
• External relations
• Giving priorities
• Donor stories
• Impact of giving
• Ways to give to CSE
• Matching gifts
• CSE directories
• Invest in your company and the future
• Recruit our students
• Connect with researchers
• K-12 initiatives
• Diversity initiatives
• Research news
• Give to CSE
• CSE priorities
• Corporate relations
• Information for faculty and staff
• Administrative offices overview
• Office of the Dean
• Academic affairs
• Finance and Operations
• Communications
• Human resources
• Undergraduate programs and student services
• CSE Committees
• CSE policies overview
• Academic policies
• Faculty hiring and tenure policies
• Finance policies and information
• Graduate education policies
• Human resources policies
• Research policies
• Research overview
• Research centers and facilities
• Research proposal submission process
• Research safety
• Award-winning CSE faculty
• National academies
• University awards
• Honorary professorships
• Collegiate awards
• Other CSE honors and awards
• Staff awards
• Performance Management Process
• Work. With Flexibility in CSE
• K-12 outreach overview
• Summer camps
• Outreach events
• Enrichment programs
• Field trips and tours
• CSE K-12 Virtual Classroom Resources
• Educator development
• Sponsor an event

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

The Fundamental Value of Troubleshooting in Computer System Servicing NCII

Troubleshooting is the ability of individuals to adopt a systematic approach towards identifying and then solving the problem or issue. In simple words, troubleshooting are the problem solving abilities of a person. The researcher choose to study “The Fundamental Value of Troubleshooting in CSS NCII” to measure the level of importance of the students towards troubleshooting. The respondents are selected based on the needed information to collect. These are 30 students from 3 sections of Grade 12 CSS. The researchers used purposive sampling for this study. The results shows that using troubleshooting in activities related to Computer System Servicing is important. The findings also implies that troubleshooting challenge the student to be a competitive student and a golden ticket to pass the NCII exam.

Related Papers

Procedia-Social and Behavioral Sciences

Educational Technology Research and Development

Peter Fenrich

Benedict Labe

The purpose of this research was to determine the extent of student teachers’ willingness to engage in troubleshooting activities and their technological problem-solving self-appraised ability. The study used a cross-sectional descriptive correlational design to collect data from 310 purposively random sampled students from three universities in Northern Nigeria. Results of data analyses indicated that student teachers from the universities surveyed reported a moderate willingness to engage in troubleshooting activities as well as a moderately positive self-appraisal of their problem-solving ability. The student teachers’ willingness to engage in troubleshooting activities was also significantly related to the pattern of their self-appraised problem-solving ability. It was therefore concluded that the findings from this research do not support the pedestrian view that students from Nigerian universities are reluctant to engage in problem-solving activities.

Johannes Strobel

Abstract Although there is an increasingly interest for people to become technologically literate, there exists a technical knowledge gap between industry needs and workforce competencies, especially in developing countries such Colombia. That is why technological skills such as troubleshooting need to be developed. Moreover, learning technology skills may be used as a tool for learning new context-specific knowledge.

Educational Psychology Review

Assad Adjams

Mostafa Al-Emran , Hani Alchalabi

nurul syafiqah

RELATED PAPERS

Vera Condori

SSRN Electronic Journal

Pablo Gervás

aditya taqwa

Journal of Ophthalmic and Vision Research

Fatemeh Bazvand

International Journal of Automotive and Mechanical Engineering

Fauzi Ahmad

Illinois Journal of Mathematics

Procedia - Social and Behavioral Sciences

Zohreh Kiani

2021 IEEE Global Communications Conference (GLOBECOM)

Jorge Baranda

Dong-Qing Wei

BMJ global health

Viviana Martinez-Bianchi

"El Falucho: En búsqueda de una identidad perdida''

Juan Carlos Cifuentes Cáceres

Proceedings of the National Academy of Sciences

Patrick Leung

Kriterion: Revista de Filosofia

Osmir Dombrowski

Le Néolithique en Anatolie, un patrimoine archéologique aux origines de nos sociétés actuelles

Martin Godon

Composite Structures

İdris Karen

Frontiers in Microbiology

Peter Girguis

Francisco Guevara-Hernández

Materials Today: Proceedings

Hrishikesh Kulkarni

IFAC-PapersOnLine

Jack Chaplin

New Directions in Elite Studies

Johan Heilbron

Global Journal of Health Science

Emmanuel Magesa

Current Issues in Linguistic Theory

Maria Gabriela Ardisson Pereira Matos

L. Borsetto see also luciana borsetto

RELATED TOPICS

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024