research paper regarding nursing

• Research article
• Open access
• Published: 09 November 2005

A qualitative study of nursing student experiences of clinical practice

• Farkhondeh Sharif 1 &
• Sara Masoumi 2  

BMC Nursing volume  4 , Article number:  6 ( 2005 ) Cite this article

355k Accesses

166 Citations

9 Altmetric

Metrics details

Nursing student's experiences of their clinical practice provide greater insight to develop an effective clinical teaching strategy in nursing education. The main objective of this study was to investigate student nurses' experience about their clinical practice.

Focus groups were used to obtain students' opinion and experiences about their clinical practice. 90 baccalaureate nursing students at Shiraz University of Medical Sciences (Faculty of Nursing and Midwifery) were selected randomly from two hundred students and were arranged in 9 groups of ten students. To analyze the data the method used to code and categories focus group data were adapted from approaches to qualitative data analysis.

Four themes emerged from the focus group data. From the students' point of view," initial clinical anxiety", "theory-practice gap"," clinical supervision", professional role", were considered as important factors in clinical experience.

The result of this study showed that nursing students were not satisfied with the clinical component of their education. They experienced anxiety as a result of feeling incompetent and lack of professional nursing skills and knowledge to take care of various patients in the clinical setting.

Peer Review reports

Clinical experience has been always an integral part of nursing education. It prepares student nurses to be able of "doing" as well as "knowing" the clinical principles in practice. The clinical practice stimulates students to use their critical thinking skills for problem solving [ 1 ]

Awareness of the existence of stress in nursing students by nurse educators and responding to it will help to diminish student nurses experience of stress. [ 2 ]

Clinical experience is one of the most anxiety producing components of the nursing program which has been identified by nursing students. In a descriptive correlational study by Beck and Srivastava 94 second, third and fourth year nursing students reported that clinical experience was the most stressful part of the nursing program[ 3 ]. Lack of clinical experience, unfamiliar areas, difficult patients, fear of making mistakes and being evaluated by faculty members were expressed by the students as anxiety-producing situations in their initial clinical experience. In study done by Hart and Rotem stressful events for nursing students during clinical practice have been studied. They found that the initial clinical experience was the most anxiety producing part of their clinical experience [ 4 ]. The sources of stress during clinical practice have been studied by many researchers [ 5 – 10 ] and [ 11 ].

The researcher came to realize that nursing students have a great deal of anxiety when they begin their clinical practice in the second year. It is hoped that an investigation of the student's view on their clinical experience can help to develop an effective clinical teaching strategy in nursing education.

A focus group design was used to investigate the nursing student's view about the clinical practice. Focus group involves organized discussion with a selected group of individuals to gain information about their views and experiences of a topic and is particularly suited for obtaining several perspectives about the same topic. Focus groups are widely used as a data collection technique. The purpose of using focus group is to obtain information of a qualitative nature from a predetermined and limited number of people [ 12 , 13 ].

Using focus group in qualitative research concentrates on words and observations to express reality and attempts to describe people in natural situations [ 14 ].

The group interview is essentially a qualitative data gathering technique [ 13 ]. It can be used at any point in a research program and one of the common uses of it is to obtain general background information about a topic of interest [ 14 ].

Focus groups interviews are essential in the evaluation process as part of a need assessment, during a program, at the end of the program or months after the completion of a program to gather perceptions on the outcome of that program [ 15 , 16 ]. Kruegger (1988) stated focus group data can be used before, during and after programs in order to provide valuable data for decision making [ 12 ].

The participants from which the sample was drawn consisted of 90 baccalaureate nursing students from two hundred nursing students (30 students from the second year and 30 from the third and 30 from the fourth year) at Shiraz University of Medical Sciences (Faculty of Nursing and Midwifery). The second year nursing students already started their clinical experience. They were arranged in nine groups of ten students. Initially, the topics developed included 9 open-ended questions that were related to their nursing clinical experience. The topics were used to stimulate discussion.

The following topics were used to stimulate discussion regarding clinical experience in the focus groups.

How do you feel about being a student in nursing education?

How do you feel about nursing in general?

Is there any thing about the clinical field that might cause you to feel anxious about it?

Would you like to talk about those clinical experiences which you found most anxiety producing?

Which clinical experiences did you find enjoyable?

What are the best and worst things do you think can happen during the clinical experience?

What do nursing students worry about regarding clinical experiences?

How do you think clinical experiences can be improved?

What is your expectation of clinical experiences?

The first two questions were general questions which were used as ice breakers to stimulate discussion and put participants at ease encouraging them to interact in a normal manner with the facilitator.

Data analysis

The following steps were undertaken in the focus group data analysis.

Immediate debriefing after each focus group with the observer and debriefing notes were made. Debriefing notes included comments about the focus group process and the significance of data

Listening to the tape and transcribing the content of the tape

Checking the content of the tape with the observer noting and considering any non-verbal behavior. The benefit of transcription and checking the contents with the observer was in picking up the following:

Parts of words

Non-verbal communication, gestures and behavior...

The researcher facilitated the groups. The observer was a public health graduate who attended all focus groups and helped the researcher by taking notes and observing students' on non-verbal behavior during the focus group sessions. Observer was not known to students and researcher

The methods used to code and categorise focus group data were adapted from approaches to qualitative content analysis discussed by Graneheim and Lundman [ 17 ] and focus group data analysis by Stewart and Shamdasani [ 14 ] For coding the transcript it was necessary to go through the transcripts line by line and paragraph by paragraph, looking for significant statements and codes according to the topics addressed. The researcher compared the various codes based on differences and similarities and sorted into categories and finally the categories was formulated into a 4 themes.

The researcher was guided to use and three levels of coding [ 17 , 18 ]. Three levels of coding selected as appropriate for coding the data.

Level 1 coding examined the data line by line and making codes which were taken from the language of the subjects who attended the focus groups.

Level 2 coding which is a comparing of coded data with other data and the creation of categories. Categories are simply coded data that seem to cluster together and may result from condensing of level 1 code [ 17 , 19 ].

Level 3 coding which describes the Basic Social Psychological Process which is the title given to the central themes that emerge from the categories.

Table 1 shows the three level codes for one of the theme

The documents were submitted to two assessors for validation. This action provides an opportunity to determine the reliability of the coding [ 14 , 15 ]. Following a review of the codes and categories there was agreement on the classification.

Ethical considerations

The study was conducted after approval has been obtained from Shiraz university vice-chancellor for research and in addition permission to conduct the study was obtained from Dean of the Faculty of Nursing and Midwifery. All participants were informed of the objective and design of the study and a written consent received from the participants for interviews and they were free to leave focus group if they wish.

Most of the students were females (%94) and single (% 86) with age between 18–25.

The qualitative analysis led to the emergence of the four themes from the focus group data. From the students' point of view," initial clinical anxiety", "theory-practice gap", clinical supervision"," professional role", was considered as important factors in clinical experience.

Initial clinical anxiety

This theme emerged from all focus group discussion where students described the difficulties experienced at the beginning of placement. Almost all of the students had identified feeling anxious in their initial clinical placement. Worrying about giving the wrong information to the patient was one of the issues brought up by students.

One of the students said:

On the first day I was so anxious about giving the wrong information to the patient. I remember one of the patients asked me what my diagnosis is. ' I said 'I do not know', she said 'you do not know? How can you look after me if you do not know what my diagnosis is?'

From all the focus group sessions, the students stated that the first month of their training in clinical placement was anxiety producing for them.

One of the students expressed:

The most stressful situation is when we make the next step. I mean ... clinical placement and we don't have enough clinical experience to accomplish the task, and do our nursing duties .

Almost all of the fourth year students in the focus group sessions felt that their stress reduced as their training and experience progressed.

Another cause of student's anxiety in initial clinical experience was the students' concern about the possibility of harming a patient through their lack of knowledge in the second year.

One of the students reported:

In the first day of clinical placement two patients were assigned to me. One of them had IV fluid. When I introduced myself to her, I noticed her IV was running out. I was really scared and I did not know what to do and I called my instructor .

Fear of failure and making mistakes concerning nursing procedures was expressed by another student. She said:

I was so anxious when I had to change the colostomy dressing of my 24 years old patient. It took me 45 minutes to change the dressing. I went ten times to the clinic to bring the stuff. My heart rate was increasing and my hand was shaking. I was very embarrassed in front of my patient and instructor. I will never forget that day .

Sellek researched anxiety-creating incidents for nursing students. He suggested that the ward is the best place to learn but very few of the learner's needs are met in this setting. Incidents such as evaluation by others on initial clinical experience and total patient care, as well as interpersonal relations with staff, quality of care and procedures are anxiety producing [ 11 ].

Theory-practice gap

The category theory-practice gap emerged from all focus discussion where almost every student in the focus group sessions described in some way the lack of integration of theory into clinical practice.

I have learnt so many things in the class, but there is not much more chance to do them in actual settings .

Another student mentioned:

When I just learned theory for example about a disease such as diabetic mellitus and then I go on the ward and see the real patient with diabetic mellitus, I relate it back to what I learned in class and that way it will remain in my mind. It is not happen sometimes .

The literature suggests that there is a gap between theory and practice. It has been identified by Allmark and Tolly [ 20 , 21 ]. The development of practice theory, theory which is developed from practice, for practice, is one way of reducing the theory-practice gap [ 21 ]. Rolfe suggests that by reconsidering the relationship between theory and practise the gap can be closed. He suggests facilitating reflection on the realities of clinical life by nursing theorists will reduce the theory-practice gap. The theory- practice gap is felt most acutely by student nurses. They find themselves torn between the demands of their tutor and practising nurses in real clinical situations. They were faced with different real clinical situations and are unable to generalise from what they learnt in theory [ 22 ].

Clinical supervision

Clinical supervision is recognised as a developmental opportunity to develop clinical leadership. Working with the practitioners through the milieu of clinical supervision is a powerful way of enabling them to realize desirable practice [ 23 ]. Clinical nursing supervision is an ongoing systematic process that encourages and supports improved professional practice. According to Berggren and Severinsson the clinical nurse supervisors' ethical value system is involved in her/his process of decision making. [ 24 , 25 ]

Clinical Supervision by Head Nurse (Nursing Unit Manager) and Staff Nurses was another issue discussed by the students in the focus group sessions. One of the students said:

Sometimes we are taught mostly by the Head Nurse or other Nursing staff. The ward staff are not concerned about what students learn, they are busy with their duties and they are unable to have both an educational and a service role

Another student added:

Some of the nursing staff have good interaction with nursing students and they are interested in helping students in the clinical placement but they are not aware of the skills and strategies which are necessary in clinical education and are not prepared for their role to act as an instructor in the clinical placement

The students mostly mentioned their instructor's role as an evaluative person. The majority of students had the perception that their instructors have a more evaluative role than a teaching role.

The literature suggests that the clinical nurse supervisors should expressed their existence as a role model for the supervisees [ 24 ]

Professional role

One view that was frequently expressed by student nurses in the focus group sessions was that students often thought that their work was 'not really professional nursing' they were confused by what they had learned in the faculty and what in reality was expected of them in practice.

We just do basic nursing care, very basic . ... You know ... giving bed baths, keeping patients clean and making their beds. Anyone can do it. We spend four years studying nursing but we do not feel we are doing a professional job .

The role of the professional nurse and nursing auxiliaries was another issue discussed by one of the students:

The role of auxiliaries such as registered practical nurse and Nurses Aids are the same as the role of the professional nurse. We spend four years and we have learned that nursing is a professional job and it requires training and skills and knowledge, but when we see that Nurses Aids are doing the same things, it can not be considered a professional job .

The result of student's views toward clinical experience showed that they were not satisfied with the clinical component of their education. Four themes of concern for students were 'initial clinical anxiety', 'theory-practice gap', 'clinical supervision', and 'professional role'.

The nursing students clearly identified that the initial clinical experience is very stressful for them. Students in the second year experienced more anxiety compared with third and fourth year students. This was similar to the finding of Bell and Ruth who found that nursing students have a higher level of anxiety in second year [ 26 , 27 ]. Neary identified three main categories of concern for students which are the fear of doing harm to patients, the sense of not belonging to the nursing team and of not being fully competent on registration [ 28 ] which are similar to what our students mentioned in the focus group discussions. Jinks and Patmon also found that students felt they had an insufficiency in clinical skills upon completion of pre-registration program [ 29 ].

Initial clinical experience was the most anxiety producing part of student clinical experience. In this study fear of making mistake (fear of failure) and being evaluated by faculty members were expressed by the students as anxiety-producing situations in their initial clinical experience. This finding is supported by Hart and Rotem [ 4 ] and Stephens [ 30 ]. Developing confidence is an important component of clinical nursing practice [ 31 ]. Development of confidence should be facilitated by the process of nursing education; as a result students become competent and confident. Differences between actual and expected behaviour in the clinical placement creates conflicts in nursing students. Nursing students receive instructions which are different to what they have been taught in the classroom. Students feel anxious and this anxiety has effect on their performance [ 32 ]. The existence of theory-practice gap in nursing has been an issue of concern for many years as it has been shown to delay student learning. All the students in this study clearly demonstrated that there is a gap between theory and practice. This finding is supported by other studies such as Ferguson and Jinks [ 33 ] and Hewison and Wildman [ 34 ] and Bjork [ 35 ]. Discrepancy between theory and practice has long been a source of concern to teachers, practitioners and learners. It deeply rooted in the history of nurse education. Theory-practice gap has been recognised for over 50 years in nursing. This issue is said to have caused the movement of nurse education into higher education sector [ 34 ].

Clinical supervision was one of the main themes in this study. According to participant, instructor role in assisting student nurses to reach professional excellence is very important. In this study, the majority of students had the perception that their instructors have a more evaluative role than a teaching role. About half of the students mentioned that some of the head Nurse (Nursing Unit Manager) and Staff Nurses are very good in supervising us in the clinical area. The clinical instructor or mentors can play an important role in student nurses' self-confidence, promote role socialization, and encourage independence which leads to clinical competency [ 36 ]. A supportive and socialising role was identified by the students as the mentor's function. This finding is similar to the finding of Earnshaw [ 37 ]. According to Begat and Severinsson supporting nurses by clinical nurse specialist reported that they may have a positive effect on their perceptions of well-being and less anxiety and physical symptoms [ 25 ].

The students identified factors that influence their professional socialisation. Professional role and hierarchy of occupation were factors which were frequently expressed by the students. Self-evaluation of professional knowledge, values and skills contribute to the professional's self-concept [ 38 ]. The professional role encompasses skills, knowledge and behaviour learned through professional socialisation [ 39 ]. The acquisition of career attitudes, values and motives which are held by society are important stages in the socialisation process [ 40 ]. According to Corwin autonomy, independence, decision-making and innovation are achieved through professional self-concept 41 . Lengacher (1994) discussed the importance of faculty staff in the socialisation process of students and in preparing them for reality in practice. Maintenance and/or nurturance of the student's self-esteem play an important role for facilitation of socialisation process 42 .

One view that was expressed by second and third year student nurses in the focus group sessions was that students often thought that their work was 'not really professional nursing' they were confused by what they had learned in the faculty and what in reality was expected of them in practice.

The finding of this study and the literature support the need to rethink about the clinical skills training in nursing education. It is clear that all themes mentioned by the students play an important role in student learning and nursing education in general. There were some similarities between the results of this study with other reported studies and confirmed that some of the factors are universal in nursing education. Nursing students expressed their views and mentioned their worry about the initial clinical anxiety, theory-practice gap, professional role and clinical supervision. They mentioned that integration of both theory and practice with good clinical supervision enabling them to feel that they are enough competent to take care of the patients. The result of this study would help us as educators to design strategies for more effective clinical teaching. The results of this study should be considered by nursing education and nursing practice professionals. Faculties of nursing need to be concerned about solving student problems in education and clinical practice. The findings support the need for Faculty of Nursing to plan nursing curriculum in a way that nursing students be involved actively in their education.

Dunn SV, Burnett P: The development of a clinical learning environment scale. Journal of Advanced Nursing. 1995, 22: 1166-1173.

Article   CAS   PubMed   Google Scholar  

Lindop E: Factors associated with student and pupil nurse wastage. Journal of Advanced Nursing. 1987, 12 (6): 751-756.

Beck D, Srivastava R: Perceived level and source of stress in baccalaureate nursing students. Journal of Nursing Education. 1991, 30 (3): 127-132.

CAS   PubMed   Google Scholar  

Hart G, Rotem A: The best and the worst: Students' experience of clinical education. The Australian Journal of Advanced Nursing. 1994, 11 (3): 26-33.

Sheila Sh, Huey-Shyon L, Shiowli H: Perceived stress and physio-psycho-social status of nursing students during their initial period of clinical practice. International Journal of Nursing Studies. 2002, 39: 165-175. 10.1016/S0020-7489(01)00016-5.

Article   Google Scholar  

Johnson J: Reducing distress in first level and student nurses. Journal of Advanced Nursing. 2000, 32 (1): 66-74. 10.1046/j.1365-2648.2000.01421.x.

Admi H: Nursing students' stress during the initial clinical experience. Journal of Nursing Education. 1997, 36: 323-327.

Blainey GC: Anxiety in the undergraduate medical-surgical clinical student. Journal of Nursing Education. 1980, 19 (8): 33-36.

Wong J, Wong S: Towards effective clinical teaching in nursing. Journal of Advanced Nursing. 1987, 12 (4): 505-513.

Windsor A: Nursing students' perceptions of clinical experience. Journal of Nursing Education. 1987, 26 (4): 150-154.

Sellek T: Satisfying and anxiety creating incidents for nursing students. Nursing Times. 1982, 78 (35): 137-140.

PubMed   Google Scholar  

Krueger RA: Focus Groups: A Practical Guide for Applied Research. Sage Publications: California. 1988

Google Scholar  

Denzin NK: The Research Act. 1989, Prentice Hall: Englewood Cliffs, New Jersey, 3

Stewart DW, Shamdasani PN: Analysing focus group data. Focus Groups: Theory and Practice. Edited by: Shamdasani PN. 1990, Sage Publications: Newbury Park

Barbour RS, Kitzinger J: Developing focus group research : politics, theory and practice. Sage. 1999

Patton MQ: Qualitative Evaluation and Research Methods. 1990, Sage publications, 2

Graneheim UH, Lundman B: Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Education Today. 2004, 24: 105-112. 10.1016/j.nedt.2003.10.001.

Streubert HJ, Carpenter DR: Qualitative Research in Nursing. Advancing the Humanistic Imperative. 1995, J.B. Lippincott Company: Philadelphia

Polit DF, Hungler BP: Nursing research: Principles and Methods. Philadelphia newyork. 1999

Allmark PA: classical view of the theory-practice gap in nursing. Journal of Advanced Nursing. 1995, 22 (1): 18-23. 10.1046/j.1365-2648.1995.22010018.x.

Tolley KA: Theory from practice for practice: Is this a reality?. Journal of Advanced Nursing. 1995, 21 (1): 184-190. 10.1046/j.1365-2648.1995.21010184.x.

Rolfe G: Listening to students: Course evaluation as action research. Nurse Education Today. 1994, 14 (3): 223-227. 10.1016/0260-6917(94)90085-X.

Johns Ch: clinical supervision as a model for clinical leadership. Journal of Nursing Management. 2003, 11: 25-34. 10.1046/j.1365-2834.2002.00288.x.

Article   PubMed   Google Scholar  

Berggren I, Severinsson E: Nurses supervisors'action in relation to their decision-making style and ethical approach to clinical supervision. Journal of Advanced Nursing. 2003, 41 (6): 615-622. 10.1046/j.1365-2648.2003.02573.x.

Begat I, Severinsson E: Nurses' satisfaction with their work environment and the outcomes of clinical nursing supervision on nurses' experiences of well-being. Journal of Nursing Management. 2005, 13: 221-230. 10.1111/j.1365-2834.2004.00527.x.

Bell P: Anxiety in mature age and higher school certificate entry student nurses – A comparison of effects on performance. Journal of Australian Congress of Mental Health Nurses. 1984, 4/5: 13-21.

Ruth L: Experiencing before and throughout the nursing career. Journal of Advanced Nursing. 2002, 39: 119-10.1046/j.1365-2648.2000.02251.x.

Neary M: Project 2000 students' survival kit: a return to the practical room. Nurse Education Today. 1997, 17 (1): 46-52. 10.1016/S0260-6917(97)80078-0.

Jinks A, Pateman B: Nither this nor that: The stigma of being an undergraduate nurse. Nursing Times. 1998, 2 (2): 12-13.

CAS   Google Scholar  

Stephen RL: Imagery: A treatment for nursing student anxiety. Journal of Nursing Education. 1992, 31 (7): 314-319.

Grundy SE: The confidence scale. Nurse Educator. 1993, 18 (1): 6-9.

Copeland L: Developing student confidence. Nurse Educator. 1990, 15 (1): 7-

Ferguson K, Jinks A: Integrating what is taught with what is practised in the nursing curriculum: A multi-dimensional model. Journal of Advanced Nursing. 1994, 20 (4): 687-695. 10.1046/j.1365-2648.1994.20040687.x.

Hewison A, Wildman S: The theory-practice gap in nursing: A new dimension. Journal of Advanced Nursing. 1996, 24 (4): 754-761. 10.1046/j.1365-2648.1996.25214.x.

Bjork T: Neglected conflicts in the discipline of nursing: Perceptions of the importance and value of practical skill. Journal of Advanced Nursing. 1995, 22 (1): 6-12. 10.1046/j.1365-2648.1995.22010006.x.

Busen N: Mentoring in advanced practice nursing. Journal of Advanced Nursing Practice. 1999, 2: 2-

Earnshaw GP: Mentorship: The students' view. Nurse Education Today. 1995, 15 (4): 274-279. 10.1016/S0260-6917(95)80130-8.

Kelly B: The professional self-concepts of nursing undergraduates and their perceptions of influential forces. Journal of Nursing Education. 1992, 31 (3): 121-125.

Lynn MR, McCain NL, Boss BJ: Socialization of R.N. to B.S.N Image:. Journal of Nursing Scholarship. 1989, 21 (4): 232-237.

Article   CAS   Google Scholar  

Klein SM, Ritti RR: Understanding Organisational Behaviour. 1980, Kent: Boston

Corwin RG: The professional employee: A study of conflict in nursing roles. The American Journal of Sociology. 1961, 66: 604-615. 10.1086/223010.

Lengacher CA: Effects of professional development seminars on role conception, role deprivation, and self-esteem of generic baccalaureate students. Nursing Connections. 1994, 7 (1): 21-34.

Pre-publication history

The pre-publication history for this paper can be accessed here: http://www.biomedcentral.com/1472-6955/4/6/prepub

Download references

Acknowledgements

The author would like to thank the student nurses who participated in this study for their valuable contribution

Author information

Authors and affiliations.

Psychiatric Nursing Department, Fatemeh (P.B.U.H) College of Nursing and Midwifery Shiraz University of Medical Sciences, Zand BlvD, Shiraz, Iran

Farkhondeh Sharif

English Department, Shiraz University, Shiraz, Iran

Sara Masoumi

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Farkhondeh Sharif .

Additional information

Competing interests.

The author(s) declare that they no competing interests.

Authors' contributions

FSH: Initiation and design of the research, focus groups conduction, data collection, analysis and writing the paper, SM: Editorial revision of paper

Rights and permissions

Open Access This article is published under license to BioMed Central Ltd. This is an Open Access article is distributed under the terms of the Creative Commons Attribution License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Sharif, F., Masoumi, S. A qualitative study of nursing student experiences of clinical practice. BMC Nurs 4 , 6 (2005). https://doi.org/10.1186/1472-6955-4-6

Download citation

Received : 10 June 2005

Accepted : 09 November 2005

Published : 09 November 2005

DOI : https://doi.org/10.1186/1472-6955-4-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Focus Group
• Nursing Student
• Professional Role
• Nursing Education
• Focus Group Session

BMC Nursing

ISSN: 1472-6955

• General enquiries: [email protected]

Why Nursing Research Matters

Affiliation.

• 1 Author Affiliation: Director, Magnet Recognition Program®, American Nurses Credentialing Center, Silver Spring, Maryland.
• PMID: 33882548
• DOI: 10.1097/NNA.0000000000001005

Increasingly, nursing research is considered essential to the achievement of high-quality patient care and outcomes. In this month's Magnet® Perspectives column, we examine the origins of nursing research, its role in creating the Magnet Recognition Program®, and why a culture of clinical inquiry matters for nurses. This column explores how Magnet hospitals have built upon the foundation of seminal research to advance contemporary standards that address some of the challenges faced by healthcare organizations around the world. We offer strategies for nursing leaders to develop robust research-oriented programs in their organizations.

Copyright © 2021 Wolters Kluwer Health, Inc. All rights reserved.

• Credentialing / standards
• Hospitals / standards*
• Leadership*
• Nursing Research / standards*
• Nursing Service, Hospital / standards*
• Nursing Staff, Hospital / standards*
• Organizational Culture
• Quality of Health Care / standards
• United States

• USF Research
• USF Libraries

Digital Commons @ USF > College of Nursing > Theses and Dissertations

Nursing Theses and Dissertations

Theses/dissertations from 2023 2023.

Psychological Distress, Resilience, and the Impact on Quality of Life in Breast Cancer Survivors With Taxane-Induced Peripheral Neuropathy , Lauren Schwab

Theses/Dissertations from 2022 2022

Medication-Assisted Treatment Versus 12-Step Group Therapy: A Comparative Analysis of Adherence and Abstinence In Patients With Opioid Use Disorder , Derrick C. Glymph

Theses/Dissertations from 2021 2021

Quality of Life of Older Adults with Complicated Grief Receiving Accelerated Resolution Therapy: A Mixed Methods Study , Tina M. Mason

Theses/Dissertations from 2020 2020

In Post-Extubated Patients What are the Preferred Methods of Communication During Their Experience of Endotracheal Intubation with Mechanical Ventilation , Lanette Dumas

The Effect of Hope on the Relationship between Personal and Disease Characteristics and Anxiety and Depression in Adolescents and Young Adults with Cancer , Sharon B. McNeil

Predictors of Nonadherence to Radiation Therapy Schedules Among Head and Neck Cancer Patients , Jennifer Lynn Miller

Theses/Dissertations from 2019 2019

Perceived Discrimination and Cardiovascular Outcomes in Blacks: A Secondary Data Analysis of the Heart SCORE Study , Marilyn Aluoch

Exploration of Gratitude in Cardiovascular Health: Mediators, Medication Adherence and Psychometrics , Lakeshia A. Cousin

Theses/Dissertations from 2018 2018

Fatigue-related Symptom Clusters and their Relationship with Depression, and Functional Status in Older Adults Hospice Patients with Cancer. , Suzan Fouad Abduljawad

Genetic Moderation of Pain and Fatigue Symptoms Resulting from the Mindfulness-Based Stress Reduction for Breast Cancer Program , Carissa Bea Alinat

The Moderating Effect of Religion on Death Distress and Quality of Life between Christian Cancer patients in the United States with Muslim cancer patients in Saudi Arabia , Doaa Almostadi

Prevention of Post Intensive Care Syndrome-Family with Sensation Awareness Focused Training Intervention: A Randomized Controlled Trial Pilot Study , Paula L. Cairns

Assessing Abstinence in Infants Greater Than 28 Days Old , Genieveve J. Cline

The Relationship Between Sleep Quality and Motor Function in Hospitalized Older Adult Survivors of Critical Illness , Maya N. Elías

The Role of Migration-Related Stress in Depression Among Haitian Immigrants in Florida: A Mixed Method Sequential Explanatory Approach , Dany Amanda C. Fanfan

The Effect of Depression, Inflammation and Sleep Quality on Risk for Cardiovascular Disease , Catherine L. O'Neil

Adapting SafeMedicate (Medication Dosage Calculation Skills software) For Use In Brazil , Samia Valeria Ozorio Dutra

Theses/Dissertations from 2017 2017

The Relationship Between Total Neuropathy Score-reduced, Neuropathy Symptoms and Function. , Ashraf Abulhaija

Validation of the Electronic Kids Dietary Index (E-KINDEX) Screening Tool for Early Identification of Risk for Overweight/Obesity (OW/OB) in a Pediatric Population: Associations with Quality of Life Perceptions , Patricia A. Hall

Theses/Dissertations from 2016 2016

The Effectiveness of an Intervention Designed to Improve Chlorhexidine (CHG) Bathing Technique in Adults Hospitalized in Medical Surgical Units , Janette Echemendia Denny

Levels of Distress Among Women Veterans Attending a Women’s Health Specialty Clinic in the VA Healthcare System , Debbie T. Devine

Examination of the Use of Accelerated Resolution Therapy (ART) in the Treatment of Symptoms of PTSD and Sleep Dysfunction in Veterans and Civilians , Marian Jevone Hardwick

Investigating the Mutual Effects of Depression and Spiritual Well-being on Quality of Life in Hospice Patients with Cancer and Family Caregivers Using the Actor-Partner Interdependence Model , Li-Ting Huang

The Change in Nutritional Status in Traumatic Brain Injury Patients: A Retrospective Descriptive A Retrospective Descriptive Study , Dina A. Masha'al

Exploring the Relationship Between Severity of Illness and Human Milk Volume in Very Low Birth Weight and Extremely Low Birth Weight Infants Over Six Weeks , Shannon Leigh Morse

Cardiovascular Disease Risk Scores and Novel Risk Factors in Relation to Race and Gender , Johanna Wilson

Theses/Dissertations from 2015 2015

A Comparative Evaluation of the Learner Centered Grading Debriefing Method in Nursing Education , Marisa J. Belote

Sleep, Depressive Symptoms and Cognition in Older Adults and Caregivers of Persons with Dementia , Glenna Shemida Brewster

The Relationship between Hearing Status and Cognitive Performance and the Influence of Depressive Symptoms in the Older Adult , Julie A. Daugherty

Basal Salivary Oxytocin and Skin to Skin Contact among Lactating Mothers of Premature Infants , Jessica Marie Gordon

The Relationship Between Nurses' Emotional Intelligence and Patient Outcomes , Mary Kutash

Sexual Functioning and Body Image in Younger Breast Cancer Survivors , Carly Lynn Paterson

Cognitive Load of Registered Nurses During Medication Administration , Sarah Faith Perron

A Comparison of Quality of Life between Intense and Non-Intense Treatment for Patients with Acute Myeloid Leukemia and High-Risk Myelodysplastic Syndrome , Sara Marie Tinsley

Theses/Dissertations from 2014 2014

Acculturation, Self-Efficacy and Breastfeeding Behavior in a Sample of Hispanic Women , Ivonne F. Hernandez

Knowledge and Acceptance of HPV and the HPV Vaccine in Young Men and Their Intention to be Vaccinated , Brenda Renee Jasper

The Relationships Between Sleep Disturbances, Depression, Inflammatory Markers, and Sexual Trauma in Female Veterans , Ellen Marcolongo

Examination of Possible Protective Effect of Rhesus D Positive Blood Factor on Toxoplasma-related Depressive Symptoms in Pregnancy , Lisa Lynn Parnell

Knowledge, Attitudes, and Practice of Primary Care Nurse Practitioners Regarding Skin Cancer Assessmnets: Validity and Reliability of a New Instrument , Debra Michelle Shelby

Theses/Dissertations from 2013 2013

Knowledge and Practice of Reproductive Health among Mothers and their Impact on Fetal Birth Outcomes: A Case of Eritrea , Winta Negusse Araya

Race/Ethnicity, Subjective and Objective Sleep Quality, Physical and Psychological Symptoms in Breast Cancer Survivors , Pinky H. Budhrani

Factors Predicting Pap Smear Adherence in HIV-infected Women: Using the Health Belief Model , Crystal L. Chapman Lambert

The Relationship Between Socioeconomic Status and Body Mass Index on Vitamin D Levels in African American Women with and without Diabetes Living in Areas with Abundant Sunshine , Shani Vann Davis

Predictors of Quality of Life in Patients with Cutaneous T cell Lymphoma , Darcie Marie Deaver

Relationship between dysphoric moods, risk-taking behaviors, and Toxoplasma gondii antibody titers in female veterans , Allyson Radford Duffy

Prenatal Stress, Depression, and Herpes Viral Titers , Pao-Chu Hsu

Factors Associated with Fear of Breast Cancer Recurrence Among Survivors , Jean Marie Lucas

Sickle Cell Disease: The Role of Self-Care Management , Nadine Matthie

Factors Influencing Vaccination Decisions in African American Mothers of Preschool Age Children , Chauntel Mckenzie Mcnair

The Strong Black Woman, Depression, and Emotional Eating , Michelle Renee Offutt

Development of an Investigator-designed Questionnaire Concerning Childbirth Delivery Options based on the Theory of Planned Behavior , Chun-Yi Tai

Theses/Dissertations from 2012 2012

The Mediating Effect of Distress Caused by Constipation on Predictors of Quality of Life of Hospice Patients with Cancer. , Abdel Alkhalouf

Testing a Model of Bacterial Vaginosis among Black Women , Jessica Brumley

The Effect of Tight Glycemic Control on Surgical Site Infection Rates in Patients Undergoing Open Heart Surgery , Sierra Gower

Development of a Tool for Pressure Ulcer Risk Assessment and Preventive Interventions in Ancillary Services Patients , Monica Shutts Messer

Hospice Nurses- Attitudes and Knowledge about Pain Management , Amie Jacqueline Miller

Theses/Dissertations from 2011 2011

Literacy and Hazard Communication Comprehension of Employees Presenting to an Occupational Health Clinic , Christine Bouchard

A Meta-Analysis of Cultural Competence Education in Professional Nurses and Nursing Students , Ruth Wilmer Gallagher

Relationship Between Cancer-Related Fatigue and Depression: A Pilot Study , Gloria Michelle Guess

A Comparison of Oncology and Non-Oncology Nurses in Their Knowledge of Cancer Pain Management , Nicole Houle

Evaluating Knowledge and Attitudes of Graduate Nursing Students Regarding Pain , Eric Bartholomew Jackson

Bone Marrow Transplant Nurses' Attitudes about Caring for Patients Who are Near the End of Life: A Quality Improvement Project , Leslie Lauersdorf

Translation and Adaptation of the Center for Epidemiologic Studies-Depression (CES-D) Scale Into Tigrigna Language for Tigrigna Speaking Eritrean Immigrants in the United States , Mulubrhan Fisseha Mogos

Nurse Manager Emotional Intelligence as a Predictor to Registered Nurse Job Satisfaction and RN Perceptions of the Practice Environment and the Relationship to Patient, Nursing and Hospital Outcomes , Jacqueline Cecilia Munro

The Relationship of Mid-Pregnancy Levels of Cytokines, Stress, and Depression with Gestational Age at Delivery , Melissa Molinari Shelton

Prophylactic, Risk-Reducing Surgery in Unaffected BRCA-Positive Women: Quality Of Life, Sexual Functioning and Psychological Well-Being , Sharon Tollin

Theses/Dissertations from 2010 2010

The Relationship Between FAM5C SNP (rs10920501) Variability, Metabolic Syndrome, and Inflammation, in Women with Coronary Heart Disease , Jennifer L. Cline

Women’s Perceptions of Postpartum Stress: A Narrative Analysis , Nancy Gilbert Crist

Lived Experience: Near-Fatal Adolescent Suicide Attempt , Phyllis Ann Dougherty

Exploring the Relationships among Work-Related Stress, Quality of Life, Job Satisfaction, and Anticipated Turnover on Nursing Units with Clinical Nurse Leaders , Mary Kohler

A Comparative Study of Knowledge of Pain Management in Certified and Non-Certified Oncology Nurses , Sherrie A. LaLande

Evaluating Knowledge and Attitudes of Undergraduate Nursing Students Regarding Pain Management , Jessica Latchman

Evaluation of Oncology Nurses' Knowledge, Practice Behaviors, and Confidence Specific to Chemotherapy Induced Peripheral Neuropathy , Rebecca Denise McAllister

Moderating the Effectiveness of Messages to Promote Physical Activity in Type 2 Diabetes , Rachel E. Myers

Factors Affecting the Process of Clinical Decision-Making in Pediatric Pain Management by Emergency Department Nurses , Teresa A. Russo

The Correlation Between Neuropathy Limitations and Depression in Chemotherapy Patients , Melissa Thebeau

Theses/Dissertations from 2009 2009

Fatigue Symptom Distress and Its Relationship with Quality Of Life in Adult Stem Cell Transplant Survivors , Suzan Fouad Abduljawad R.N., B.S.N.

Nursing Advocacy and the Accuracy of Intravenous to Oral Opioid Conversion at Discharge in the Cancer Patient , Maria L. Gallo R.N., O.C.N.

Transitional Care for Adolescents with HIV: Characteristics and Current Practices of the Adolescent Trials Network Systems of Care , Patricia Gilliam

The Effect of Ethical Ideology and Professional Values on Registered Nurses’ Intentions to Act Accountably , Susan R. Hartranft

Falls in Bone Marrow Transplant Patients: A Retrospective Study , Lura Henderson R.N., B.S.N.

Predictors of cancer caregiver depression symptomatology , Henry R. Rivera

Psychosocial outcomes of weight stigma among college students , Sabrina Joann Robinson

The Experience of Fatigue and Quality of Life in Patients with Advanced Lung Cancer , Andrea Shaffer

The Relationship Between Uncertainty in Illness and Anxiety in Patients With Cancer , Naima Vera

Shifting Paradigms: The Development of Nursing Identity in Foreign-Educated Physicians Retrained as Nurses Practicing in the United States , Liwliwa Reyes Villagomeza

Theses/Dissertations from 2008 2008

Prostate Cancer Screening Intention Among African American Men: An Instrument Development Study , Susan Anita Baker

The Geriatric Cancer Experience in End of Life: Model Adaptation and Testing , Harleah G. Buck

Communication Systems and HIV/AIDS Sexual Decision Making in Older Adolescent and Young Adult Females , Rasheeta D. Chandler MS, ARNP, FNP-BC

Relationship of Anger Trait and Anger Expression to C-Reactive Protein in Post-Menopausal Women , Rosalyn Gross

Identifying Patients with Cancer at Risk of Experiencing a Fall While Hospitalized , Joann M. Heaton

Modulation of Monocyte-Derived Dendritic Cell Maturation and Function by Cigarette Smoke Condensate in a Bronchial Epithelial Cell Co-Culture Model , Alison J. Montpetit

Cancer Patients with Pain: Examination of the Role of the Spouse/Partner Relationship In Mediating Quality of Life Outcomes for the Couple , Mary Ann Morgan

Development of an Ecological Model to Predict Risk for Acquisition of Clostridium difficile -Associated Diarrhea During Acute Care Hospitalization , Susan Elaine Steele

Development and Psychometric Evaluation of the Chemotherapy Induced Peripheral Neuropathy Assessment Tool , Cindy S. Tofthagen

Health Decision Behaviors: Appropriateness of Dietary Choice , Daryle Hermelin Wane

Theses/Dissertations from 2007 2007

The Relationship Between Sleep-Wake Disturbance and Pain in Cancer Patients Admitted to Hospice Home Care , Marjorie Acierno

Wheelchair Positioning and Pulmonary Function in Children with Cerebral Palsy , Lee Barks

Structural Equation Model of Exercise in Women Utilizing the Theory of Unpleasant Symptoms and Social Cognitive Variables , Sarah Elizabeth Cobb

Advanced Search

• Email Notifications and RSS
• All Collections
• USF Faculty Publications
• Open Access Journals
• Conferences and Events
• Theses and Dissertations
• Textbooks Collection

Useful Links

• College of Nursing Homepage
• Rights Information
• SelectedWorks
• Submit Research

Home | About | Help | My Account | Accessibility Statement | Language and Diversity Statements

Privacy Copyright

Best Nursing Research Topics for Students

What is a nursing research paper.

• What They Include
• Choosing a Topic
• Best Nursing Research Topics
• Research Paper Writing Tips

Writing a research paper is a massive task that involves careful organization, critical analysis, and a lot of time. Some nursing students are natural writers, while others struggle to select a nursing research topic, let alone write about it.

If you're a nursing student who dreads writing research papers, this article may help ease your anxiety. We'll cover everything you need to know about writing nursing school research papers and the top topics for nursing research.  

Continue reading to make your paper-writing jitters a thing of the past.

A nursing research paper is a work of academic writing composed by a nurse or nursing student. The paper may present information on a specific topic or answer a question.

During LPN/LVN and RN programs, most papers you write focus on learning to use research databases, evaluate appropriate resources, and format your writing with APA style. You'll then synthesize your research information to answer a question or analyze a topic.

BSN , MSN , Ph.D., and DNP programs also write nursing research papers. Students in these programs may also participate in conducting original research studies.

Writing papers during your academic program improves and develops many skills, including the ability to:

• Select nursing topics for research
• Conduct effective research
• Analyze published academic literature
• Format and cite sources
• Synthesize data
• Organize and articulate findings

About Nursing Research Papers

When do nursing students write research papers.

You may need to write a research paper for any of the nursing courses you take. Research papers help develop critical thinking and communication skills. They allow you to learn how to conduct research and critically review publications.

That said, not every class will require in-depth, 10-20-page papers. The more advanced your degree path, the more you can expect to write and conduct research. If you're in an associate or bachelor's program, you'll probably write a few papers each semester or term.

Do Nursing Students Conduct Original Research?

Most of the time, you won't be designing, conducting, and evaluating new research. Instead, your projects will focus on learning the research process and the scientific method. You'll achieve these objectives by evaluating existing nursing literature and sources and defending a thesis.

However, many nursing faculty members do conduct original research. So, you may get opportunities to participate in, and publish, research articles.

Example Research Project Scenario:

In your maternal child nursing class, the professor assigns the class a research paper regarding developmentally appropriate nursing interventions for the pediatric population. While that may sound specific, you have almost endless opportunities to narrow down the focus of your writing. 

You could choose pain intervention measures in toddlers. Conversely, you can research the effects of prolonged hospitalization on adolescents' social-emotional development.

What Does a Nursing Research Paper Include?

Your professor should provide a thorough guideline of the scope of the paper. In general, an undergraduate nursing research paper will consist of:

Introduction : A brief overview of the research question/thesis statement your paper will discuss. You can include why the topic is relevant.

Body : This section presents your research findings and allows you to synthesize the information and data you collected. You'll have a chance to articulate your evaluation and answer your research question. The length of this section depends on your assignment.

Conclusion : A brief review of the information and analysis you presented throughout the body of the paper. This section is a recap of your paper and another chance to reassert your thesis.

The best advice is to follow your instructor's rubric and guidelines. Remember to ask for help whenever needed, and avoid overcomplicating the assignment!

How to Choose a Nursing Research Topic

The sheer volume of prospective nursing research topics can become overwhelming for students. Additionally, you may get the misconception that all the 'good' research ideas are exhausted. However, a personal approach may help you narrow down a research topic and find a unique angle.

Writing your research paper about a topic you value or connect with makes the task easier. Additionally, you should consider the material's breadth. Topics with plenty of existing literature will make developing a research question and thesis smoother.

Finally, feel free to shift gears if necessary, especially if you're still early in the research process. If you start down one path and have trouble finding published information, ask your professor if you can choose another topic.

The Best Research Topics for Nursing Students

You have endless subject choices for nursing research papers. This non-exhaustive list just scratches the surface of some of the best nursing research topics.

1. Clinical Nursing Research Topics

• Analyze the use of telehealth/virtual nursing to reduce inpatient nurse duties.
• Discuss the impact of evidence-based respiratory interventions on patient outcomes in critical care settings.
• Explore the effectiveness of pain management protocols in pediatric patients.

2. Community Health Nursing Research Topics

• Assess the impact of nurse-led diabetes education in Type II Diabetics.
• Analyze the relationship between socioeconomic status and access to healthcare services.

3. Nurse Education Research Topics

• Review the effectiveness of simulation-based learning to improve nursing students' clinical skills.
• Identify methods that best prepare pre-licensure students for clinical practice.
• Investigate factors that influence nurses to pursue advanced degrees.
• Evaluate education methods that enhance cultural competence among nurses.
• Describe the role of mindfulness interventions in reducing stress and burnout among nurses.

4. Mental Health Nursing Research Topics

• Explore patient outcomes related to nurse staffing levels in acute behavioral health settings.
• Assess the effectiveness of mental health education among emergency room nurses .
• Explore de-escalation techniques that result in improved patient outcomes.
• Review the effectiveness of therapeutic communication in improving patient outcomes.

5. Pediatric Nursing Research Topics

• Assess the impact of parental involvement in pediatric asthma treatment adherence.
• Explore challenges related to chronic illness management in pediatric patients.
• Review the role of play therapy and other therapeutic interventions that alleviate anxiety among hospitalized children.

6. The Nursing Profession Research Topics

• Analyze the effects of short staffing on nurse burnout .
• Evaluate factors that facilitate resiliency among nursing professionals.
• Examine predictors of nurse dissatisfaction and burnout.
• Posit how nursing theories influence modern nursing practice.

Tips for Writing a Nursing Research Paper

The best nursing research advice we can provide is to follow your professor's rubric and instructions. However, here are a few study tips for nursing students to make paper writing less painful:

Avoid procrastination: Everyone says it, but few follow this advice. You can significantly lower your stress levels if you avoid procrastinating and start working on your project immediately.

Plan Ahead: Break down the writing process into smaller sections, especially if it seems overwhelming. Give yourself time for each step in the process.

Research: Use your resources and ask for help from the librarian or instructor. The rest should come together quickly once you find high-quality studies to analyze.

Outline: Create an outline to help you organize your thoughts. Then, you can plug in information throughout the research process. 

Clear Language: Use plain language as much as possible to get your point across. Jargon is inevitable when writing academic nursing papers, but keep it to a minimum.

Cite Properly: Accurately cite all sources using the appropriate citation style. Nursing research papers will almost always implement APA style. Check out the resources below for some excellent reference management options.

Revise and Edit: Once you finish your first draft, put it away for one to two hours or, preferably, a whole day. Once you've placed some space between you and your paper, read through and edit for clarity, coherence, and grammatical errors. Reading your essay out loud is an excellent way to check for the 'flow' of the paper.

Helpful Nursing Research Writing Resources:

Purdue OWL (Online writing lab) has a robust APA guide covering everything you need about APA style and rules.

Grammarly helps you edit grammar, spelling, and punctuation. Upgrading to a paid plan will get you plagiarism detection, formatting, and engagement suggestions. This tool is excellent to help you simplify complicated sentences.

Mendeley is a free reference management software. It stores, organizes, and cites references. It has a Microsoft plug-in that inserts and correctly formats APA citations.

Don't let nursing research papers scare you away from starting nursing school or furthering your education. Their purpose is to develop skills you'll need to be an effective nurse: critical thinking, communication, and the ability to review published information critically.

Choose a great topic and follow your teacher's instructions; you'll finish that paper in no time.

Joleen Sams is a certified Family Nurse Practitioner based in the Kansas City metro area. During her 10-year RN career, Joleen worked in NICU, inpatient pediatrics, and regulatory compliance. Since graduating with her MSN-FNP in 2019, she has worked in urgent care and nursing administration. Connect with Joleen on LinkedIn or see more of her writing on her website.

Plus, get exclusive access to discounts for nurses, stay informed on the latest nurse news, and learn how to take the next steps in your career.

By clicking “Join Now”, you agree to receive email newsletters and special offers from Nurse.org. We will not sell or distribute your email address to any third party, and you may unsubscribe at any time by using the unsubscribe link, found at the bottom of every email.

This paper is in the following e-collection/theme issue:

Published on 29.3.2024 in Vol 26 (2024)

Telehealth Care Through Internet Hospitals in China: Qualitative Interview Study of Physicians’ Views on Access, Expectations, and Communication

Authors of this article:

Original Paper

• Yuqiong Zhong 1, 2 , Mphil   ; 
• Jessica Hahne 3 , MA, MPH   ; 
• Xiaomin Wang 4, 5 , PhD   ; 
• Xuxi Wang 1 , Bphil   ; 
• Ying Wu 1 , MPhil   ; 
• Xin Zhang 2, 6 * , MD, PhD   ; 
• Xing Liu 6, 7 * , PhD  

1 School of Humanities, Central South University, Changsha, China

2 Xiangya Hospital, Central South University, Changsha, China

3 Department of Psychological & Brain Sciences, Washington University in St Louis, St Louis, MO, United States

4 Center for Clinical Pharmacology, The Third Xiangya Hospital of Central South University, Changsha, China

5 Center for Medical Ethics, Central South University, Changsha, China

6 Medical Humanities Research Center, Central South University, Changsha, China

7 Office of International Cooperation and Exchanges, Xiangya Hospital, Central South University, Changsha, China

*these authors contributed equally

Corresponding Author:

Xing Liu, PhD

Office of International Cooperation and Exchanges

Xiangya Hospital, Central South University

No 87 Xiangya Road, Kaifu District

Changsha, 410008

Phone: 86 18229765509

Email: [email protected]

Background: Internet hospitals in China are an emerging medical service model similar to other telehealth models used worldwide. Internet hospitals are currently in a stage of rapid development, giving rise to a series of new opportunities and challenges for patient care. Little research has examined the views of chronic disease physicians regarding internet hospitals in China.

Objective: We aimed to explore the experience and views of chronic disease physicians at 3 tertiary hospitals in Changsha, China, regarding opportunities and challenges in internet hospital care.

Methods: We conducted semistructured qualitative interviews with physicians (n=26) who had experience working in internet hospitals affiliated with chronic disease departments in 3 tertiary hospitals in Changsha, Hunan province, south central China. Interviews were transcribed verbatim and analyzed by content analysis using NVivo software (version 11; Lumivero).

Results: Physicians emphasized that internet hospitals expand opportunities to conduct follow-up care and health education for patients with chronic illnesses. However, physicians described disparities in access for particular groups of patients, such as patients who are older, patients with lower education levels, patients with limited internet or technology access, and rural patients. Physicians also perceived a gap between patients’ expectations and the reality of limitations regarding both physicians’ availability and the scope of services offered by internet hospitals, which raised challenges for doctor-patient boundaries and trust. Physicians noted challenges in doctor-patient communication related to comprehension and informed consent in internet hospital care.

Conclusions: This study explored the experience and views of physicians in 3 tertiary hospitals in Changsha, China, regarding access to care, patients’ expectations versus the reality of services, and doctor-patient communication in internet hospital care. Findings from this study highlight the need for physician training in telehealth communication skills, legislation regulating informed consent in telehealth care, public education clarifying the scope of internet hospital services, and design of internet hospitals that is informed by the needs of patient groups with barriers to access, such as older adults.

Introduction

As information technology develops rapidly in the current era, telehealth is growing exponentially in use [ 1 - 3 ]. Particularly in the wake of the COVID-19 pandemic, the use of telehealth for both primary and specialist care has accelerated around the globe [ 4 ]. In particular, telehealth is being implemented at an increasing scale in various countries with aging populations to improve health care access and quality for growing numbers of patients with chronic diseases [ 5 - 9 ].

The internet hospital is 1 major emerging telehealth model that is distinct to China, a country with a particularly large aging population and a high chronic disease burden [ 10 - 12 ]. Designed to make health care services more available, convenient, affordable, and efficient, internet hospitals are a type of online platform through which certain health care services can be conducted remotely. There are 3 main types of internet hospitals—those initiated by physical hospitals, those jointly established by physical hospitals and business enterprises, and those initiated by business enterprises relying on physical medical institutions. Research suggests that internet hospitals initiated solely by physical hospitals are the most widespread type [ 13 ]. In terms of the target patient population, internet hospitals primarily aim to facilitate services for patients with common illnesses requiring relatively simple treatment [ 14 ], patients with chronic diseases (diabetes, hypertension, and cancer) [ 15 ], and patients in remote and rural areas [ 16 ].

However, the scope of internet hospitals goes beyond telemedicine services for patients. Services provided by internet hospitals can be classified into three categories, (1) “core medical services,” which mainly include follow-up care for in-person medical services, telemedicine consultations, guidance on chronic disease management, and guidance on medication use; (2) “non-core medical services,” which mainly include medical consultations between health care providers and remote education for health care providers; and (3) “convenience services,” which mainly include health care appointment scheduling, mobile payment for health services, remote examination of medical test results, and dispensation and distribution of some medications [ 10 , 13 , 17 , 18 ]. Thus, the internet hospital model has the potential to increase access to health care for patients and training for providers, and to decrease costs across the health care system.

A number of recent policies by the Chinese national government have promoted rapid development and uptake of the internet hospital model. In 2015 [ 19 ] and 2018 [ 20 ], the State Council issued guidelines promoting “‘Internet +’ Healthcare,” which emphasized the development of internet hospitals as part of the “Health China” strategy for health care reform. Concurrently in 2018, the National Health Commission formulated specific regulations on internet hospital management, which officially authorized internet hospitals to facilitate a range of telehealth services and marked the start of their standardized development [ 21 - 23 ]. In 2020, the National Health Commission issued the “Notice on Strengthening Informatization to Support the Prevention and Control of the Novel Coronavirus Pneumonia Epidemic,” emphasizing the advantages of internet hospitals in controlling the spread of the COVID-19 pandemic [ 24 ]. In the wake of these policies, by June 2023, the number of internet hospitals had reached more than 3000, and 364 million of China’s 1.079 billion internet users were using online medical services [ 25 , 26 ]. However, research suggests that most internet hospitals are not yet fully developed or providing the full scope of services intended to achieve these goals [ 27 - 29 ].

At this early stage of the model’s development, little research to date has evaluated the views of Chinese medical professionals and patients regarding internet hospitals. However, research on telehealth in other countries reveals that telehealth raises many new concerns and challenges alongside the aforementioned opportunities [ 30 - 32 ]. One of the most common concerns raised by patients is the potential for misdiagnosis due to the inability to conduct physical examinations through telehealth [ 33 , 34 ]. Particular groups such as lower-income older adults also commonly report barriers to the use of telehealth such as lack of familiarity with technology or limited access to technological devices or internet connections [ 35 ]. Smartphone data or internet connection problems can also lower patient satisfaction and limit access among rural patient populations [ 36 , 37 ]. Various groups of patients also commonly report feeling concerned about patient privacy and the security and protection of medical data when using telehealth [ 38 ].

In order to guide the direction of internet hospital development in China, further research is needed to examine the emerging challenges and opportunities to patient care presented by this country-specific telehealth platform. The aim of this study was to explore the experience and views of chronic disease physicians at 3 tertiary hospitals in Changsha, China, regarding opportunities and challenges presented by internet hospital care.

The methodology whereby this study was designed and conducted is reported following the items in the Consolidated Criteria for Reporting Qualitative Research (COREQ) checklist [ 39 ]. See Multimedia Appendix 1 [ 39 ] for more information.

Setting, Participant Recruitment, and Eligibility Criteria

We conducted in-depth, semistructured interviews with physicians at 3 tertiary hospitals in Changsha, Hunan Province, south central China. Inclusion criteria for participants were 18 years of age or older, experience working in internet hospitals, and employment in a chronic disease department at one of the study hospitals. Our rationale for these inclusion criteria was to select doctors who had work experience relevant to the research questions. Enrollment occurred over a 2-month period from April to May 2022. Using a purposive sampling approach, we obtained a list of doctors who had previous experience working in internet hospitals. We then messaged or called the doctors on the list to briefly explain the primary and secondary objectives of the study, invited them to share their perspectives related to the study, and asked them to be available for interviews. Out of the 28 doctors contacted, 26 agreed to participate, while 2 physicians declined due to a lack of time. Those who responded positively to the invitation were subsequently contacted by the author, YZ, either via WeChat (Tencent Holdings Limited) or telephone to schedule an interview.

We recruited participants until reaching data saturation, at which point no new information about the meaning of codes or themes and the relationship between them continued to appear [ 40 ].

In order to allow spontaneous answers and mitigate bias, participants were given minimum information in advance about specific interview topics.

Ethical Considerations

In April 2022, the research protocol was approved by the institutional review board of Xiangya Hospital, Central South University (#202204092). No prior relationships existed between study participants and members of the research team. Verbal informed consent was recorded via an audio recorder for each participant before participation. Participants were informed in advance that their interviews would be recorded, with the assurance that these recordings would be subject to encryption for security purposes, and they provided their verbal consent accordingly. All participants received a compensation of 200 RMB (1 CNY=US $0.15 on May 2022) for their participation, which was disbursed through a WeChat transfer. To protect the information of the interviewees, the interview data were deidentified in the process of transcription from audio recordings.

Data Collection

The interview guide was collaboratively developed and then subjected to pilot-testing by the research team. Throughout the concurrent phases of data collection and analysis, the interview guide underwent iterative refinement in response to emerging insights and participant responses. This adaptive approach is considered vital to the robustness of qualitative research [ 41 ]. Revisions were implemented subsequent to discussions involving YZ, JH, and Xiaomin Wang, aiming to clarify, define, and critically examine emerging content from interviews as relevant to the research questions. All questions from the finalized interview guide are listed in Textbox 1 .

Q1: What are your views on internet hospitals?

Q2: Could you tell me about your experience working in internet hospitals?

Q3: What do you think are the biggest advantages of internet hospitals? Can you give some examples?

Q4: What do you think are the most troubling or difficult aspects of internet hospitals? Can you give some examples?

Q5: How do you inform the patients who come to the internet hospitals before treatment?

Q6: Do you think there are any differences between doctor-patient communication in internet hospitals and physical hospitals?

Q7: What impact do you think internet hospitals have on doctor-patient communication? Can you give some examples?

Q8: What training have you participated in regarding internet hospitals? What do you think of this training?

Q9: What do you think about the current status of the development of internet hospital laws and regulations in China? How could they be improved? What other aspects can promote the further development of internet hospitals?

Q10: Is there anything else you would like to add about doctor-patient communication in internet hospitals?

Data for this study were collected from April to May 2022. We carried out interviews through a combination of online and offline modalities, depending on each participant’s preference and availability. Online interviews were conducted remotely by video call, via the mobile app WeChat. Offline interviews were conducted in private rooms at the study hospitals. The interviews were conducted in Mandarin Chinese by the authors, YZ (a postgraduate student) and Xiaomin Wang (an associate professor). Both interviewers have received professional training in qualitative interviewing and had extensive experience conducting qualitative research prior to this study.

The research team discussed possible probes and follow-up questions before beginning interviews, and interviewers used them when necessary to draw out more information relevant to the main research question. Concurrently, a second interviewer assumed the role of an observer to ensure the standardization of interview methods and to mitigate potential biases.

Interviews were audio recorded, transcribed, and uploaded into qualitative data management NVivo software (version 11; Lumivero) on password-protected computers to facilitate the analysis. Field notes made by interviewers during the interview process were also stored on password-protected computers, to be used for reference by the research team during analysis. Interviews ranged from 20 to 50 minutes long. Transcripts were sent to participants upon request, but no corrections, comments, or notes were made to transcripts.

Data Analysis

Analysis of the data was performed through conventional content analysis, using guidelines described by Hsieh and Shannon [ 42 ]. An advantage of conventional content analysis is that it avoids using preconceived categories, to generate codes inductively from the data. This modality is considered appropriate when current knowledge of the phenomenon being researched is limited [ 42 ].

Authors YZ and Xuxi Wang transcribed all interviews verbatim and reviewed transcripts several times to acquire a thorough understanding of the whole data set. They then read transcripts line-by-line and highlighted keywords and sentences from a set of initial transcripts, to generate primary codes that captured key concepts. Primary codes were repeatedly reviewed and revised through discussion among the authors and comparison across the transcripts. A finalized codebook including 17 codes and 61 subcodes was used to code all interviews, using NVivo software (version 11). Data saturation was reached after 26 interviews, once the research team determined by the consensus that we had interviewed a sufficiently varied sample of physicians from the 3 study hospitals, while also having obtained sufficiently content-rich data.

Following the coding of all transcripts, all coded segments of the interview data were translated into English by authors YZ and Xuxi Wang, native Mandarin speakers, and double-checked for accuracy by author JH, a native English speaker. Codes and subcodes were repeatedly reviewed and were grouped into clusters according to similarities and differences. Clusters of codes were then treated as subcategories and aggregated into the main categories that were representative of the key findings. These categories were repeatedly reviewed until fully developed, through a process of identifying and comparing exemplary excerpts for each code, category, and subcategory.

This process of analysis culminated in three main categories describing the experiences and views of chronic disease physicians regarding the opportunities and challenges presented by internet hospital care, (1) advancements and shortcomings in care access due to internet hospitals, (2) patients’ expectations versus limitations on doctors’ availability and the scope of services—implications for doctor-patient boundaries and trust, and (3) advantages and downsides of online communication for comprehension and informed consent. These main categories are shown in Textbox 2 , alongside the subcategories from which they were aggregated, and further explained in the results narrative below.

Advancements and shortcomings in care access due to internet hospitals

• Enhanced ability to conduct follow-up care for patients with chronic illness
• More efficient channels for health education
• Disparities in access (ie, for older adults, patients with lower education levels, patients with limited internet or technology access, rural patients)

Patients’ expectations versus limitations on doctors’ availability and the scope of services

• Patients’ expectations of doctors’ availability create unclear professional boundaries.
• Patients’ expectations of the service scope of internet hospitals affect doctor-patient trust.

Advantages and downsides of online communication for comprehension and informed consent:

• Doctors value having extra time to think carefully about replies to patients’ messages, compared to in-person communication.
• Internet hospitals’ restrictions on consultation times, procedures, and arbitrary rules or schedules can hinder effective patient communication.
• Doctors have concerns about the quality of online diagnoses and advice, as well as patient accuracy and comprehension, due to the limitations of online care.
• Doctors have concerns about the completeness and uniformity of clinical informed consent in internet hospitals.

Description of Study Participants

The 26 participants came from 3 different affiliated hospitals with 10, 5, and 11 participants interviewed from each hospital, respectively. Participants ranged in age from 29 to 49 years, and all 26 participants had PhD degrees. Only 5 participants stated that they had received specific training for working in internet hospitals, and 1 of them stated that training included discussion of clinical ethics in internet hospital care. We interviewed doctors from several departments involved in care for patients with chronic disease—oncology, cardiovasology, hematology, endocrinology, gastroenterology, nephrology, and infection departments. Aggregated participant characteristics are presented in Table 1 .

a Some percentages may not add up to 100 due to rounding.

Advancements and Shortcomings in Care Access Due to Internet Hospitals: Follow-Up Care, Health Education, and Disparities

Most doctors stated that internet hospitals affiliated with their physical hospitals of employment were still in the early stages of development and that their internet hospital work experience mainly took place in enterprise-initiated internet hospitals. Doctors stated that internet hospitals initiated by physical hospitals were “not fully operational yet,” (Dr B) and “the consultation volume of patients is relatively small” (Dr C). They also suggested that there was currently a “lack of incentives” (Dr D) to work in internet hospitals initiated by physical hospitals, whereas enterprise-initiated internet hospitals offered “higher income” (Dr B) and a setting where “doctors set their own prices” (Dr C).

In both internet hospitals initiated by physical hospitals and enterprise-initiated internet hospitals, doctors stated that the majority of their work consisted of online consultation for common or easily diagnosable diseases, and follow-up services for patients with chronic diseases who had previously received care at physical hospitals, such as adjusting medications and ordering medical tests to be scheduled in person. Most doctors were motivated to work for internet hospitals particularly because of the opportunity to be part of expanding follow-up care for patients with chronic diseases.

Most of the patients who come to the internet hospitals are chronic disease patients, with conditions such as hypertension, diabetes, coronary heart disease, etc. These patients have been clearly diagnosed in our hospital, and some of them need to be guided or communicated with about what needs to be paid attention to in the process of home-based management. For example, patients' blood pressure might fluctuate, or they can consult online if they have any uncomfortable symptoms, which is quite common. [Dr X]

Some doctors also believed based on experience that internet hospitals could serve as a more efficient channel to provide health education for patients, particularly for the management of chronic diseases.

We also feel that doctors in tertiary hospitals do not have much time to do health education with patients, but through the internet hospitals platforms, we can explain to patients the concept of health or a healthy way of life. For example, for a patient with heart failure, he isn’t expected to come back to the hospital again and again, because I have instilled him with an understanding of healthy lifestyle and diet, and the workload of the doctor will be reduced in the long run. [Dr C]

Despite the ways in which doctors felt internet hospitals expanded access to care and services, they had also observed disparities in access to internet hospitals across several groups, including older adults and patients with lower levels of education or technological literacy: “Older patients may not use smartphones or might need assistance to do so from family members” (Dr E).

Relatively speaking, if the patients come to the internet hospital for consultation, the education level of these patients will be higher, otherwise they will not be able to fully communicate with their doctor. [Dr O]

Most doctors mentioned that internet hospitals are especially suitable for patients with chronic diseases. Doctors also stated that while older adults are one of the most common groups of patients with chronic diseases, many older adults have difficulties in using or accessing internet hospitals (Dr D and Dr X). Some doctors mentioned that internet hospitals currently have limited connections and overlap with local health services in rural areas. They believed moving toward more connection with local services was an important goal—particularly because the demand at tertiary hospitals frequently outstrips resources (Dr P and Dr Q), and because many rural patients travel long distances to receive care at tertiary hospitals.

Even for follow-up visits for chronic diseases or common diseases, many patients will still go to tertiary hospitals. Instead, the patient can go to a qualified local hospital and send us the results of the test, and then [through internet hospital care,] we can advise the patient or tell him how to adjust the medication, or refer the patient to a tertiary hospital for testing. But at present, internet hospitals have not played a big enough role in these aspects. [Dr E]

Patients’ Expectations Versus Limitations on Doctors’ Availability and the Scope of Services: Implications for Doctor-Patient Boundaries and Trust

Patients’ expectations of doctors’ availability create unclear professional boundaries.

When asked about new challenges in patient care posed by internet hospitals, only 1 doctor mentioned risks related to patient privacy and data security.

The internet hospital platform where I am located requires patients to provide their name, gender, age, ID number, and other information, which can be seen on the doctor's portal, but as a doctor I will definitely not disclose the patient's private information but just give him diagnostic advice according to the necessary information provided by the patient. [Dr G]

By contrast, many doctors expressed concerns about their own privacy. Some doctors shared stories from their work in physical hospitals of willingly sharing their personal WeChat with patients in case patients had questions after discharge (Dr D, Dr M, and Dr R). While some doctors did not seem to mind-bending this boundary with patients, others remembered negative experiences when patients had sent messages making demands of doctors’ time at all hours (Dr E, Dr M, Dr P, and Dr Z). They also recalled times when patients obtained the doctor’s personal contact information through their own means and contacted them after leaving the hospital without the doctor’s consent (Dr V and Dr Z). As a result, some doctors had positive views of internet hospitals because they can serve as a means for online communication with patients without requiring the doctor to disclose their own personal contact information.

I prefer to use the official platform to communicate with patients, rather than through private WeChat or phone, because I really don't want to receive phone calls or text messages from patients after I work. But if the phone does ring, I will take into account that he is an old patient of mine, and I will still answer it, because I am not sure if there is any emergency. But for patients on such online hospital platforms, I rarely give them my phone number and personal WeChat. [Dr P]

Several doctors were also uncomfortable that they were required to post information about themselves when working on internet hospital platforms, such as their name, location, and credentials (Dr P and Dr X). Because the audience of patients in internet hospitals is wider, they worried that patients who were dissatisfied with care may have the ability to post negative information about them in public forums online, citing their personal information. As a result, doctors stated that they would be more cautious in diagnosis and giving advice when dealing with patients in internet hospitals with whom they were less familiar (Dr J, Dr O, and Dr V).

Patients’ Expectations of the Service Scope of Internet Hospitals Affect Doctor-Patient Trust

Another concern expressed by many doctors was that patients held unrealistic expectations of the scope of services that internet hospital doctors provide. Some doctors mentioned that some patients feel that just because they spend money in an internet hospital, they should be able to get all their problems solved at once or get immediate treatment (Dr E and Dr G)—when in reality in many cases, the doctor might need to conduct tests over multiple online consultations or might recommend that patients seek further medical services in offline, physical settings. Doctors were concerned that patients’ dissatisfaction with unmet expectations might generate distrust toward the doctor.

What patients don't know is that in fact, most of the time, seeing a doctor is a step-by-step process, and it is necessary to do examinations step-by-step to exclude diseases or diagnose diseases. They often have high expectations for the effect of consultation in internet hospitals, and they think that doctors should be able to diagnose their diseases at one time; and not only to diagnose them, but also to propose a treatment plan. [Dr E]

Some doctors suggested that this gap between expectations and reality could be especially strong for new patients whom the doctor had not seen previously for in-person care. They described that they often recommend for new internet hospital patients to go to physical hospitals to be examined before receiving further internet hospital care or advice, and that patients who are expecting immediate solutions can find this disappointing (Dr U and Dr X). One doctor suggested the need to educate patients and the general public on the scope of internet hospital care, in light of this mismatch in patients’ and doctors’ expectations (Dr Y).

However, some doctors raised concerns about issues related to doctor-patient trust that had less to do with adjusting patients’ expectations and more to do with the format of online communication itself.

Doctor-patient trust will be better in physical hospitals. Because the doctor-patient relationship is a very special relationship, offline communication can observe the patient's expression, speed of speech, action, etc, and is more suitable for empathy with patients. When it comes to the doctor-patient relationship and trust, I think face-to-face consultation is still necessary. [Dr J]

Face-to-face communication in physical hospitals may be more detailed, because if it is through text messages or phone calls, I may be able to talk to the patient in a few words, but if the patient is communicating in our hospital, it may take me half an hour. Because there is unequal information in medicine itself, the patient himself is not very clear about medicine, and without adequate communication, there is no trust between doctors and patients. [Dr W]

Advantages and Downsides of Online Communication for Comprehension and Informed Consent

Doctors working in internet hospitals mainly used pictures and texts, and rarely video calls, to communicate with patients. Some doctors valued the extra time gained by this format to think carefully about their replies to patients’ messages (Dr E, Dr U, and Dr X). However, most doctors pointed out how the lack of nonverbal communication could increase miscommunication and misunderstandings.

Online communication is through typing, and some doctors can't see the facial expressions of patients, which is very inconvenient. The communication between doctors and patients may need body language, facial expressions and other aspects.... I want the patient to really understand me in terms of attitude or tone or feeling or whatever. [Dr G]

Doctors also expressed dissatisfaction with limitations on the time and procedures for consultation through various internet hospitals, and how sometimes arbitrary rules or schedules hindered communication with patients.

The doctors in our department need to be on duty every month in the internet hospitals. When it is my turn to be on duty, a patient will send his questions to me through the platform, but I think this mode is not good. For example, the patient might leave a message for me, but I am busy and don’t reply to him in time, and he may not see my reply in time when I reply. If I go back and forth with him several times, this problem will not be solved until I come back on duty next month, and then the patient's problem will not be solved at all. [Dr C]

The internet hospital at our hospital stipulates that patients can ask five questions at a time.... Sometimes doctors are not able to inquire in detail in order to understand the condition. [Dr G]

Doctors had concerns about the quality of diagnoses and advice that they provided online, due to the inability to do direct physical examination. These concerns were intensified by their perception that many patients could not describe their symptoms clearly and accurately.

Currently, a lot of people still lack of basic medical knowledge, which will lead to ineffective or inefficient consultation on the internet, because they cannot describe their own symptoms, or cannot collect their own data and then summarize it. Patients cannot provide information about their condition sufficiently and accurately, which will seriously affect the efficiency of consultation. [Dr E]

Two doctors also specifically mentioned the difficulty in internet hospital care of not being able to use the “four-diagnosis method”—a method used by doctors in traditional Chinese medicine for diagnosing illness, including diagnosis through observation, diagnosis through auscultation and olfaction, diagnosis through inquiry, and diagnosis through pulse feeling [ 43 ]. Although doctors in this study practiced mainly “Western” medicine, they described integrating certain traditional practices such as this method into their care at physical hospitals (Dr D and Dr H).

In light of concerns about the potential for miscommunication with patients, a few doctors also expressed uncertainty about the completeness and uniformity of clinical informed consent as it is currently practiced in internet hospitals. While they believed that a standardized process of online informed consent for medical advice and treatment was needed, they did not know of any relevant laws or procedures.

Because we have so little time to communicate online, and such a narrow scope of care services, we don't usually obtain informed consent online. I might listen to the patient explain his symptoms. I might tell him what tests he needs before I give my advice, or if I'm dealing with a familiar patient, I might just prescribe his medication, so there's no need for informed consent. However, I think how to issue online informed consent, whether online informed consent is legally effective, and how to sign online informed consent all need to be considered. This is also for the protection of medical staff. [Dr U]

Principal Findings

This study sheds light on previously underresearched aspects of internet hospitals in China, as both the first interview study to examine physicians’ perceptions of internet hospitals and one of the few studies on internet hospitals conducted in China outside of its most major cities. Our research revealed that physicians see enhanced opportunities in internet hospitals to conduct follow-up care for patients with chronic illnesses and to provide health education. However, physicians noted disparities in access for different groups, such as older adults, patients with lower education levels, patients with limited internet or technology access, and rural patients. One particularly novel finding was the conflict between patients’ expectations and the reality of limitations on doctors’ availability and the scope of services available through internet hospitals. Physicians perceived that this gap affected both boundaries and trust in the doctor-patient relationship. Physicians also discussed opportunities and challenges in doctor-patient communication, including issues of comprehension and informed consent. Considering that the development of internet hospitals involves multiple industries, including medical institutions, national policymaking departments, and technology providers, we raise several suggestions below on physician training, patient education, regulations, and design, as well as directions for future research.

Training for Doctors

Internet hospital care involves real-time online sharing of medical data. Information about both doctors and patients is centralized and easily accessible to authorized users on the internet hospital platform. Some doctors in our study were uncomfortable when required to publicly post their names, basic personal information, and credentials on internet hospital platforms because it might make them more vulnerable to public criticism. Doctors’ reasons for being concerned about this were in line with previous research showing a high degree of conflict in the doctor-patient relationship in China [ 44 , 45 ]. This underscores the importance of current efforts both locally and internationally that aim to rebuild trust in the doctor-patient relationship [ 46 - 48 ]. Considering doctors’ concerns about patients requesting for them to disclose their WeChat in both physical and internet hospital work, communication skills training for doctors should prepare doctors for how to interact with patients with empathy and care, while also maintaining their preferred professional boundaries.

It was also notable that only 1 doctor who was interviewed discussed concerns related to patient privacy and data security when asked about challenges presented by internet hospitals. By contrast, Li et al’s [ 49 ] study on the determinants of patients’ use of internet hospitals in China showed that while patients generally desire to use internet hospitals, they are apprehensive about the associated risk of their personal information being leaked. Due to the heightened potential for data leaks and breaches of patient health information associated with the use of internet hospitals, it is imperative that health care professionals undergo training to raise their awareness of data security precautions. For instance, physicians should be trained on proactive measures that they can take to guarantee that the internet hospital services they are affiliated with implement adequate security protocols around patient information. Furthermore, physicians should be trained to communicate with internet hospital patients or their legal proxies about potential risks related to data security and to apprise them of protective measures enacted to safeguard information. Future research should also evaluate the frequency with which data leaks and breaches in internet hospitals actually occur.

Findings from our study also suggest that internet hospitals have led to changes in doctor-patient communication. Doctors in our study considered it to be an advantage of internet hospital care that they generally had more time to communicate with patients compared to in-person care. However, a previous study on internet hospitals suggested that while doctors can obtain key information from patients within a few minutes through in-person communication and examination, information received in the same amount of time online tends to be more limited [ 43 ]. Research conducted by Deng et al [ 50 ] also highlights that engaging in online consultation work while simultaneously engaging in a main career providing in-person medical consultation may place excessive demands on doctors’ time and energy. This phenomenon of work overload could potentially impede the widespread adoption of internet hospitals and introduce added risk to medical practice.

Relatedly, doctors in our study mentioned that when working in internet hospitals, they could only communicate with patients in the form of text, pictures, or video-based consultations, and that they had to rely largely on patient self-report. Both of these factors caused doctors to worry about the accuracy of their diagnoses. This aligns with recent research showing that about 70% of surveyed health care providers believe communication difficulties between patients and health care providers result in online consultations being insufficient [ 51 ], and about 70% of providers report feeling apprehensive about the possibility of misdiagnosis when providing care through internet hospitals [ 51 ]. Recent research has also found that patients who use internet hospitals have more negative views on the doctor-patient relationship than nonusers—including both interpersonal factors such as the degree to which patients trust doctors and practical factors such as the degree to which patients agree with their doctors’ medical opinions. Studies from other countries have similarly shown that telehealth can present new challenges or deficiencies in communication [ 52 - 55 ].

To address such challenges, telehealth communication competencies need to become a core component of both future research and physician training for internet hospitals in China—just as similar competencies are emerging as a priority for telehealth enhancement around the world [ 56 ]. Physicians providing care through internet hospitals should undergo standardized training for web-based communication skills, as research from other countries suggests such training can adapt interpersonal skills to the telehealth environment [ 57 ] and enhance empathic expression. More training for physicians on this skill set might reduce their apprehension about communicating through internet hospitals and assist them in communicating in a manner that improves outcomes for patients. Considering that doctors in our study expressed concern about patient comprehension and diagnostic accuracy, further research is also needed to evaluate and establish methods for measuring patient satisfaction, patient comprehension of information communicated by doctors, and diagnostic accuracy in internet hospital care. Future research should also examine the feasibility of integrating traditional Chinese medical practices such as the four-diagnosis method into telehealth care in China.

Education for Patients or the Public

Findings from our study highlight new challenges in the doctor-patient relationship posed by internet hospital care. One especially novel finding in our study was doctors’ perception that patients subconsciously expected them to be online 24 hours a day, while doctors actually had limited hours working in the internet hospital and could not meet this expectation. Particularly when patients still needed to ask questions after the end of the physician’s available time for consultation, doctors described the risk of conflicts with patients. These findings suggest many patients may be unaware when message-based interactions with physicians in internet hospitals are discontinuous or asynchronous. Therefore, public information about internet hospitals should specify the boundaries of physicians’ availability for internet hospital consultations. While the scope of services may expand as internet hospitals continue to develop, information disseminated to the public should make it clear that internet hospital care is currently only intended for either follow-up care for previously diagnosed patients with chronic diseases, or for new patients with common and more easily diagnosable conditions. Finally, public education should equip patients or their proxies for distinct ways in which they might self-advocate for optimal care in the context of internet hospitals compared to in-person care. This may involve the development of interventions such as question-prompt lists that are specific to equipping patients for internet hospital consultations.

Regulations and Laws

Doctors in our study believed that difficulties with nonverbal communication in internet hospitals often led to miscommunication and misunderstanding, and many raised concerns that there were no specific laws regulating online doctor-patient communication. As a result, most doctors in our study expressed that they felt they were walking on eggshells concerning possible conflicts with patients. This builds on findings from the “2022 China E-hospital development research report” [ 51 ], in which one of the most common suggestions made by health care providers for the further development of internet hospitals was to standardize legal protection for doctors practicing in internet hospitals. Gaps in relevant laws and regulations may reduce the willingness of risk-conscious clinicians to provide medical services through internet hospitals.

Existing internet hospital laws and regulations in China are still mainly in the trial stage [ 51 , 58 ], and are being outpaced by evolving challenges in internet hospital care. The doctors in our study believed that internet hospitals may increase the difficulty of diagnosis and treatment, increase medical safety risks related to miscommunication, and increase the risk of medical malpractice liability. Research by Zhi et al describes how the inability of doctors to perform physical examinations or certain laboratory or imaging examinations through internet hospitals may compromise the accuracy of doctors’ judgments [ 59 ]. However, the legal responsibilities of physical medical institutions, internet hospitals, and doctors regarding issues such as these have not been fully clarified. We suggest that further refinement and clarification of these and other aspects of internet hospital law will help doctors feel more protected in their work and increase the motivation of doctors to work in internet hospitals.

Doctors in our study mentioned that China also lacks detailed legal provisions on the implementation of online informed consent. Internationally recognized ethics standards highlight 4 core elements of informed consent—capacity to consent, information disclosure, comprehension, and voluntary authorization [ 60 ]. Informed consent issues involved in telehealth in other countries are similar to those described by doctors working in internet hospitals in this study, namely, the degree of discernment required from providers to ensure that patients are sufficiently informed to provide consent increases dramatically in telehealth [ 61 ]. In the United States, different states have different regulations on remote informed consent, and no federal policy has been formed at present. Some states require patients to fill out and sign written consent forms, while others do not [ 62 ]. In China, the Administrative Measures for internet hospitals stipulate that “internet hospitals must warn patients of risks and obtain informed consent from patients” [ 22 ]. However, current laws in China do not provide clear rules regarding the validity of electronic signatures for informed consent in internet hospital care. Informed consent in internet hospital care also involves unique information security issues due to the use of electronic health records, but there is currently no specific legal guidance for internet hospital platform developers or doctors concerning data security protection of informed consent in internet hospital care.

Tackling Disparities

Our research revealed that while older adults are at higher risk for chronic illness and are the main target population for internet hospitals, they are also reported by doctors to experience a number of barriers to internet hospital use. This finding aligns with research from various countries showing that older adults are less likely than younger patients to express positive attitudes toward using telehealth [ 63 , 64 ]. Health care providers in China and other countries have observed that older adults may be apprehensive about telehealth due to difficulty in operating computers or smartphones [ 65 ], may need the help of care partners to log into telehealth accounts successfully, and may need more time on average to download and set up applications [ 43 , 66 ]. Previous surveys have also shown that medical personnel believe that the low efficiency of online communication between doctors and patients and the low internet use rate of some patient groups (such as older adults) are the main factors hindering the development of internet hospitals [ 51 ].

Previous research in various countries has shown that the ease of use and perceived usefulness of telehealth systems have a positive impact on the acceptance of telehealth in patients who are older [ 67 , 68 ]. However, to date, China has not established an effective quality control system for internet hospitals [ 27 , 69 ], and the aforementioned ways in which internet hospitals currently pose increased risks for patient safety may affect general patients’, let alone older adults’ willingness to use them [ 49 ]. We recommend that the needs of older adults be considered in the design and development of internet hospital platforms and that older adults participate in the system design process [ 70 , 71 ]. Community health workers may be a workforce that could be mobilized to support telehealth training efforts among patients who are older, assist individuals with limited telehealth literacy in attending online appointments, and provide culturally and linguistically appropriate information about telehealth to rural patients and communities [ 64 , 72 , 73 ]. In general, health care organizations should invest in developing internet hospitals that are functional and easy to use. Drawing from research on telehealth design improvement in other countries, internet hospitals could be designed with features in mind to help physicians communicate more clearly with patients, such as providing notifications to physicians when patients read messages [ 74 ]. In addition, it may be beneficial for platforms to provide training materials to patients when patients register and log into internet hospitals for the first time [ 75 ]. Considering that a major goal of internet hospital development is to expand health care access, it will be crucial to address disparities in internet hospital use through these and other educational and design considerations.

Limitations

This study should be interpreted in the light of certain limitations. As most participants interviewed were attending physicians, findings may not be generalizable to the perspective of other health care workers or patients. The generalizability of our study findings from a single region and time point may also be limited, as there may be variations in internet hospital features and practice in other regions in China, and over time as internet hospitals continue to develop rapidly.

Conclusions

This study explored the experience and views of physicians in 3 tertiary hospitals in Changsha, China regarding access to care, patients’ expectations versus the reality of services, and doctor-patient communication in internet hospital care. Findings from this study indicate that there is a need to train physicians in telehealth-specific communication skills. National policymaking departments should also further refine laws and regulations concerning internet hospitals, particularly those related to online informed consent. Technology developers should take the needs of older adults into particular account in the design of internet hospital platforms.

Acknowledgments

This study was supported by Major Program of National Social Science Fund of China, Research on Moral Issues in the Field of Contemporary Science and Technology (22&ZD044), the Xiangya Medical Humanities Series: Principles of Biomedical Ethics (monograph Award) Fund (KTZZPT019), Hunan Provincial Innovation Foundation for Postgraduate (CX20220133), the Fundamental Research Funds for the Central Universities of Central South University (2022ZZTS0041), the China Scholarship Council (CSC,202206370069), and the Natural Science of Changsha City (kq2202362).

Data Availability

The data generated and analyzed during this study are available from the corresponding author upon reasonable request.

Authors' Contributions

YZ and XL conceptualized this study and designed the methodology. YZ and Xiaomin Wang conducted the interviews for data collection. Xuxi Wang and YZ transcribed the interviews. YZ, Xuxi Wang, YW, and XZ conducted and provided resources for preliminary analysis of the data. YZ and JH wrote and edited the paper. JH, XL, and XZ oversaw the implementation of all study activities. All authors read and approved the final paper.

Conflicts of Interest

None declared.

COREQ (Consolidated Criteria for Reporting Qualitative Research) checklist.

• Barbosa W, Zhou K, Waddell E, Myers T, Dorsey ER. Improving access to care: telemedicine across medical domains. Annu Rev Public Health. Apr 01, 2021;42:463-481. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kirkland EB, DuBose-Morris R, Duckett A. Telehealth for the internal medicine resident: a 3-year longitudinal curriculum. J Telemed Telecare. Oct 2021;27(9):599-605. [ CrossRef ] [ Medline ]
• Losorelli SD, Vendra V, Hildrew DM, Woodson EA, Brenner MJ, Sirjani DB. The future of telemedicine: revolutionizing health care or flash in the pan? Otolaryngol Head Neck Surg. Aug 2021;165(2):239-243. [ CrossRef ] [ Medline ]
• Hafner M, Yerushalmi E, Dufresne E, Gkousis E. The potential socio-economic impact of telemedicine in Canada. Rand Health Q. Jun 2022;9(3):6. [ FREE Full text ] [ Medline ]
• Alhamam NM, Buhalim RA, Almakhayitah IH, AlBahr AW, AlYaeesh IA. Telemedicine for musculoskeletal care during the COVID-19 pandemic: evaluating readiness of Saudi citizens. Cureus. Feb 16, 2021;13(2):e13380. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bhaskar S, Bradley S, Chattu VK, Adisesh A, Nurtazina A, Kyrykbayeva S, et al. Telemedicine across the globe-position paper from the COVID-19 pandemic health system resilience PROGRAM (REPROGRAM) international consortium (part 1). Front Public Health. 2020;8:556720. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Cioti AM, Stanescu AMA, Grajdeanu IV, Serban B, Popescu E, Bratu OG, et al. Telemedicine in Europe—current status and future perspectives. Rev Med Mod. 2019;26(4):165-168. [ FREE Full text ] [ CrossRef ]
• Raja M, Bjerkan J, Kymre IG, Galvin KT, Uhrenfeldt L. Telehealth and digital developments in society that persons 75 years and older in European countries have been part of: a scoping review. BMC Health Serv Res. Oct 26, 2021;21(1):1157. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Brucksch S. Japan and its rapid ageing society: does e-health technology provide a solution? J Aging Sci. 2018;6(2):1-4. [ FREE Full text ] [ CrossRef ]
• Han Y, Lie RK, Guo R. The internet hospital as a telehealth model in China: systematic search and content analysis. J Med Internet Res. Jul 29, 2020;22(7):e17995. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Scott R, Mars M. Telehealth in the developing world: current status and future prospects. SHTT. Feb 2015;2015(3):25-37. [ FREE Full text ] [ CrossRef ]
• Yip W, Fu H, Chen AT, Zhai T, Jian W, Xu R, et al. 10 years of health-care reform in China: progress and gaps in Universal Health Coverage. Lancet. Sep 28, 2019;394(10204):1192-1204. [ CrossRef ] [ Medline ]
• 2021 China E-hospital development report. National Telemedicine and Connected Health Center and CN-Healthcare. 2021. URL: https://zk.cn-healthcare.com/doc-show-53644.html [accessed 2023-09-17]
• Chen X, Wu X, Zhang Q, Jing R, Cheng W, Tian J, et al. The construction and operational models of internet hospitals in China: a hospital-based survey study. BMC Health Serv Res. Jun 21, 2023;23(1):669. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Houston TK, Allison JJ. Users of Internet health information: differences by health status. J Med Internet Res. 2002;4(2):E7. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lai Y, Chen S, Li M, Ung COL, Hu H. Policy interventions, development trends, and service innovations of internet hospitals in China: documentary analysis and qualitative interview study. J Med Internet Res. Jul 20, 2021;23(7):e22330. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Yu T, Jin C, Wu X, Yue D. Implementation of shared decision-making within internet hospitals in China based on patients' needs: feasibility study and content analysis. JMIR Form Res. Jan 06, 2023;7:e39965. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tu J, Wang C, Wu S. The internet hospital: an emerging innovation in China. Lancet Glob Health. Aug 2015;3(8):e445-e446. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Guiding opinions of the State Council on actively promoting the "Internet +" action. The State Council of the People's Republic of China. 2015. URL: https://www.gov.cn/zhengce/content/2015-07/04/content_10002.htm [accessed 2023-09-16]
• Opinions on promoting the development of "Internet + medical health". General Office of the State Council. 2018. URL: https://www.gov.cn/zhengce/content/2018-04/28/content_5286645.htm [accessed 2023-09-16]
• Internet diagnosis and treatment management measures (trial implementation). National Health Commission of the People's Republic of China. 2018. URL: https://www.gov.cn/gongbao/content/2019/content_5358684.htm [accessed 2023-09-16]
• Internet hospital management measures (trial implementation). National Health Commission of the People's Republic of China. 2018. URL: https://www.gov.cn/gongbao/content/2019/content_5358684.htm [accessed 2023-09-16]
• Telemedicine service management standards (trial implementation). National Health Commission of the People's Republic of China. 2018. URL: https://www.gov.cn/gongbao/content/2019/content_5358684.htm [accessed 2023-09-16]
• Notice of the general office of the NHC on strengthening informatization to support the prevention and control of the novel coronavirus pneumonia epidemic. National Health Commission of China. 2020. URL: https://www.gov.cn/zhengce/zhengceku/2020-02/05/content_5474692.htm [accessed 2023-09-16]
• The 52nd statistical report on China's internet development. China Internet Network Information Center. 2023. URL: https://www.cnnic.cn/NMediaFile/2023/0908/MAIN1694151810549M3LV0UWOAV.pdf [accessed 2024-02-28]
• There are more than 3,000 internet hospitals, and it is necessary to explore a better development path. Health Times. 2023. URL: http://www.jksb.com.cn/index.php?m=content&c=index&a=show&catid=788&id=200853 [accessed 2024-02-28]
• Qiu Y, Liu Y, Ren W, Qiu Y, Ren J. Internet-based and mobile-based general practice: cross-sectional survey. J Med Internet Res. Sep 25, 2018;20(9):e266. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Xie X, Zhou W, Lin L, Fan S, Lin F, Wang L, et al. Internet hospitals in China: cross-sectional survey. J Med Internet Res. Jul 04, 2017;19(7):e239. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Yuan J, He F. Status quo and problems of the internet hospitals in China. Chin Rural Health Serv Adm. 2023;43(02):126-128+140. [ CrossRef ]
• Pogorzelska K, Chlabicz S. Patient satisfaction with telemedicine during the COVID-19 pandemic-a systematic review. Int J Environ Res Public Health. 2022;19(10):6113. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sagaro GG, Battineni G, Amenta F. Barriers to sustainable telemedicine implementation in Ethiopia: a systematic review. Telemed Rep. 2020;1(1):8-15. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Şahin E, Yavuz Veizi BG, Naharci MI. Telemedicine interventions for older adults: a systematic review. J Telemed Telecare. Feb 2024;30(2):305-319. [ CrossRef ] [ Medline ]
• Imlach F, McKinlay E, Middleton L, Kennedy J, Pledger M, Russell L, et al. Telehealth consultations in general practice during a pandemic lockdown: survey and interviews on patient experiences and preferences. BMC Fam Pract. Dec 13, 2020;21(1):269. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kumar S, Kumar A, Kumar M, Kumar A, Arora R, Sehrawat R. Feasibility of telemedicine in maintaining follow-up of orthopaedic patients and their satisfaction: a preliminary study. J Clin Orthop Trauma. Oct 2020;11(Suppl 5):S704-S710. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Latulipe C, Gatto A, Nguyen HT, Miller DP, Quandt SA, Bertoni AG, et al. Design considerations for patient portal adoption by low-income, older adults. Proc SIGCHI Conf Hum Factor Comput Syst. Apr 2015;2015:3859-3868. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Greenberg AJ, Haney D, Blake KD, Moser RP, Hesse BW. Differences in access to and use of electronic personal health information between rural and urban residents in the United States. J Rural Health. 2018;34(Suppl 1):s30-s38. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Johnson BA, Lindgren BR, Blaes AH, Parsons HM, LaRocca CJ, Farah R, et al. The new normal? Patient satisfaction and usability of telemedicine in breast cancer care. Ann Surg Oncol. 2021;28(10):5668-5676. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Solimini R, Busardò FP, Gibelli F, Sirignano A, Ricci G. Ethical and legal challenges of telemedicine in the era of the COVID-19 pandemic. Medicina (Kaunas). 2021;57(12):1314. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tong A, Sainsbury P, Craig J. Consolidated Criteria for Reporting Qualitative Research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349-357. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hennink MM, Kaiser BN, Marconi VC. Code saturation versus meaning saturation: how many interviews are enough? Qual Health Res. 2017;27(4):591-608. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Klem NR, Bunzli S, Smith A, Shields N. Demystifying qualitative research for musculoskeletal practitioners part 5: rigor in qualitative research. J Orthop Sports Phys Ther. 2022;52(2):60-62. [ CrossRef ] [ Medline ]
• Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277-1288. [ CrossRef ] [ Medline ]
• Li Y, Hu H, Rozanova L, Fabre G. COVID-19 and internet hospital development in China. Epidemiologia (Basel). 2022;3(2):269-284. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Jiang MM, Wu ZY, Tu AX. Research on the cooperative governance path of multiple stakeholders in doctor-patient disputes under the environment of information asymmetry. Int J Environ Res Public Health. 2023;20(2):1597. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zhao D, Zhang Z. Changes in public trust in physicians: empirical evidence from China. Front Med. 2019;13(4):504-510. [ CrossRef ] [ Medline ]
• Du L, Xu J, Chen X, Zhu X, Zhang Y, Wu R, et al. Rebuild doctor-patient trust in medical service delivery in China. Sci Rep. 2020;10(1):21956. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Fu Y, Tang T, Long J, Lin B, Li J, Quan G, et al. Factors associated with using the internet for medical information based on the doctor-patient trust model: a cross-sectional study. BMC Health Serv Res. 2021;21(1):1268. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tucker JD, Wong B, Nie JB, Kleinman A, Patient-Physician Trust Team. Rebuilding patient-physician trust in China. Lancet. 2016;388(10046):755. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Li D, Hu Y, Pfaff H, Wang L, Deng L, Lu C, et al. Determinants of patients' intention to use the online inquiry services provided by internet hospitals: empirical evidence from China. J Med Internet Res. 2020;22(10):e22716. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Deng W, Yang T, Deng J, Liu R, Sun X, Li G, et al. Investigating factors influencing medical practitioners' resistance to and adoption of internet hospitals in China: mixed methods study. J Med Internet Res. 2023;25:e46621. [ FREE Full text ] [ CrossRef ] [ Medline ]
• 2022 China E-hospital development research report. Shanghai Jiao Tong University. 2022. URL: https://www.100ec.cn/detail--6620464.html [accessed 2023-01-08]
• Alpert JM, Dyer KE, Lafata JE. Patient-centered communication in digital medical encounters. Patient Educ Couns. 2017;100(10):1852-1858. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Alpert JM, Krist AH, Aycock RA, Kreps GL. Applying multiple methods to comprehensively evaluate a patient portal's effectiveness to convey information to patients. J Med Internet Res. May 17, 2016;18(5):e112. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Gerber DE, Beg MS, Duncan T, Gill M, Craddock Lee SJ. Oncology nursing perceptions of patient electronic portal use: a qualitative analysis. Oncol Nurs Forum. Mar 01, 2017;44(2):165-170. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sieck CJ, Hefner JL, Schnierle J, Florian H, Agarwal A, Rundell K, et al. The rules of engagement: perspectives on secure messaging from experienced ambulatory patient portal users. JMIR Med Inform. 2017;5(3):e13. [ FREE Full text ] [ CrossRef ] [ Medline ]
• van Galen LS, Wang CJ, Nanayakkara PWB, Paranjape K, Kramer MHH, Car J. Telehealth requires expansion of physicians' communication competencies training. Med Teach. 2019;41(6):714-715. [ CrossRef ] [ Medline ]
• Wright HH, O'Shea MC, Sekula J, Mitchell LJ. Assessment of communication skills using telehealth: considerations for educators. Front Med (Lausanne). 2022;9:841309. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Huang M, Wang J, Nicholas S, Maitland E, Guo Z. Development, status quo, and challenges to China's health informatization during COVID-19: evaluation and recommendations. J Med Internet Res. 2021;23(6):e27345. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zhi L, Yin P, Ren J, Wei G, Zhou J, Wu J, et al. Running an internet hospital in China: perspective based on a case study. J Med Internet Res. 2021;23(9):e18307. [ FREE Full text ] [ CrossRef ] [ Medline ]
• World Medical Association. World Medical Association declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA. 2013;310(20):2191-2194. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Blake JH, Schwemmer MK, Sade RM. The patient-surgeon relationship in the cyber era: communication and information. Thorac Surg Clin. 2012;22(4):531-538. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Fields BG. Regulatory, legal, and ethical considerations of telemedicine. Sleep Med Clin. 2020;15(3):409-416. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Chiang KF, Wang HH, Chien IK, Liou JK, Hung CL, Huang CM, et al. Healthcare providers' perceptions of barriers in implementing of home telecare in Taiwan: a qualitative study. Int J Med Inform. 2015;84(4):277-287. [ CrossRef ] [ Medline ]
• Gray DM, Joseph JJ, Olayiwola JN. Strategies for digital care of vulnerable patients in a COVID-19 world-keeping in touch. JAMA Health Forum. 2020;1(6):e200734. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Miller DP, Latulipe C, Melius KA, Quandt SA, Arcury TA. Primary care providers' views of patient portals: interview study of perceived benefits and consequences. J Med Internet Res. 2016;18(1):e8. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Varsi C, Gammon D, Wibe T, Ruland CM. Patients' reported reasons for non-use of an internet-based patient-provider communication service: qualitative interview study. J Med Internet Res. 2013;15(11):e246. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Cimperman M, Brenčič MM, Trkman P. Analyzing older users' home telehealth services acceptance behavior-applying an extended UTAUT model. Int J Med Inform. 2016;90:22-31. [ CrossRef ] [ Medline ]
• Zhou M, Zhao L, Kong N, Campy KS, Qu S, Wang S. Factors influencing behavior intentions to telehealth by Chinese elderly: an extended TAM model. Int J Med Inform. 2019;126:118-127. [ CrossRef ] [ Medline ]
• 2020 research report on the development of internet hospitals in China. Institute of Healthcare. 2020. URL: http://zk.cn-healthcare.com/doc-show-39773.html [accessed 2023-09-17]
• Bhattarai P, Phillips JL. The role of digital health technologies in management of pain in older people: an integrative review. Arch Gerontol Geriatr. 2017;68:14-24. [ CrossRef ] [ Medline ]
• Cosco TD, Fortuna K, Wister A, Riadi I, Wagner K, Sixsmith A. COVID-19, social isolation, and mental health among older adults: a digital catch-22. J Med Internet Res. 2021;23(5):e21864. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Huang W, Long H, Li J, Tao S, Zheng P, Tang S, et al. Delivery of public health services by Community Health Workers (CHWs) in primary health care settings in China: a systematic review (1996-2016). Glob Health Res Policy. 2018;3:18. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lohr AM, Ingram M, Nuñez AV, Reinschmidt KM, Carvajal SC. Community-clinical linkages with community health workers in the United States: a scoping review. Health Promot Pract. 2018;19(3):349-360. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Laukka E, Huhtakangas M, Heponiemi T, Kujala S, Kaihlanen AM, Gluschkoff K, et al. Health care professionals' experiences of patient-professional communication over patient portals: systematic review of qualitative studies. J Med Internet Res. 2020;22(12):e21623. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hefner JL, Sieck CJ, Walker DM. Patient and physician perspectives on training to improve communication through secure messaging: clarifying the rules of engagement. Health Care Manage Rev. 2022;47(1):3-11. [ CrossRef ] [ Medline ]

Abbreviations

Edited by T de Azevedo Cardoso; submitted 22.03.23; peer-reviewed by Y Cao, N Mungoli, A Gangadhara Rao; comments to author 06.09.23; revised version received 27.09.23; accepted 26.02.24; published 29.03.24.

©Yuqiong Zhong, Jessica Hahne, Xiaomin Wang, Xuxi Wang, Ying Wu, Xin Zhang, Xing Liu. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 29.03.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Wiley-Blackwell Online Open

Language: English | Chinese

A quantitative systematic review of the association between nurse skill mix and nursing‐sensitive patient outcomes in the acute care setting

急症护理情况下护士技巧组合与护理敏感类患者结果之间联系的定量系统综述, diane e. twigg.

1 School of Nursing and Midwifery, Edith Cowan University, Joondalup Western Australia, Australia

2 Centre for Nursing Research, Sir Charles Gairdner Hospital, Nedlands Western Australia, Australia

Yvonne Kutzer

Elisabeth jacob, karla seaman, associated data.

To examine the association between nurse skill mix (the proportion of total hours provided by Registered Nurses) and patient outcomes in acute care hospitals.

A quantitative systematic review included studies published in English between January 2000 – September 2018.

Data sources

Cochrane Library, CINAHL Plus with Full Text, MEDLINE, Scopus, Web of Science and Joanna Briggs Institute were searched. Observational and experimental study designs were included. Mix‐methods designs were included if the quantitative component met the criteria.

Review methods

The Systematic Review guidelines of the Joanna Briggs Institute and its critical appraisal instrument were used. An inverse association was determined when seventy‐five percent or more of studies with significant results found this association.

Sixty‐three articles were included. Twelve patient outcomes were inversely associated with nursing skill mix (i.e., higher nursing skill mix was significantly associated with improved patient outcomes). These were length of stay; ulcer, gastritis and upper gastrointestinal bleeds; acute myocardial infarction; restraint use; failure‐to‐rescue; pneumonia; sepsis; urinary tract infection; mortality/30‐day mortality; pressure injury; infections and shock/cardiac arrest/heart failure.

Nursing skill mix affected 12 patient outcomes. However, further investigation using experimental or longitudinal study designs are required to establish causal relationships. Consensus on the definition of skill mix is required to enable more robust evaluation of the impact of changes in skill mix on patient outcomes.

Skill mix is perhaps more important than the number of nurses in reducing adverse patient outcomes such as mortality and failure to rescue, albeit the optimal staffing profile remains elusive in workforce planning.

目的

在于探讨护士技巧组合(注册护士提供的总时数比例)与急性护理医院患者结果之间的联系。

设计

一项定量系统综述包括2000年1月至2018年9月期间以英语发表的各项研究。

资料来源

搜索了Cochrane Library、CINAHL Plus with Full Text、MEDLINE、Scopus、Web of Science和Joanna Briggs Institute。纳入了观察性和实验性研究设计。如果定量成分符合标准,则纳入混合方法设计。

综述方法

使用了Joanna Briggs Institute的系统综述指南及其关键评估工具。当75%或者更多具备显著结果的研究发现这种联系时,就确定了反向关联。

结果

共纳入了63篇文章。12名患者的结果与护理技巧组合呈负相关(即,较高的护理技能组合与患者结果的改善显著相关)。这些是住院天数;溃疡、胃炎和上消化道出血;急性心肌梗死;约束使用;救援失败;肺炎;败血症;尿路感染;死亡率/30天死亡率;压力性损伤;感染和休克/心脏骤停/心力衰竭。

结论

护理技巧组合影响了12例患者的结果。然而,需要使用实验性或纵向研究设计来进一步调查,从而建立因果关系。需要就技巧组合的定义达成共识,以便能够更有力地评估技巧组合变化对患者结果的影响。

影响

在降低死亡率和救援失败等不良患者结果方面,技巧组合可能比护士人数更为重要,尽管最佳的人员配置在人力资源规划中仍然让人难懂。

1. INTRODUCTION

In their report in the year 2000, the World Health Organization highlighted both the importance and the challenges associated with finding the right mixture of healthcare personnel to achieve the best possible outcomes with the staffing resources available (World Health Organization, 2000 ). In this regard, consideration of the nursing workforce is critical. Skill mix, defined as ‘the proportion of total nurse hours provided by Registered Nurses’ (Twigg, Duffield, Bremner, Rapley, & Finn, 2012 ) (page 2,711), is an important element and involves assessing the mix of nursing staff, both registered and unregistered on a ward and varies in configuration dependent on the country where it is discussed. The nurse mix may include Registered Nurses (RNs), Enrolled Nurses, licensed practical nurses (LPNs), certified nursing assistants (CNAs), assistants in nursing (AINs), healthcare assistants, or other classifications (Jacob, McKenna, & D'Amore, 2015 ).

Various studies have since focussed on the importance of nurse staffing levels in an attempt to define an optimal configuration (Aiken, Clarke, & Sloane, 2002 ; Kim & Bae, 2018 ; Leary et al., 2016 ). Whilst research has explored staffing levels and their impact on patient outcomes, it does not always address nursing skill mix. A systematic review and meta‐analysis published in 2007 for example (Kane, Shamliyan, Mueller, Duval, & Wilt, 2007 ), which analysed nurse staffing levels, focussed on the ratio of Registered Nurses (RNs) to patients and patient outcomes, but not on nurse skill mix.

Despite these efforts, clear evidence‐based guidelines on staffing levels are lacking, particularly in regard to skill mix (Brennan, Daly, & Jones, 2013 ; Sharma, Hastings, Suter, & Bloom, 2016 ). A recent systematic review (Myers, Pugh, & Twigg, 2018 ) that examined the importance of nurse skill mix on patient outcomes focussed specifically on stand‐alone high acuity areas, meaning that findings from this review may not be easily transferrable to other care settings. The only other systematic review on skill mix and patient outcomes (Lankshear, Sheldon, & Maynard, 2005 ) examined articles published up to 2004 and highlighted the relationship between nurse staffing factors (Registered Nurse staffing levels and proportion of RNs in the skill mix) and patient outcomes. Their outcomes included mortality rates, complication rates (pneumonia, urinary tract infections, nosocomial infections, wound infections), failure‐to‐rescue, incidence of adverse events (falls, medication errors), length of stay, or patient satisfaction.

Of additional concern, continuing economic constraints and impending nursing shortages worldwide have led to the increased deployment of less qualified and unregulated health professionals, leading to changes in skill mix in the configuration of the nursing teams (Jacob et al., 2015 ; Roche, Duffield, Friedman, Dimitrelis, & Rowbotham, 2016 ). This is despite at least one study finding increases in unregulated workers has been associated with poorer patient outcomes (Twigg et al., 2016 ). With limited synthesis of the evidence to inform such staffing decisions, there is little guidance for policy makers and managers making those decisions.

The present review aims to expand on previous research by examining more recent studies up to 2018 exploring the impact of nurse skill mix on patient outcomes. The outcome of the review may help inform staffing policy in regard to skill mix changes.

1.1. Background

The conceptual framework developed by McCloskey and Diers ( 2005 ) was used to guide this review and the selection of variables. McCloskey and Diers ( 2005 ) examined the effects of health policy on nursing and patient outcomes sing the work of Aiken et al. ( 2002 ). McCloskey and Diers ( 2005 ) modified Aiken's framework to embed the seminal work of Donabedian's structure‐process‐outcomes framework (Donabedian, 1966 ). Structure was identified as nursing workforce characteristics such as nurse‐to‐patient ratios and skill mix. Process was identified as the processes of care. Outcome was identified as nurse and patient outcomes. This framework has been used to guide further studies including an examination of skill mix (Twigg, Duffield, Bremner, Rapley, & Finn, 2011 ; Twigg et al., 2012 ; Twigg, Gelder, & Myers, 2015 ). In this review, the proportion of total nurse hours provided by Registered Nurses was the structural variable and patient outcomes the outcome variable under review. It was hypothesized that changes in skill mix, for example, fewer Registered Nurse hours, affect the processes of care (such as recognition and response to patient changes) which in turn may impact on patient outcomes.

2. THE REVIEW

The aim of the review was to synthesize the available quantitative evidence on the association of nursing skill mix and patient outcomes sensitive to nursing care in adult patients in acute care hospitals. As such, the review question was: What effect does skill mix have on nurse sensitive patient outcomes?

2.2. Design

The review used a quantitative systematic literature review. Methods of the analysis and inclusion and exclusion criteria were identified in advance and documented in a research protocol. The PICOS framework (Population; Intervention; Comparator; Outcome; Study design) (Schardt, Adams, Owens, Keitz, & Fontelo, 2007 ) was used to refine the inclusion and exclusion criteria and processes and outcomes were guided by the Joanna Brigg Institute for Systematic Reviews. Table ​ Table1 1 outlines the PICOS framework.

PICOS framework

Relevant patient outcomes were identified by reviewing existing literature on the topic (Aiken et al., 2014 ; Duffield et al., 2011 ; Needleman, Buerhaus, Mattke, Stewart, & Zelevinsky, 2002 ; Twigg et al., 2011 ). The review focussed on acute care hospitals and included studies set in general medical, surgical, combined medical/surgical and step‐down wards, telemetry units and emergency departments. Acute care settings were chosen as the purpose of this review was to expand on previous studies by specifically examining the importance of skill mix in optimal staffing and previous studies were primarily undertaken in acute care settings. Nurse‐reported patient outcomes were not included in the study as the outcome was not directly measured. Exclusion criteria were applied as listed in Table ​ Table1 1 .

2.3. Search methods

Prior to commencing the review, searches were run in Medline and CINAHL to identify whether similar systematic reviews had already been completed in the last 10 years. The PROSPERO register was also checked to determine whether similar reviews were already underway. One study was found to be similar and at the initial protocol development stage, the corresponding author was contacted to determine the progress of the review; however; there was no response provided nor was any output from the systematic review published. A copy of the protocol for this study is provided in the Data S1 (Skill Mix OnlineSUPP_A_SystReviewProtocol).

Studies were identified through searching electronic databases, hand searching the reference lists of relevant articles, author searches, and via examining grey literature databases. In addition, an EBSCOhost Alert Notification and a Web of Science Search Alert was set up to capture any new publications between September 2016 and June 2017. An updated search strategy was conducted in September 2018. A health information librarian was consulted during the development of the MeSH terms (nursing assistants; mortality; sepsis; urinary tract infection; pneumonia; clinical deterioration; quality of health care; patient safety; accidental falls; surgical wound infection; venous thrombosis; shock; heart arrest; medication errors; infection; central nervous system infections). The search strategy was applied to Medline and CINAHL Plus with Full Text and was adapted for Embase, the Cochrane Library, Scopus, Web of Science and the Joanna Briggs Institute Database. The search strategy for each database is outlined in the Data S2 (Skill Mix OnlineSUPP_B_ElectDatabaseSearchStrat.pdf). Grey literature catalogues used included Research Online, Open Grey/EThOS, OAIster, Google Advanced Search and PsycEXTRA.

Limiters included articles published between January 2000 – September 2018 and written in English.

2.4. Search outcome

The search strategy identified a total of 2,702 citations from electronic databases. A comprehensive search for grey literature and hand searches produced an additional 12 articles. After removing duplicates, 2,576 articles were retained and their titles and abstracts subsequently screened for inclusion. This process led to 2,481 citations being excluded and the full‐text of the 95 remaining publications was assessed for eligibility.

2.5. Quality appraisal

The critical appraisal of the selected 95 records was undertaken using the Joanna Briggs Institute (JBI) critical appraisal tools (Godfrey & Harrison, 2015 ). All studies were examined independently by two reviewers, and the results of the critical appraisal reviewed after both reviewers had completed their assessment and discrepancies resolved by consensus.

Thirty‐two articles were excluded following the full‐text assessment and are detailed further in Data S3 (Skill Mix OnlineSUPP_C1‐4_CritAppraisals.pdf). Of these, three studies were excluded as they were deemed to be of insufficient quality (no reported data, unequal comparison group, small sample size); six did not review any skill mix variables and a further five did not measure any nursing‐sensitive patient outcomes; two studies were conducted in a nursing home setting and therefore did not meet inclusion criteria; five studies were literature reviews, three articles could not be sourced and an additional eight articles were excluded for other reasons (not primary research articles, measured nurse perception, cost analysis). This left 63 articles, which were included in the narrative summary. A record of study assignment was kept, which contained the reviewing authors recommendations about inclusion or exclusion as well as some comments on why an article was retained or rejected. Figure ​ Figure1 1 outlines the study flow chart and the articles selected and excluded at each step of the review. The critical appraisal scores for each of the 95 appraised studies are outlined in Data S3 (Skill Mix OnlineSUPP_C1‐4_CritAppraisals.pdf).

Skill mix systematic review prisma flow diagram [Colour figure can be viewed at http://www.wileyonlinelibrary.com ]

2.6. Data abstraction

One author conducted the initial extraction of the data from the selected studies and a second author checked for accuracy. It was planned that disagreements would be resolved via discussion between the two authors; if no agreement was reached, a third author would be consulted. Information was extracted from the included studies about country, population, setting and sample size, aim, study design, number of participants, type of skill mix variable measured, patient outcomes and general findings. The included articles were treated as equal in this study and are summarized in Data S4 (Skill Mix OnlineSUPP_D_IncStudyCharact.pdf) and include estimates of precision for each outcome. An abbreviated summary is reported in Table ​ Table2 2 .

Included study characteristics (a summary)

Abbreviations: ACS, acute coronary syndrome; ADE, adverse drug event; AIN, Assistants in Nursing; b/w, between; CHF, congestive heart failure; CNS, Clinical Nurse Specialist; COPD, chronic obstructive pulmonary disease; ED, Emergency department; EN, Enrolled Nurse; FTE, full time equivalent; FTR, Failure to rescue; HAPU, Hospital acquired pressure ulcers; HCSW, Healthcare Support Workers; HPPD, hours per patient day; Hr, hours; ICU, Intensive care unit; LOS, Length of stay; LPN, Licensed practical nurse; LVN, licensed vocational nurse; LWBS, left without being seen; Mgt, management; MI, myocardial infraction; N , number; NA, Nursing assistant; NDNQI, National Database of Nursing Quality Indicators; Nsg, nursing; NSO, nurse sensitive outcomes; OR, odds ratio; PUFRQCI, PU and Fall Rate Quality Composite Index; Pt, patient; Pnem, pneumonia; Post op, Post‐operative; PU, pressure ulcer; RPN, registered practical nurse; RN, Registered Nurse; SM, skill mix; TEN, Trainee Enrolled Nurses; UAP, unlicensed assistive personnel; UAPU, Unit acquired pressure ulcer; UTI, urinary tract infection; VA, Veteran Affairs; %, Percent; ↑, Increase/higher; ↓, Decrease/lower; −ve, negative; +ve, positive.

2.7. Synthesis

The included studies showed great variation as to how nurse skill mix was measured and conceptualized. They also used a variety of different study designs, making it not feasible to perform a meta‐analysis. The data were thus summarized narratively, comparing results where appropriate.

3.1. Settings

Most studies were completed in general acute care settings, including general medical, general surgical, combined medical/surgical, step‐down, telemetry units, and emergency departments in public hospitals. One study was conducted in a non‐federal hospital and four in veteran affairs or military hospital settings. Two studies were conducted in a respiratory care centre and trauma centre, respectively. Thirty‐six studies were assessed at unit level, 23 at hospital‐level, and four at shift level. Most studies were conducted in North America—40 in the USA and five in Canada. Another five studies were conducted in Australia, three in Taiwan and South Korea, one in New Zealand, Italy, UK, Germany and Finland, and two studies were conducted in Europe involving multiple countries.

3.2. Study designs

The studies included in this review were mainly observational studies without a comparison group. Thirty‐eight studies fell into this category. Another 14 studies were cross‐sectional, seven used a cohort study design with a comparison group, one was a quasi‐experimental design, one was a randomized controlled trial, one was descriptive correlations, and one was a two‐phase measure development study. The study designs of the included papers were generally low. There was one randomized control trial and two pre‐test–posttest quasi‐experimental studies. The remaining studies were either an observational study, analytic design or an observational descriptive study.

3.3. Nursing skill mix variables

The included studies used a variety of methods to define and assess skill mix. The most frequently used variable to measure nurse skill mix was ‘ percentage of nursing hours provided by RNs ’. Nineteen studies used this measure, defining skill mix either as ‘RN hours provided per day’, ‘RN hours provided per patient day’, ‘RN hours provided per shift, ‘number of productive hours worked by RNs’, or ‘RN hours provided per inpatient earned hours’. The ‘ percentage of RNs on staff ’ was also a commonly used way to define skill mix (16 studies). Percentage of RNs on staff was defined as ‘RN full‐time equivalent (FTE)’, ‘percentage of total RN nursing full‐time positions’, ‘RNs on the unit’, ‘percentage of professional nurses’, ‘proportion of licensed nurses (RN & Licensed practicing nurses (LPN))’, ‘percentage of RN on staff with direct caring responsibilities’ and ‘ratio of FTE RNs to average daily census’. Additional skill mix variables used for analysis included ‘nursing skill mix excluding assistants in nursing (AINs) (compared with skill mix including AINs)’, ‘the number of licensed nurses (RNs & LPNs) per 1,000 patient days’, ‘proportion of all licensed nurses (RN or LPN)’, ‘percentage of nurses with a minimum of three years training’, ‘proportion of regulated workers’, ‘mandated nurse ratios’, ‘triage systems using nurses only (vs. Unlicensed assistive personnel (UAPs) only)’ and ‘skill mix on the day shift (compared with night shift)’.

3.4. Nursing‐sensitive patient outcomes

Twenty‐six outcomes sensitive to nursing care were identified in the reviewed studies. These indicators were: length of stay, gastric ulcer/gastritis/upper gastrointestinal bleeding, acute myocardial infarction, restraint use, failure‐to‐rescue, pneumonia, sepsis, urinary tract infections, mortality, pressure injury, infections (excluding urinary tract infections), shock/cardiac arrest/heart failure, falls and falls injury, deep vein thrombosis, central nervous system complications, pulmonary failure or pulmonary embolism, medication error, physiological/metabolic derangement, pain control, ventilator weaning, patient wait time/leaving without being seen, quality of care, 30‐day readmission, postoperative respiratory failure, unplanned endotracheal tube extubation, and hypoglycaemia.

Infections (including wound infections, central line‐associated bloodstream infections, respiratory tract infections and mediastinitis) were grouped together for the purpose of data synthesis, with the exception of urinary tract infections, which were examined separately as they can account for over 30% of healthcare‐associated infections (Gardner, Mitchell, Beckingham, & Fasugba, 2014 ). Mortality and 30‐day mortality were also grouped together as they were often assessed simultaneously. In addition, ‘care processes relating to acute myocardial infarction’ were grouped with ‘acute myocardial infarction’ and ‘heart failure’ and ‘heart failure measures associated with care’ with ‘shock & cardiac arrest’.

3.5. Identified associations

Due to the heterogeneity with regard to significance and directionality of findings, patient outcomes were considered sensitive to nurse skill mix if they fulfilled the following criteria: of those studies with significant findings, three quarters or more of the studies for each patient outcome found an inverse significant relationship, that is, higher skill mix was associated with fewer adverse patient outcomes (Table ​ (Table3). 3 ). Non‐significant findings were also reported.

Relationship between nursing skill mix and patient outcomes

Based on these guidelines, 12 outcomes met the criteria for a higher skill mix associated with a decrease in adverse outcomes. These outcomes were: length of stay, ulcer, gastritis and upper gastrointestinal bleeds, acute myocardial infarction, restraint use, failure‐to‐rescue, pneumonia, sepsis, urinary tract infection, mortality/30‐day mortality, pressure injury, infections (excluding urinary tract infections) and shock/cardiac arrest/heart failure.

Thirteen studies examined skill mix and length of stay, six of which did not report a statistically significant relationship. Of the seven reporting significant results, all found a higher proportion of skill mix was associated with lower length of stay. Six studies examined skill mix and gastritis & upper gastrointestinal bleeds of which one did not report significant results. Of the five significant results, all found an increase in skill mix resulted is a decrease in gastritis & upper gastrointestinal bleeds. Four studies examined skill mix and acute myocardial infarction, two with significant results both of which found an increase in skill mix was associated with a decrease in acute myocardial infarction. Two studies did not have any significant findings. Two studies examined restraint use and both were significant finding a decrease in restraint use with an increase in skill mix (see Table ​ Table3 3 ).

A further eight patient outcomes were found to have decreased adverse outcomes with a higher skill mix in three quarters or more of the studies with significant findings reviewed for each outcome. Twelve studies examined failure to rescue of which three did not report a statistically significant relationship. Of the nine significant findings, eight of these identified an inverse relationship between higher skill mix and fewer failure to rescue events. Thirteen studies examined pneumonia, six of which did not report a statistically significant relationship. Seven studies had significant findings about pneumonia, with six inversely related to skill mix. Twelve studies examined sepsis and six studies did not report a significant relationship between skill mix and sepsis. Six studies did report significant findings about sepsis and five were inversely related to skill mix. Eighteen studies examined urinary tract infections of which seven did not report significant results. Eleven did report significant findings about urinary tract infections and nine of these were inversely related to skill mix. Seventeen studies examined mortality and six did not report significant findings. Eleven studies had significant findings about mortality and nine of these were inversely related to skill mix. Twenty‐four studies examined pressure injuries, of which fourteen did not report significant findings. Of the 10 studies reporting significant findings, most ( N  = 8) reported that a higher skill mix was associated with fewer pressure injuries. Fifteen studies examined infections (excluding urinary tract infections) of which seven did not report significant findings. Of the eight studies with significant findings, six were inversely related to skill mix. Finally, eights studies examined shock/cardiac arrest/heart failure of which four did not report significant findings. Of the four significant findings, three of these were inversely related to skill mix (see Table ​ Table3). 3 ). However, one study examining heart failure (Newhouse et al., 2013 ) was undertaken using a RTC and found a non‐significant outcome for the effect of skill mix.

The remainder of the patient outcomes examined were inconclusive. Eighteen studies examined the relationship between skill mix and falls and falls injury, seven did not report a significant relationship. Of the eleven studies with significant results, six studies found an inverse relationship and five did not, providing mixed results. Deep vein thrombosis, central nervous system complications, pulmonary failure/pulmonary embolism, medication error, physiologic/metabolic derangement and pain control had fewer studies (2–7) per outcome and in each occasion only two studies were significant. Of these, again in each case, one of the two had an inverse relationship with skill mix, again providing mixed results (see Table ​ Table3). 3 ). In addition, five patient outcomes were excluded as although they technically satisfied the stated inclusion criteria, they were not considered suitable outcomes as only one study respectively investigated these outcomes. These were ventilator weaning, hypoglycaemia, patient wait time, quality of care, and 30‐day readmission (Table ​ (Table3). 3 ). One patient outcome, blood incompatibilities, was not included in the final synthesis, as the study in question did not find any occurrences of the outcome during their data collection (Frith et al., 2010 ).

Table ​ Table3 3 outlines the studies that found a significant relationship between nursing skill mix and those that did not, as well as the proportion of significant studies according to directionality, that is, whether a higher skill mix containing more RNs was associated with an increase or decrease in adverse patient outcomes.

4. DISCUSSION

4.1. summary of evidence.

Out of 26 patient outcomes reported in the 63 studies included in this review, 12 showed an inverse association with skill mix, when assessed using pre‐defined criteria. These outcomes were: length of stay, ulcer, gastritis and upper gastrointestinal bleeds, acute myocardial infarction, restraint use, failure‐to‐rescue, pneumonia, sepsis, urinary tract infection, mortality/30‐day mortality, pressure injury, infections (excluding urinary tract infections) and shock/cardiac arrest/heart failure. There was a reduction in these 12 patient outcomes when a higher nursing skill mix containing more Registered Nurses was present. These results are similar to relationships found in regard to a systematic review of nurse staffing hours (Kane et al., 2007 ) where mortality, hospital acquired pneumonia, unplanned extubation, cardiac arrest in ICUs, risk of failure to rescue in surgical patients and length of stay was shorter in ICUs and in surgical patients. Additionally, a systematic review of acute specialist units found higher staffing levels were associated with reduced mortality, medication errors, ulcers, restraint use, infections, and pneumonia (Driscoll et al., 2017 ).

Any relationship between the remainder of the patient outcomes and skill mix was inconclusive. These outcomes were falls and falls injury, deep vein thrombosis, central nervous system complications, pulmonary failure/pulmonary embolism, medication error (in contrast to Driscoll et al. ( 2017 )), physiologic/metabolic derangement, and pain control. Five patient outcomes were excluded as not being suitable as only one study investigated each outcome. These were ventilator weaning, hypoglycaemia, patient wait time, quality of care and 30‐day readmission.

Since the seminal study conducted in the 1980’s found that interaction and coordination amongst clinicians (medical and nursing staff) reduced patient deaths in ICU settings, nurse staffing has been under the research microscope (Knaus, Draper, Wagner, & Zimmerman, 1986 ). Fundamental to this research is the recognition that Registered Nurses provide a continuous (24 hr per day, 7 days per week) surveillance system for patients. This surveillance system enables early detection and prompt intervention when a patient's condition deteriorates, or health issues emerge (Twigg, Duffield, Thompson, & Rapley, 2010 ). The ability of nurses to undertake this critical role is dependent on the hours of care and the skill mix of those providing that care. It is not surprising, therefore, that this review found 12 outcomes inversely related to skill mix. What remains less well understood is how nurses manage less than desired hours of care or skill mix. Emerging research suggests nurses ration their care when there is insufficient nurses or lower skill mix and prioritize the most urgent or critical aspects of care and some care may be altogether missed (Bail & Grealish, 2016 ). How this might have an impact on patient outcomes over their length of stay is less well understood but this care rationing may provide some explanation as to why skill mix is associated with only some but not all patient outcomes (Griffiths et al., 2018 ).

Many decades that have now passed since the first seminal works in this area, however, the research examining the association between skill mix and patient outcomes remains in an exploratory stage evidenced by the number of outcomes studied and the number of non‐significant results. The research is plagued by methodological issues including; many research studies rely on secondary administrative data to recode patient discharge diagnoses into adverse events, different measures and definitions are still being used for staffing and skill mix variables as well as patient outcomes reducing the comparability of results and data and coding processes often vary or are not transparent (Unruh & Zhang, 2012 ).

4.2. Limitations

The results of this quantitative systematic review should be interpreted with caution. The methodological quality of the included studies is far from ideal, with only very few studies using experimental designs. In this review, only three out of the sixty‐three studies included in the narrative summary employed experimental or quasi‐experimental designs. Many of the studies collected self‐reported data using single measurement methods, for example, self‐reported pain scales (Huston, 2001 ) and self‐reported use of falls risk assessment and policy use (Aydin, Donaldson, Aronow, Fridman, & Brown, 2015 ), raising the potential for common method bias (Podsakoff, MacKenzie, & Podsakoff, 2011 ; Wingate, Sng, & Loprinzi, 2018 ). Self‐reported studies may inflate, deflate or have no effect on the relationships being studied due to common method bias (Podsakoff et al., 2011 ). Hence, these results may have over‐ or underestimated the effect of skill mix on patient outcomes. Whilst common method bias has been identified as an issue for self‐reported studies, the studies did not discuss how common method bias was addressed. Common method bias can be decreased by the use of different data collection methods, times and locations and checking the wording, number of items and placement of items in data collection tools to maximize motivation and minimize task difficulty to encourage participants to provide accurate results (Podsakoff et al., 2011 ; Wingate et al., 2018 ).

There was also a large degree of variety of heterogeneity in the definitions and of research methods used for determining the effect of skill mix on patient outcomes makes comparison of data difficult. This review was only able to provide a narrative analysis of results as meta‐analysis could not be performed. Consequently, inferences about causal relationships between nurse skill mix and nursing‐sensitive patient outcomes cannot be drawn.

5. CONCLUSION

Of those studies identifying significant results, there were 12 outcomes where three quarters or more of those studies found an inverse association with increases in nurse skill mix: length of stay, ulcer, gastritis and upper gastrointestinal bleeds, acute myocardial infarction, restraint use, failure‐to‐rescue, pneumonia, sepsis, urinary tract infection, mortality/30‐day mortality, pressure injury, infections (excluding urinary tract infections), and shock/cardiac arrest/heart failure. Nevertheless, there was insufficient evidence to draw inferences about causal relationships since research into nursing‐sensitive outcomes continues to suffer from methodological flaws and heterogeneity of results. This has an impact on the synthesis of research findings and recommendations for future research and policy. More experimental or longitudinal study designs are required, which have the potential to establish causal relationships. Currently, a plethora of studies investigating interactions between nurse skill mix and patient outcomes employ observational designs that lack control of basic confounding variables.

Furthermore, future research must not only produce a widely agreed definition of nurse skill mix and its most appropriate form of measurement but must also investigate the existence of an optimal level of RNs in skill mix and explore potential non‐linear relationships between nursing skill mix and patient outcomes. Nonetheless, this systematic review suggests that critical patient outcomes such as mortality, failure‐to‐rescue, and length of stay can be improved with a higher skill mix. Those making staffing decisions cannot ignore this association.

CONFLICT OF INTEREST

No conflict of interest has been declared by the authors.

Supporting information

Twigg DE, Kutzer Y, Jacob E, Seaman K. A quantitative systematic review of the association between nurse skill mix and nursing‐sensitive patient outcomes in the acute care setting . J Adv Nurs . 2019; 75 :3404–3423. 10.1111/jan.14194 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]

• Aiken, L. H. , Clarke, S. P. , & Sloane, D. M. (2002). Hospital staffing, organization and quality of care: Cross‐national findings . Nursing Outlook , 50 ( 5 ), 187–194. 10.1067/mno.2002.126696 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Aiken, L. H. , Sloane, D. M. , Bruyneel, L. , Van den Heede, K. , Griffiths, P. , Busse, R. , … Sermeus, W. (2014). Nurse staffing and education and hospital mortality in nine European countries: A retrospective observational study . Lancet , 383 ( 9931 ), 1824–1830. 10.1016/s0140-6736(13)62631-8 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Aiken, L. H. , Sloane, D. , Griffiths, P. , Rafferty, A. M. , Bruyneel, L. , & McHugh, M. ; (RN Cast Consortium) . (2016). Nursing skill mix in European hospitals: Cross‐sectional study of the association with mortality, patient ratings and quality of care . BMJ Quality & Safety , 26 ( 7 ), 559–568. 10.1136/bmjqs-2016-005567 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Ambrosi, E. , De Togni, S. , Guarnier, A. , Barelli, P. , Zambiasi, P. , Allegrini, E. , … Palese, A. (2017). In‐hospital elderly mortality and associated factors in 12 Italian acute medical units: Findings from an exploratory longitudinal study . Aging Clinical and Experimental Research , 29 ( 3 ), 517–527. 10.1007/s40520-016-0576-8 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Anthony, M. (2008). Relationships among nurse staffing, adherence to practice guidelines and patient outcomes in the treatment of hypoglycemia . Quality Management in Health Care , 17 ( 4 ), 312–319. 10.1097/01.QMH.0000338552.13136.1e [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Aydin, C. , Donaldson, N. , Aronow, H. U. , Fridman, M. , & Brown, D. S. (2015). Improving hospital patient falls: Leveraging staffing characteristics and processes of care . The Journal of Nursing Administration , 45 ( 5 ), 254–262. 10.1097/NNA.0000000000000195 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Bae, S.‐H. , Kelly, M. , Brewer, C. S. , & Spencer, A. (2014). Analysis of nurse staffing and patient outcomes using comprehensive nurse staffing characteristics in acute care nursing units . Journal of Nursing Care Quality , 29 ( 4 ), 318–326. 10.1097/NCQ.0000000000000057 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Bail, K. , & Grealish, L. (2016). 'Failure to Maintain': A theoretical proposition for a new quality indicator of nurse care rationing for complex older people in hospital . International Journal of Nursing Studies , 63 , 146–161. 10.1016/j.ijnurstu.2016.08.001 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Ball, J. E. , Bruyneel, L. , Aiken, L. H. , Sermeus, W. , Sloane, D. M. , Rafferty, A. M. , … Griffiths, P. (2018). Post‐operative mortality, missed care and nurse staffing in nine countries: A cross‐sectional study . International Journal of Nursing Studies , 78 , 10 10.1016/j.ijnurstu.2017.08.004 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Barkell, N. P. , Killinger, K. A. , & Schultz, S. D. (2002). The relationship between nurse staffing models and patient outcomes: A descriptive study . Outcomes Management , 6 ( 1 ), 27–33. [ PubMed ] [ Google Scholar ]
• Berkow, S. , Vonderhaar, K. , Stewart, J. , Virkstis, K. , & Terry, A. (2014). Analyzing staffing trade‐offs on acute care hospital units . Journal of Nursing Administration , 44 ( 10 ), 507–516. 10.1097/NNA.0000000000000109 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Blegen, M. A. , Goode, C. J. , Spetz, J. , Vaughn, T. , & Park, S. H. (2011). Nurse staffing effects on patient outcomes safety‐net and non‐safety‐net hospitals . Medical Care , 49 ( 4 ), 406–414. 10.1097/MLR.0b013e318202e129 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Bolton, L. B. , Aydin, C. E. , Donaldson, N. , Brown, D. S. , Sandhu, M. , Fridman, M. , & Aronow, H. U. (2007). Mandated nurse staffing ratios in California: A comparison of staffing and nursing‐sensitive outcomes pre‐ and postregulation . Policy, Politics & Nursing Practice , 8 ( 4 ), 238–250. 10.1177/1527154407312737 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Boyle, D. K. , Jayawardhana, A. , Burman, M. E. , Dunton, N. E. , Staggs, V. S. , Bergquist‐Beringer, S. , & Gajewski, B. J. (2016). A pressure ulcer and fall rate quality composite index for acute care units: A measure development study . International Journal of Nursing Studies , 63 , 73 10.1016/j.ijnurstu.2016.08.020 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Breckenridge‐Sproat, S. , Johantgen, M. , & Patrician, P. (2012). Influence of unit‐level staffing on medication errors and falls in military hospitals . Western Journal of Nursing Research , 34 ( 4 ), 455–474. 10.1177/0193945911407090 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Brennan, C. W. , Daly, B. J. , & Jones, K. R. (2013). State of the science: The relationship between nurse staffing and patient outcomes . Western Journal of Nursing Research , 35 ( 6 ), 760–794. 10.1177/0193945913476577 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Buchan, J. , & Dal Poz, M. R. (2002). Skill mix in the health care workforce: Reviewing the evidence . Bulletin of the World Health Organisation , 80 , 575–580. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Chang, Y. , & Mark, B. (2011). Effects of learning climate and registered nurse staffing on medication errors . The Journal of Nursing Administration , 41 ( 7–8 Suppl ), S6–S13. 10.1097/NNA.0b013e318221c213 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Cho, S. , Ketefian, S. , Barkauskas, V. H. , & Smith, D. G. (2003). The effects of nurse staffing on adverse events, morbidity, mortality and medical costs . Nursing Research , 52 ( 2 ), 71–79. 10.1097/00006199-200303000-00003 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Choi, J. , & Staggs, V. S. (2014). Comparability of nurse staffing measures in examining the relationship between RN staffing and unit‐acquired pressure ulcers: A unit‐level descriptive, correlational study . International Journal of Nursing Studies , 51 ( 10 ), 1344–1352. 10.1016/j.ijnurstu.2014.02.011 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Choi, J. , & Staggs, V. S. (2014). Comparability of nurse staffing measures in examining the relationship between RN staffing and unit‐acquired pressure ulcers: a unit‐level descriptive, correlational study . International Journal Of Nursing Studies , 51 ( 10 ), 1344–1352. 10.1016/j.ijnurstu.2014.02.011 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• de Cordova, P. B. , Phibbs, C. S. , Schmitt, S. K. , & Stone, P. W. (2014). Night and day in the VA: Associations between night shift staffing, nurse workforce characteristics and length of stay . Research in Nursing & Health , 37 ( 2 ), 90–97. 10.1002/nur.21582 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Donabedian, A. (1966). Evaluating the quality of medical care . The Milbank Memorial Fund Quarterly , 44 ( 3 ), 166–206. 10.2307/3348969 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Donaldson, N. , Bolton, L. B. , Aydin, C. , Brown, D. , Elashoff, J. D. , & Sandhu, M. (2005). Impact of California's licensed nurse‐patient ratios on unit‐level nurse staffing and patient outcomes . Policy, Politics & Nursing Practice , 6 ( 3 ), 198–210. 10.1177/1527154405280107 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Driscoll, A. , Grant, M. J. , Carroll, D. , Dalton, S. , Deaton, C. , Jones, I. , … Astin, F. (2017). The effect of nurse‐to‐patient ratios on nurse‐sensitive patient outcomes in acute specialist units: A systematic review and meta‐analysis . European Journal of Cardiovascular Nursing , 17 ( 1 ), 6–22. 10.1177/1474515117721561 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Duffield, C. , Diers, D. , O'Brien‐Pallas, L. , Aisbett, C. , Roche, M. , King, M. , & Aisbett, K. (2011). Nursing staffing, nursing workload, the work environment and patient outcomes . Applied Nursing Research , 24 ( 4 ), 244–255. 10.1016/j.apnr.2009.12.004 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Esparza, S. J. , Zoller, J. S. , White, A. W. , & Highfield, M. E. (2012). Nurse staffing and skill mix patterns: Are there differences in outcomes? Journal of Healthcare Risk Management , 31 ( 3 ), 14–23. 10.1002/jhrm.20092 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Estabrooks, C. A. , Midodzi, W. K. , Cummings, G. G. , Ricker, K. L. , & Giovannetti, P. (2005). The impact of hospital nursing characteristics on 30‐day mortality . Nursing Research , 54 ( 2 ), 74–84. 10.1097/00006199-200503000-00002 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Francis, K. , Nawafleh, H. , & Chapman, Y. (2005). Spotlight on rural nurses: Implications for a new nursing discipline in Jordan . Online Journal of Rural Nursing and Health Care , 5 ( 2 ), 54–72. [ Google Scholar ]
• Frith, K. H. , Anderson, E. F. , Caspers, B. , Tseng, F. , Sanford, K. , Hoyt, N. G. , & Moore, K. (2010). Effects of nurse staffing on hospital‐acquired conditions and length of stay in community hospitals . Quality Management in Health Care , 19 ( 2 ), 147–155. 10.1097/QMH.0b013e3181dafe3f [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Furukawa, M. F. , Raghu, T. S. , & Shao, B. B. M. (2011). Electronic medical records, nurse staffing and nurse‐sensetive patient outcomes: Evidence from the national database of nursing quality indicators . Medical Care Research and Review , 68 ( 3 ), 311–331. 10.1177/1077558710384877 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gardner, A. , Mitchell, B. , Beckingham, W. , & Fasugba, O. (2014). A point prevalence cross‐sectional study of healthcare‐associated urinary tract infections in six Australian hospitals . British Medical Journal Open , 4 ( 7 ), e005099 10.1136/bmjopen-2014-005099 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Glance, L. G. , Dick, A. W. , Osler, T. M. , Mukamel, D. B. , Li, Y. , & Stone, P. W. (2012). The association between nurse staffing and hospital outcomes in injured patients . BMC Health Services Research , 12 , 247–247. 10.1186/1472-6963-12-247 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Godfrey, C. M. , & Harrison, M. B. (2015). Systematic Review Resource Package (Queen's Joanna Briggs Collaboration Version 4.0) . Retrieved from http://joannabriggs.org/assets/docs/jbc/operations/can-synthesise/CAN_SYNTHSISE_Resource-V4.pdf
• Goode, C. J. , Blegen, M. A. , Park, S. H. , Vaughn, T. , & Spetz, J. (2011). Comparison of patient outcomes in Magnet® and non‐Magnet hospitals . The Journal of Nursing Administration , 41 ( 12 ), 517–523. 10.1097/NNA.0b013e3182378b7c [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Griffiths, P. , Ball, J. , Murrells, T. , Jones, S. , & Rafferty, A. M. (2016). Registered nurse, healthcare support worker, medical staffing levels and mortality in English hospital trusts: A cross‐sectional study . British Medical Journal Open , 6 , e008751 10.1136/bmjopen-2015-008751 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Griffiths, P. , Recio‐Saucedo, A. , Dall'Ora, C. , Briggs, J. , Maruotti, A. , Meredith, P. , … the Missed Care Study, G. (2018). The association between nurse staffing and omissions in nursing care: A systematic review . Journal of Advanced Nursing , 74 ( 7 ), 1474–1487. 10.1111/jan.13564 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hart, P. , & Davis, N. (2011). Effects of nursing care and staff skill mix on patient outcomes within acute care nursing units . Journal of Nursing Care Quality , 26 ( 2 ), 161–168. 10.1097/NCQ.0b013e3181efc9cb [ PubMed ] [ CrossRef ] [ Google Scholar ]
• He, J. , Almenoff, P. L. , Keighley, J. , & Yu‐Fang, L. (2013). Impact of patient‐level risk adjustment on the findings about nurse staffing and 30‐Day mortality in Veterans Affairs acute care hospitals . Nursing Research , 62 ( 4 ), 226–232. 10.1097/NNR.0b013e318295810c [ PubMed ] [ CrossRef ] [ Google Scholar ]
• He, J. , Staggs, V. S. , Bergquist‐Beringer, S. , & Dunton, N. (2016). Nurse staffing and patient outcomes: A longitudinal study on trend and seasonality . BMC Nursing , 15 ( 1 ), 10.1186/s12912-016-0181-3 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hendrix, T. J. , & Foreman, S. E. (2001). Optimal long‐term care nurse‐staffing levels . Nursing Economics , 19 ( 4 ), 164–175. [ Google Scholar ]
• Huang, L.‐C. , Lee, J.‐L. , Liang, Y.‐W. , Hsu, M.‐Y. , Cheng, J.‐F. , & Mei, T.‐T. (2011). The skill mix model: A preliminary study of changing nurse role functions in Taiwan . The Journal of Nursing Research , 19 ( 3 ), 220–229. 10.1097/JNR.0b013e318228cd5d [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Huston, C. J. (2001). Contemporary staffing‐mix changes: The impact on postoperative pain management . Pain Management Nursing , 2 ( 2 ), 65–72. 10.1053/jpmn.2001.23556 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Ibe, T. , Ishizaki, T. , Oku, H. , Ota, K. , Takabatake, Y. , Iseda, A. , … Ueda, A. (2008). Predictors of pressure ulcer and physical restraint prevalence in Japanese acute care units . Japan Journal of Nursing Science , 5 , 91–98. 10.1111/j.1742-7924.2008.00106.x [ CrossRef ] [ Google Scholar ]
• Irvin, C. , Sedlak, E. , Walton, C. , Collier, S. , & Bernhofer, E. I. (2017). Hospital‐acquired pressure injuries: The significance of the advanced practice registered nurse’s role in a community hospital . Journal of the American Association of Nurse Practitioners , 29 , 203–208. 10.1002/2327-6924.12440 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Jacob, E. R. , McKenna, L. , & D'Amore, A. (2015). The changing skill mix in nursing: Considerations for and against different levels of nurse . Journal of Nursing Management , 23 ( 4 ), 421–426. 10.1111/jonm.12162 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Johansen, M. L. , de Cordova, P. B. , Duan, W. , Martinez, M. E. , & Cimiotti, J. P. (2015). The implications of nurse resources on cardiac care in the emergency department . Applied Nursing Research , 28 ( 2 ), 210–212. 10.1016/j.apnr.2014.06.002 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kane, R. L. , Shamliyan, T. A. , Mueller, C. , Duval, S. , & Wilt, T. J. (2007). The association of registered nurse staffing levels and patient outcomes: Systematic review and meta‐analysis . Medical Care , 45 ( 12 ), 1195–1204. 10.1097/MLR.0b013e3181468ca3 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kenny, P. (2001). Maintaing quality care during a nursingshortage using licensed practical nurses in acute care . Journal of Nursing Care Quality , 15 ( 4 ), 60–68. [ PubMed ] [ Google Scholar ]
• Kim, C. G. , & Bae, K. S. (2018). Relationship between nurse staffing level and adult nursing‐sensitive outcomes in tertiary hospitals of Korea: Retrospective observational study . International Journal of Nursing Studies , 80 , 155–164. 10.1016/j.ijnurstu.2018.01.001 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kim, S. J. , Park, E.‐C. , Han, K.‐T. , Kim, S. J. , & Kim, T. H. (2016). Nurse staffing and 30‐day readmission of chronic obstructive pulmonary disease patients: A 10‐year retrospective study of patient hospitalization . Asian Nursing Research , 10 ( 4 ), 283–288. 10.1016/j.anr.2016.09.003 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kim, Y. , Kim, S.‐H. , & Ko, Y. (2016). Effect of nurse staffing variation and hospital resource utilization . Nursing and Health Sciences , 18 ( 4 ), 473–480. 10.1111/nhs.12294 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kim, Y. , & Han, K. (2018). Longitudinal associations of nursing staff turnover with patient outcomes in longterm care hospitals in Korea . J Nurs Management , 26 , 518–524. [ PubMed ] [ Google Scholar ]
• Knaus, W. A. , Draper, E. A. , Wagner, D. P. , & Zimmerman, J. E. (1986). An evaluation of outcome from intensive care in major medical centers . Annals of Internal Medicine , 104 ( 3 ), 410–418. 10.7326/0003-4819-104-3-410 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kutney‐Lee, A. , & Aiken, L. H. (2008). Effect of nurse staffing and education on the outcomes of surgical patients with comorbid serious mental illness . Psychiatric Services (Washington, D. C.) , 59 ( 12 ), 1466–1469. 10.1176/appi.ps.59.12.1466 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Lake, E. T. , Shang, J. , Klaus, S. , & Dunton, N. E. (2010). Patient falls: Association with hospital magnet status and nursing unit staffing . Research in Nursing & Health , 33 ( 5 ), 413–425. 10.1002/nur.20399 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Lankshear, A. J. , Sheldon, T. A. , & Maynard, A. (2005). Nurse staffing and healthcare outcomes: A systematic review of the international research evidence . Advances in Nursing Science , 28 ( 2 ), 163–174. 10.1097/00012272-200504000-00008 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Leary, A. , Cook, R. , Jones, S. , Smith, J. , Gough, M. , Maxwell, E. , … Radford, M. (2016). Mining routinely collected acute data to reveal non‐linear relationships between nurse staffing levels and outcomes . British Medical Journal Open , 6 ( 12 ), e011177 10.1136/bmjopen-2016-011177 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Lee, T. , Yeh, M. , Chen, H. , & Lien, G. (2005). The skill mix practice model for nursing: Measuring outcome . Journal of Advanced Nursing , 51 ( 4 ), 406–413. 10.1111/j.1365-2648.2005.03511.x [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Manojlovich, M. , Sidani, S. , Covell, C. L. , & Antonakos, C. L. (2011). Nurse dose: Linking staffing variables to adverse patient outcomes . Nursing Research , 60 ( 4 ), 214–220. 10.1097/NNR.0b013e31822228dc [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Martsolf, G. R. , Auerbach, D. , Benevent, R. , Stocks, C. , Jiang, H. J. , Pearson, M. L. , … Gibson, T. B. (2014). Examining the value of inpatient nurse staffing: An assessment of quality and patient care costs . Medical Care , 52 ( 11 ), 982–988. 10.1097/MLR.0000000000000248 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• McCloskey, B. A. , & Diers, D. K. (2005). Effects of New Zealand's health reengineering on nursing and patient outcomes . Medical Care , 43 ( 11 ), 1140–1146. 10.1097/01.mlr.0000182549.85761.cd [ PubMed ] [ CrossRef ] [ Google Scholar ]
• McGillis Hall, L. , & Doran, D. (2004). Nurse staffing, care delivery model and patient care quality . Journal of Nursing Care Quality , 19 ( 1 ), 27–33. 10.1097/00001786-200401000-00007 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• McGillis Hall, L. , Doran, D. , Baker, G. R. , Pink, G. H. , Sidani, S. , O'Brien‐Pallas, L. , & Donner, J. (2003). Nurse staffing models as predictors of patient outcomes . Medical Care , 41 ( 9 ), 1096–1109. 10.1097/01.MLR.0000084180.07121.2B [ PubMed ] [ CrossRef ] [ Google Scholar ]
• McGillis Hall, L. , Doran, D. , & Pink, G. H. (2004). Nurse staffing models, nursing hours and patient safety outcomes . Journal of Nursing Administration , 34 ( 1 ), 41–45. 10.1097/00005110-200401000-00009 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Myers, H. , Pugh, J. D. , & Twigg, D. E. (2018). Identifying nurse‐sensitive indicators for stand‐alone high acuity areas: A systematic review . Collegian , 25 ( 4 ), 447–456. 10.1016/j.colegn.2017.10.004 [ CrossRef ] [ Google Scholar ]
• Needleman, J. , Buerhaus, P. , Mattke, S. , Stewart, M. , & Zelevinsky, K. (2002). Nurse‐staffing levels and the quality of care in hospitals . New England Journal of Medicine , 346 ( 22 ), 1715–1722. 10.1056/NEJMsa012247 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Needleman, J. , Buerhaus, P. , Mattke, S. , Stewart, M. , & Zelevinsky, K. (2002). Policy highlights: Nurse staffing and quality of care in hospitals in the United States . New England Journal of Medicine , 3 ( 4 ), 306–308. 10.1177/152715402237442 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Needleman, J. , Buerhaus, P. , Stewart, M. , Zelevinsky, K. , & Mattke, S. (2006). Nurse staffing In hospitals: Is there a business case for quality? Health Affairs , 25 ( 1 ), 204–211. 10.1377/hlthaff.25.1.204 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Newhouse, R. P. , Dennison Himmelfarb, C. , Morlock, L. , Frick, K. D. , Pronovost, P. , & Liang, Y. (2013). A phased cluster‐randomized trial of rural hospitals testing a quality collaborative to improve heart failure care: Organizational context matters . Medical Care , 51 ( 5 ), 396–403. 10.1097/MLR.0b013e318286e32e [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Park, S. H. , Blegen, M. A. , Spetz, J. , Chapman, S. A. , & De Groot, H. (2012). Patient turnover and the relationship between nurse staffing and patient outcomes . Research in Nursing & Health , 35 ( 3 ), 277–288. 10.1002/nur.21474 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Patrician, P. A. , Loan, L. , McCarthy, M. , Fridman, M. , Donaldson, N. , Bingham, M. , & Brosch, L. R. (2011). The association of shift‐level nurse staffing with adverse patient events . The Journal of Nursing Administration , 41 ( 2 ), 64–70. 10.1097/NNA.0b013e31820594bf [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Patrician, P. A. , McCarthy, M. S. , Swiger, P. , Raju, D. , Breckenridge‐Sproat, S. , Su, X. , … Loan, L. A. (2016). Association of temporal variations in staffing with hospital‐acquired pressure injury in military hospitals . Research in Nursing & Health , 40 ( 2 ), 111–119. 10.1002/nur.21781 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Paulson, D. L. (2004). A comparison of wait times and patients leaving without being seen when licensed nurses versus unlicensed assistive personnel perform triage . Journal of Emergency Nursing , 30 ( 4 ), 307–311. 10.1016/j.jen.2004.04.022 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Person, S. D. , Allison, J. J. , Kiefe, C. I. , Weaver, M. T. , Williams, O. D. , Centor, R. M. , & Weissman, N. W. (2004). Nurse staffing and mortality for Medicare patients with acute myocardial infarction . Medical Care , 42 ( 1 ), 4–12. 10.1097/01.mlr.0000102369.67404.b0 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Pitkaaho, T. , Partanen, P. , Miettinen, M. , & Vehvilainen‐Julkunen, K. (2015). Non‐linear relationships between nurse staffing and patients' length of stay in acute care units: Bayesian dependence modelling . Journal of Advanced Nursing , 71 ( 2 ), 458–473. 10.1111/jan.12550 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Podsakoff, P. M. , MacKenzie, S. B. , & Podsakoff, N. P. (2011). Sources of method bias in social science research and recommendations on how to control it . Annual Review of Psychology , 63 ( 1 ), 539–569. 10.1146/annurev-psych-120710-100452 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Potter, P. , Barr, N. , McSweeney, M. , & Sledge, J. (2003). Identifying nurse staffing and patient outcome relationships: A guide for change in care delivery . Nursing Economic$ , 21 ( 4 ), 158–166. [ PubMed ] [ Google Scholar ]
• Robinson, S. , Griffiths, P. , & Maben, J. (2009). Calculating skill mix: Implications for patient outcomes and costs . Nursing Management , 16 ( 8 ), 22–23. 10.7748/nm2009.12.16.8.22.c7393 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Roche, M. , Duffield, C. , Aisbett, C. , Diers, D. , & Stasa, H. (2012). Nursing work directions in Australia: Does evidence drive the policy? Collegian , 19 ( 4 ), 231–238. 10.1016/j.colegn.2012.03.006 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Roche, M. A. , Duffield, C. , Friedman, S. , Dimitrelis, S. , & Rowbotham, S. (2016). Regulated and unregulated nurses in the acute hospital setting: Tasks performed, delayed or not completed . Journal of Clinical Nursing , 25 ( 1–2 ), 153–162. 10.1111/jocn.13118 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Schardt, C. , Adams, M. B. , Owens, T. , Keitz, S. , & Fontelo, P. (2007). Utilization of the PICO framework to improve searching PubMed for clinical questions . BMC Medical Informatics and Decision Making , 7 , 16–16. 10.1186/1472-6947-7-16 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Schneider, P. P. , & Geraedts, M. (2016). Staffing and the incidence of pressure ulcers in German hospitals: A multicenter cross‐sectional study . Nursing and Health Sciences , 18 ( 4 ), 457–464. 10.1111/nhs.12292 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Schreuders, L. W. , Bremner, A. P. , Geelhoed, E. , & Finn, J. (2015). The relationship between nurse staffing and inpatient complications . Journal of Advanced Nursing , 71 ( 4 ), 800–812. 10.1111/jan.12572 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Seago, J. A. , Williamson, A. , & Atwood, C. (2006). Longitudinal analyses of nurse staffing and patient outcomes: More about failure to rescue . The Journal of Nursing Administration , 36 ( 1 ), 13–21. 10.1097/00005110-200601000-00005 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sharma, K. , Hastings, S. E. , Suter, E. , & Bloom, J. (2016). Variability of staffing and staff mix across acute care units in Alberta, Canada . Human Resources for Health , 14 ( 1 ), 74–74. 10.1186/s12960-016-0172-1 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sochalski, J. , Konetzka, R. T. , Zhu, J. , & Volpp, K. (2008). Will mandated minimum nurse staffing ratios lead to better patient outcomes? Medical Care , 46 ( 6 ), 606–613. 10.1097/MLR.0b013e3181648e5c [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sovie, M. D. , & Jawad, A. F. (2001). Hospital restructuring and its impact on outcomes: Nursing staff regulations are premature . Journal of Nursing Administration , 31 ( 12 ), 588–600. 10.1097/00005110-200112000-00010 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Staggs, V. S. , & Dunton, N. (2014). Associations between rates of unassisted inpatient falls and levels of registered and non‐registered nurse staffing . International Journal for Quality in Health Care , 26 ( 1 ), 87–92. intqhc/mzt080 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Staggs, V. S. , Knight, J. E. , & Dunton, N. (2012). Understanding unassisted falls: Effects of nurse staffing level and nursing staff characteristics . Journal of Nursing Care Quality , 27 ( 3 ), 194–199. 10.1097/NCQ.0b013e318241da2d [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Staggs, V. S. , Olds, D. M. , Cramer, E. , & Shorr, R. I. (2016). Nursing skill mix, nurse staffing level and physical restraint use in US hospitals: A longitudinal study . Journal of General Internal Medicine , 32 ( 1 ), 35–41. 10.1007/s11606-016-3830-z [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Tourangeau, A. E. , Giovannetti, P. , Tu, J. V. , & Wood, M. (2002). Nursing‐related determinants of 30‐day mortality for hospitalized patients . The Canadian Journal of Nursing Research , 33 ( 4 ), 71–88. [ PubMed ] [ Google Scholar ]
• Twigg, D. , Duffield, C. , Bremner, A. , Rapley, P. , & Finn, J. (2011). The impact of the nursing hours per patient day (NHPPD) staffing method on patient outcomes: A retrospective analysis of patient and staffing data . International Journal of Nursing Studies , 48 ( 5 ), 540–548. 10.1016/j.ijnurstu.2010.07.013 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Twigg, D. , Duffield, C. , Bremner, A. , Rapley, P. , & Finn, J. (2012). Impact of skill mix variations on patient outcomes following implementation of nursing hours per patient day staffing: A retrospective study . Journal of Advanced Nursing , 68 ( 12 ), 2710–2718. 10.1111/j.1365-2648.2012.05971.x [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Twigg, D. , Duffield, C. , Thompson, P. L. , & Rapley, P. (2010). The impact of nurses on patient morbidity and mortality the need for a policy change in response to the nursing shortage . Australian Health Review , 34 ( 3 ), 312–316. 10.1071/AH08668 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Twigg, D. E. , Gelder, L. , & Myers, H. (2015). The impact of understaffed shifts on nurse‐sensitive outcomes . Journal of Advanced Nursing , 71 ( 7 ), 1564–1572. 10.1111/jan.12616 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Twigg, D. E. , Myers, H. , Duffield, C. , Pugh, J. D. , Gelder, L. , & Roche, M. (2016). The impact of adding assistants in nursing to acute care hospital ward nurse staffing on adverse patient outcomes: An analysis of administrative health data . International Journal of Nursing Studies , 63 , 189–200. 10.1016/j.ijnurstu.2016.09.008 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Tzeng, H.‐M. , Hu, H. M. , & Yin, C.‐Y. (2011). The relationship of the hospital‐acquired injurious fall rates with the quality profile of a hospital's care delivery and nursing staff patterns . Nursing Economic$ , 29 ( 6 ), 299–316. [ PubMed ] [ Google Scholar ]
• Unruh, L. (2003). Licensed nurse staffing and adverse events in hospitals . Medical Care , 41 ( 1 ), 142–152. 10.1097/00005650-200301000-00016 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Unruh, L. , & Zhang, N. J. (2012). Nurse staffing and patient safety in hospitals: New variable and longitudinal approaches . Nursing Research , 61 ( 1 ), 3–12. 10.1097/NNR.0b013e3182358968 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Whitman, G. R. , Kim, Y. , Davidson, L. J. , Wolf, G. A. , & Wang, S. (2002). The impact of staffing on patient outcomes across specialty units . JONA: the Journal of Nursing Administration , 32 ( 12 ), 633–639. 10.1097/00005110-200212000-00008 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Wingate, S. , Sng, E. , & Loprinzi, P. D. (2018). The influence of common method bias on the relationship of the socio‐ecological model in predicting physical activity behavior . Health Promot Perspect , 8 ( 1 ), 41–45. 10.15171/hpp.2018.05 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• World Health Organization . (2000). The World Health Report, 2000—Health systems: Improving performance . Geneva, Switzerland: World Health Organization; Retrieved from https://www.who.int/whr/2000/en/ [ Google Scholar ]
• Yang, K. (2003). Relationships between nurse staffing and patient outcomes . Journal of Nursing Research (Taiwan Nurses Association) , 11 ( 3 ), 149–157. 10.1097/01.JNR.0000347631.87723.de [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Yang, P.‐H. , Hung, C.‐H. , & Chen, Y.‐C. (2015). The impact of three nursing staffing models on nursing outcomes . Journal of Advanced Nursing , 71 ( 8 ), 1847–1856. 10.1111/jan.12643 [ PubMed ] [ CrossRef ] [ Google Scholar ]