medical assistant research paper

• Open access
• Published: 16 May 2022

A qualitative assessment of medical assistant professional aspirations and their alignment with career ladders across three institutions

• Stacie Vilendrer 1 ,
• Alexis Amano 1 ,
• Cati Brown Johnson 1 ,
• Timothy Morrison 2 &
• Steve Asch 1  

BMC Primary Care volume  23 , Article number:  117 ( 2022 ) Cite this article

2332 Accesses

8 Altmetric

Metrics details

Growing demand for medical assistants (MAs) in team-based primary care has led health systems to explore career ladders based on expanded MA responsibilities as a solution to improve MA recruitment and retention. However, the practical implementation of career ladders remains a challenge for many health systems. In this study, we aim to understand MA career aspirations and their alignment with available advancement opportunities.

Semi-structured focus groups were conducted August to December 2019 in primary care clinics based in three health systems in California and Utah. MA perspectives of career aspirations and their alignment with existing career ladders were discussed, recorded, and qualitatively analyzed.

Ten focus groups conducted with 59 participants revealed three major themes: mixed perceptions of expanded MA roles with concern over increased responsibility without commensurate increase in pay; divergent career aspirations among MAs not addressed by existing career ladders; and career ladder implementation challenges including opaque advancement requirements and lack of consistency across practice settings.

MAs held positive perceptions of career ladders in theory, yet recommended a number of improvements to their practical implementation across three institutions including improving clarity and consistency around requirements for advancement and matching compensation to job responsibilities. The emergence of two distinct clusters of MA professional needs and desires suggests an opportunity to further optimize career ladders to provide tailored support to MAs in order to strengthen the healthcare workforce and talent pipeline.

Peer Review reports

Primary care practices have increasingly turned to team-based primary care models in their efforts to efficiently provide high quality care [ 1 , 2 ]. As work processes shift in these multi-disciplinary teams to allow each member to perform “at the top of their license” [ 3 , 4 ], medical assistants (MAs) have seen their responsibilities expand to include panel management, health coaching, scribing, translating, phlebotomy, and other multi-functional roles, which vary by site and state licensing [ 5 , 6 , 7 ]. Demand is skyrocketing for MAs; growth projections exceed the average for all occupations by over four-fold [ 8 ]. Factors contributing to demand include the relative value of MAs in health systems (2019 median salary $34,800 [ 8 ]), short training periods, scope of work flexibility, and contribution to positive patient outcomes [ 5 , 9 ].

Efforts to employ and retain such a valuable workforce are of considerable interest to healthcare organizations, given the shortage of available MAs, annual turnover rates of 20–30%, and replacement costs that reach 40% of MA yearly salary [ 10 , 11 ]. Research around these challenges is limited; lack of career advancement opportunity [ 10 , 12 ] and negative perceptions of organizational culture may contribute to MA turnover [ 13 ].

Given these challenges, many organizations are exploring novel solutions to recruit and retain the MA workforce [ 5 , 14 ] with the goal of ultimately improving patient outcomes and workforce efficiency [ 15 , 16 , 17 , 18 ]. One such solution is the implementation of career ladders— paths of professional advancement that provide employees with greater compensation as they cultivate and demonstrate additional skills and increase job responsibilities [ 15 ]. Formal MA career advancement opportunities have been associated with improved quality of care, teamwork, employee satisfaction and intent to stay with current employer [ 5 , 9 , 19 , 20 ]. An evaluation of 15 case studies in which new MA roles and opportunities for advancement were implemented alongside primary care model redesigns found associated improvements in patient and employee satisfaction, cost reduction, and quality [ 5 ]. Notably, MA advancement opportunities also have equity implications. Both women and racial minorities make up the majority of working MAs [ 21 ]. By improving wage earning and career advancement opportunities, healthcare organizations have the opportunity to address racial and gender equity in the healthcare workforce.

Despite these benefits, health organizations face challenges expanding the MA role and structuring meaningful advancement opportunities [ 7 , 22 ]. Lags in implementation of a career ladder following MA role expansion can lead to MA frustration, particularly as these workers may see their responsibilities, but not pay, increase [ 7 ]. Career ladders often require institutional-level support, given that adjustments to compensation often occur at a system-wide level [ 7 , 23 ]. Variation in MA training as well as in state certification and licensure requirements present further obstacles [ 7 , 16 ]. MA education and training programs range from 6-month certificate programs to two-year associates degree programs, and the curriculum offered often varies between programs. Although no states require MA licensure or professional certification, many require certification in specific practice settings or require job training [ 16 ]. MA certification is offered by a number of professional and certification organizations, but the associated education and training requirements vary [ 19 ].

As healthcare organizations continue to establish and refine career ladders, an understanding of MA career aspirations and how they align with current implementations of career ladders is needed. This assessment aims to fill this gap with a qualitative analysis of MA focus groups discussing career aspirations and career ladders implemented at three institutions.

MA perspectives of career aspirations and existing career ladders within their institutions were assessed through a series of semi-structured focus groups. Implementation outcomes were drawn from the Implementation Outcomes Framework, including acceptability, appropriateness, and perceived effectiveness at improving recruitment and retention [ 24 ].

Sites included primary care clinics in three health systems across urban, suburban and partially rural U.S. geographies (University Healthcare Alliance, Newark, CA; Stanford Health Care, Stanford, CA; Intermountain Healthcare, Salt Lake City, UT). Within each institution, a subset of sites were chosen to represent urban (including suburban) and partial rural settings where available [ 25 ].

MA career ladders at the three institutions ranged between 3 and 4 levels, where combination of clinical responsibility and tenure within a site determined a promotion. Administrative contacts reported these were in place for 1 year or more across each institution, though the details varied by clinical site and were often not documented. Two of the organizations were also in the process of revising career ladder details at the time of analysis, thus it was not feasible to capture the details of each of these heterogenous career ladder structures in this analysis. This evaluation was reviewed by the Stanford School of Medicine and Intermountain Healthcare Institutional Review Boards and did not meet the definition of human subjects research; it therefore followed institutional protocols governing quality improvement efforts rather than research (Protocols #51945, #1051215, respectively). As such, early findings were reported back to operational leaders at each institution partway through the analysis to inform ongoing improvement (Fig.  1 ) [ 26 ]. Informed consent was obtained from all participants.

Lightning report on focus group findings

Data collection

From August to December 2019, all MAs within each selected clinic were emailed an invitation to participate in an hour-long focus group by managers who were not present during the conversation. The focus group methodology was chosen to optimize limited research resources, draw out the collective views of the MA population and engage otherwise hesitant participants, particularly given their relative vulnerability as the lowest paid members of the clinical team [ 27 , 28 ]. An unknown minority of MA participants who were invited did not attend the focus group due to clinical care activities. Participants did not receive financial compensation, though lunch was provided. No author practiced within these clinics. Focus groups (led by physician and health services researcher SV) were conducted at clinic sites and consisted of a qualitative semi-structured discussion around MA perceptions of career ladders and financial incentives, the latter of which is the focus of other work [ 14 ]. (See protocol in Additional file  1 : Appendix A.) Conversations were recorded with permission from all participants and transcribed (Rev, Austin, TX). Field notes were also taken by an author (AA) in three focus groups. Data collection continued until thematic saturation was achieved.

Data analysis

Analysis of data collected was rooted in grounded theory [ 29 , 30 , 31 ]. Authors (SV, CBJ, AA) created an initial codebook based on emergent themes from early transcripts and used a constant comparative method [ 30 , 31 , 32 ] to categorize remaining data using software (NVivo 12, Burlington, MA). Authors (SV, CBJ, AA) collectively reviewed a subset of three transcripts to reach consensus on a coding structure before recoding all remaining transcripts in sequence to ensure consistency. Codes were further analyzed by a single author (AA) to identify any potential differences in MA perceptions across clinic organizations and geographies. The consolidated criteria for reporting qualitative research (COREQ) were used to inform reporting of the study findings (Supplemental file  1 ) [ 33 ].

Across the three institutions, ten focus groups were conducted with 4 to 9 participants each for a total of 59 participants. Most MA participants (78.0%) worked in urban/suburban settings, 44% were 30–39 years of age, 92% identified as women, 37% were white, and 54% were non-Hispanic. Nearly half had worked as an MA for 10+ years (Additional file 1 : Appendix B). The demographic composition of study participants with female sex and non-White individuals predominating was consistent with national and local trends [ 16 , 34 , 35 ]. Findings were consistent across institutions as well as urban versus partial rural areas and are therefore described uniformly.

Our qualitative analysis surfaced three major themes: mixed perceptions of expanded MA roles with primary concern over increased responsibility without commensurate increase in pay; divergent career aspirations among MAs not addressed by existing career ladders; and career ladder implementation challenges including opaque advancement requirements and lack of consistency across practice settings. Underlying each of these themes were feelings of underappreciation for the MA role by healthcare organizations. Beyond these themes, a full accounting of factors reported to influence MAs’ decisions to join and remain within their organization can be found in Additional file 1 : Appendix C.

Mixed perceptions of MA career ladders and expanded roles

MAs overall welcomed the existence of a career ladder that would help them understand steps to gaining skills and increasing professional and economic growth. MAs also reported expanded responsibilities at all career levels when compared with their historical roles. However, this increased responsibility was often not associated with increased compensation, which led to subthemes of burnout, frustration over licensure limitations, and skepticism about the value placed on MA’s by the organization.

Several MAs elucidated the tension of being open to more responsibilities so long as they were associated with increased compensation. One MA shared, “I think it’s [career ladder] a positive thing. Also, if they’re going to pay you more, then it’s a really, really good positive thing” (MA 1, FG 2). Another was frustrated with recent changes to her workload: “Two [years ago the work increased]…My workload’s way different …a lot more computer stuff, reports, calling patients” (MA4, FG7). MAs expressed that this felt unfair: “It’s just discouraging if we’re doing all this work, and we’re not being recognized on our title and on our paycheck” (MA8, FG6). Some expressed a desire for a return to their prior responsibilities, or reported the variety of responsibilities and sheer workload created time pressures that reduced job satisfaction.

At the same time, MAs shared frustration at the limitations of what their licenses or job responsibilities allowed them to do. This sentiment clustered around the lack of upward growth opportunities available as well as limitations in day-to-day activities. One MA expressed dissatisfaction at the loss of her ability to place intravenous lines (IVs) due to changes in institutional protocols. Activities that were valued included patient-facing interaction, minor procedures (e.g. IV placement); less valued were computer work and scheduling. Overall, MAs’ desire for increased patient-facing and procedural responsibilities was uniform and appeared conditional on having enough time during the day to complete such tasks and the recognition of this added value in their paychecks.

Underlying these sentiments was skepticism of the organizational value of MAs. This was another factor that some MAs described as driving their desire to leave a given organization. Even while MAs were reportedly in short supply, they reported hearing the message from administration that they were dispensable:

“MA1: …a lot of people say that they don't feel like they're protected here. Like you could literally get fired for the smallest things.

MA3: …I was always afraid that I was getting fired because of things that were said…And just constantly getting talked to, or at, about certain things and never having that representative for myself in there. It was always my word against the manager's word…

MA2: You feel like management is against you and trying to get rid of you kind of thing. And then when you try to reach out to HR, they kind of give you that whole, ‘It's your manager. I'm going to have his or her's back, not your back, because you're replaceable and management's not replaceable.’” (MA1, MA2, MA3, FG4)

Some MAs described their human resource contact as being largely unhelpful, particularly related to questions of work performance, promotion, or career ladders.

Divergent MA career aspirations: “springboard” vs “career” MAs

MA career aspirations varied considerably and fell in two clusters: “springboard” MAs who pursued their current role as one step along a path to obtain a higher level license in healthcare, and “career” MAs who were not interested in obtaining a higher license in healthcare but rather were interested in growing within their careers as MAs (Table  1 ). Whether a given MA fell into one category or another depend in part on their backgrounds: “…personality-wise, we’re not all the same person. We have huge diversity groups in how you were raised or what your projection is or what you want out of life.” (MA 2, FG 7).

“Springboard” MAs reported a desire to gain experience and save money in order to return to school primarily to become a nurse, though individuals also shared plans to become a physician or health administrator. Understanding their personal interest in healthcare before committing to additional training was felt to be a key reason for choosing the MA role: “Nursing... it’s expensive, and then it’s hard to get into. So, you don’t want to be that committed [before knowing you are ready]... I have friends that went into the medical field, and then after [they] were done or close to being done, they found out that they hate blood” (MA6, FG 6). The cost of making a mistake in investing in one’s career was thought to be high.

Alternately, “career” MAs did not nurture plans to return to school or switch professions. Instead, they expressed general contentment in their field and even described the benefits of being an MA over other healthcare careers:

“MA3:…I wouldn't even want to go to school as an RN... You just don't get that interaction with the patient…they [nurses] have time to go in, start the IV, run the machine, change bags, and then they're gone….I don't want to be that, I want to do patient interactions.

MA4: Our patients know our names.” (MA3, MA4, FG9)

A majority hoped to grow within their existing career and shared a desire to move into administration, teaching, or other leadership opportunities. Rare individuals expressed no desire to move up the career ladder. One attributed this to being late in her career:

“Maybe at age 60, I might want to retire…So, why stress myself out even further along…my mental health is something to consider too. So then, I said, ‘I'd rather leave it for somebody that's younger.’” (MA 3, FG 2)

Career ladders fall short of meeting “springboard” and “career” MA needs

Career ladders fell short when viewed through the lens of diverse MA career aspirations. The “springboard” MAs described above who hope to return to school face challenges obtaining financial resources to pursue this education, often while balancing family responsibilities: “[Returning to school requires] debt, time. Hard especially if you have family.” (MA8, FG6) While MAs in several settings described receiving funds for continuing education for their employer, these were a small portion of what was required for additional training. One MA described a loan-forgiveness program where the health system paid a fraction of her loans in exchange for an agreement to work at the institution following training. This program did not seem to entice the MA to shift her plans.

“Career” MAs who expressed a desire to stay within their given roles and clinics still hoped for increased professional growth opportunities. Many felt this was lacking: “I’m in that mode where I’m struggling…I want to be more but I have to do X, Y, and Z, and leave where I’m currently happy at in order to do that.” (MA1, FG1) Other MAs gave clues as to what might constitute these growth opportunities within their given roles. For example, an MA reported that her friend who worked as an MA at an outside health system was eventually hired into an administrative leaderhip role without having to go back for more schooling. The MAs in this focus group agreed that such a professional growth opportunity would be motivating, though no such opportunity existed within their institution. Another participant identified that taking on a new specialized responsibility such as patient coaching might increase her job satisfaction.

Career ladder implementation challenges across MAs

Other implementation challenges were noted across all MAs, regardless of their career aspirations. While MAs largely reacted positively to career ladders in theory, they desired increased clarity as to the requirements for advancement, consistency of these requirements across practice settings within a given institution, and local advancement opportunities.

MAs uniformly described a lack of clarity regarding career ladder details at all three health systems. The exchange below was typical across focus groups:

“Facilitator: So if you wanted to move up [the career ladder],… what would you have to do?

MA2: I have no idea.

MA4: We have no idea.” (FG3)

This challenge was attributed to a lack of communication from administration, both about the overall system and where individuals fit within that system: “We don’t know what level we’re in.” (MA1, FG 5). Other challenges included inconsistent recognition of responsibilities, inability to advance without re-applying for an open position or specializing, lack of individual career counseling, education funds that were challenging to use in practice, and desire for greater appreciation from local physicians and the health system overall. These sub-themes have been converted to direct and implied recommendations for career ladder improvement (Table  2 ).

Where career ladder knowledge existed, MAs faced other obstacles to advancement, such as the need for self-funded education: “You can become MA3, …but you have to have specific certification and you have to do CME [continuing medical education]. You have to pay for that yourself.” (MA 3, FG 7) In addition, many MAs who wanted to advance up the career ladder reported having to wait until a position of that particular level opened.

Further, MAs felt the career ladder did not acknowledge responsibility differences across clinic sites within the same institution, or differences in individual years of experience and training. Several MAs reported that job responsibilities for the same career level varied between clinics. For example, some entry-level MAs are asked to do front desk, back office, and phlebotomy work while others simply obtain vitals and room patients; MAs reported these differences were not reflected in the career ladder.

Again underscoring these concerns was a sense that MAs were not appreciated for their work. MAs highlighted the need to build this recognition into career ladder and compensation structure: “I think being more appreciated is a huge thing… knowing that I’m making a difference.” (MA5, FG9) These collective challenges made it difficult for MAs to advance within their existing role and clinic.

Well-designed career ladders have the potential to improve job satisfaction, thereby improving recruitment and retention of health workers with downstream benefits on patient care and operational efficiency. We found positive MA perceptions of career ladders in principle, though elements of their practical implementation were reported to need improvement across three institutions. Two disinct groups of MAs emerged with regard to their professional ambitions: “springboard” MAs hoped to advance to higher paying non-MA roles while “career” MAs desired professional and financial growth opportunities within the MA profession. Reported and implied recommendations for career ladder improvement included the need for health systems to provide MAs with clear and transparent requirements to ascend career ladders; consistent recognition of training, experience, and work responsibilities across the organization as demonstrated through career ladders; the ability to advance in place or with increased specialization; career counseling; and streamlined opportunities to use educational funds. The need for transparency and consistency in career ladder implementation is consistent with prior work [ 7 ], though this evaluation further contributes to discussions around structuring opportunities for advancement including continuing education and recognition that MAs may cluster into distinct segments based on their needs and career aspirations.

MAs varied in terms of their professional ambitions, including the degree to which they hoped to grow within their existing role and whether they planned to pursue additional training to move into another profession. Designing career experiences around employee career aspirations, including “grouping employees into clusters based on their wants and needs” has been briefly explored in business literature [ 36 ], yet such programs have yet to be formally explored in healthcare. Diverse MA needs discovered here suggest opportunities to optimize career ladders from the perspective of two distinct groups: “springboard” MAs and “career” MAs.

For “springboard MAs”, these results suggest health systems may benefit from anticipating—and moreover supporting—transitions from MA to other health professions, particularly for individuals who hope to remain within a given medical system. MAs frequently reported considering nursing as the next step in their career, a profession with well-documented worker shortages and high turnover cost [ 37 , 38 , 39 , 40 ]. Supporting these “springboard” MAs in their desire to become fully-trained nurses or other types of healthcare professionals may be a savvy way for health systems to create talent pipelines. We heard a single example in which one health system paid a small amount of tuition for additional education in exchange for an agreement to work after training for a minimum number of years. Such agreements exist in other industries and are increasingly used with physician trainees [ 41 ]; extending an adapted program to other health professions, including MAs, deserves further exploration.

MAs’ varied levels of ambition suggest that at least some turnover should be anticipated. Further study is needed to quantify the impact MA career intentions have on turnover, including the portion of MAs who may be retained or positively directed towards other roles within a given health system. We also note that supporting such MA advancement opportunities—whether within the MA role or in non-MA roles within the same institution—may benefit institutional goals towards diversifying workforce and leadership, as MAs typically come from diverse backgrounds that often closely align with the patient population they serve [ 5 , 34 ].

For the “career” MAs, we heard that opportunities that allow for advancement within their current MA profession may increase job satisfaction and thereby retention with its downstream financial and organizational benefits [ 10 , 11 ]. Literature outside healthcare also suggests that organizations can benefit when promoting from within, given that employees retain institution-specific knowledge that increases productivity [ 42 , 43 ]. We note major barriers to facilitating MA advancement within their current roles include licensing restrictions and common staffing structures in primary care—MA role expansion may mean MAs have taken over the historical positions they might have once stepped up into. Some primary care settings, including those in this analysis, are actively exploring further specializing MA roles based on additional training in mental health, population health management, or value-based care [ 5 , 44 ]. These opportunities may facilitate higher level advancement-in-place opportunities for MAs without requiring years of additional training.

We recognize another tension in that local clinic needs can vary significantly, and each may require different competencies from their MAs (e.g. phlebotomy, population health measures). This goes against MAs’ voiced desire that a career ladder consistently reflect competencies across an organization. Based on our overall findings, is seems that allowing for some local clinic-level flexibility to facilitate advancement-in-place opportunities may outweigh MA desire for career ladder consistency across the organization. Administrators must recognize and balance this tension in their efforts to optimize career ladder design.

Underlying these conversations was the dominant theme of MA role expansion in the last several years. While prior work has largely emphasized the benefits of this transition [ 5 , 15 , 16 ], we were struck by the unfavorable perspectives many MAs held when role expansion was discussed in the context of their career progression and, indirectly, compensation. In particular, MAs seemed to recognize they were providing more value to the health system than before, generally without increased compensation; this manifested in the perception that their organizations did not value them. It appears that career ladders, if implemented effectively, may begin to combat negative MA perceptions of fairness in their workplace, thereby improving “organizational justice” and retention [ 45 , 46 ]. Fortunately, despite these sentiments, early literature based on a subset of the population represented here suggests MAs do not experience significantly elevated rates of burnout [ 13 ], though additional study is needed.

Focus groups within three institutions across two geographies cannot encompass the full range of MA perspectives across the U.S., particularly as licensing laws vary from state to state. This evaluation reflects learnings to inform institutional practices, and extrapolation to outside settings is therefore limited. Future efforts to understand and optimize career ladders may benefit from expanded participation from administrators and MAs from diverse settings. Furthermore, we acknowledge two institutions at the time of interviews were working on career ladder improvements; this period of ongoing change may have reduced overall MA knowledge and satisfaction with the pre-existing programs. Additionally, we are unable to provide specific examples of the career ladders at each institution due to variation between clinic sites and ongoing revisions to their structures; understanding these trends is an area for future research. Our use of focus groups may also have limited certain individual disclosures, though we felt the benefits from a synergistic discussion with multiple voices outweighed that risk. Finally, the focus group structure also prohibited us from making comparisons on MA perceptions between racial/ethnic groups, which may be an important area of future study.

Conclusions

MA roles have undergone significant expansion in recent years, and identifying the right balance between organizational and employee needs is ongoing. Career ladders are perceived favorably by MAs in principle but their practical implementation merits further attention. Segmenting MAs into distinct clusters based on their career aspirations may serve as a useful model to further tailor career ladders to employee needs, though additional evaluation is still needed. Such efforts have the potential to strengthen the healthcare workforce and talent pipeline, with downstream benefits to patient care and operational efficiency.

Availability of data and materials

Original qualitative interview recordings and transcripts will not be shared given possible risk to individual privacy. For questions related to the data, please contact Dr. Stacie Vilendrer at [email protected] .

Abbreviations

Medical assistant

Focus group

Continuing medical educator

Findlay S. Implementing MACRA. Health Policy Briefing. Published March 27, 2017. https://www.rwjf.org/en/library/research/2017/03/implementing-the-medicare-and-chip-reauth-act.html .

Reiss-Brennan B, Brunisholz KD, Dredge C, et al. Association of Integrated Team-Based Care with Health Care Quality, utilization, and cost. JAMA. 2016;316(8):826–34. https://doi.org/10.1001/jama.2016.11232 .

Article   PubMed   Google Scholar  

Bodenheimer T, Laing BY. The Teamlet model of primary care. Ann Fam Med. 2007;5(5):457–61. https://doi.org/10.1370/afm.731 .

Article   PubMed   PubMed Central   Google Scholar  

Anderson P, Halley MD. A new approach to making your doctor-nurse team more productive. Fam Pract Manag. 2008;15(7):35–40.

PubMed   Google Scholar  

Chapman SA, Blash LK. New roles for medical assistants in innovative primary care practices. Health Serv Res. 2017;52(S1):383–406. https://doi.org/10.1111/1475-6773.12602 .

Ferrante JM, Shaw EK, Bayly JE, et al. Barriers and facilitators to expanding roles of medical assistants in patient-centered medical homes (PCMHs). J Am Board Fam Med. 2018;31(2):226–35. https://doi.org/10.3122/jabfm.2018.02.170341 .

Dill J, Morgan JC, Chuang E, Mingo C. Redesigning the Role of Medical Assistants in Primary Care: Challenges and Strategies During Implementation. Med Care Res Rev. Published online August 14, 2019:107755871986914. https://doi.org/10.1177/1077558719869143 .

Bureau of Labor Statistics, US Department of Labor. Medical Assistants.; 2020. Accessed 14 Aug 2020. https://www.bls.gov/ooh/healthcare/medical-assistants.htm .

Nelson K, Pitaro M, Tzellas A, Lum A. Transforming the role of medical assistants in chronic disease management. Health Aff. 2010;29(5):963–5. https://doi.org/10.1377/hlthaff.2010.0129 .

Article   Google Scholar  

Taché S, Chapman SA. Medical assistants in California. San Francisco: University of California; 2004. https://healthforce.ucsf.edu/sites/healthforce.ucsf.edu/files/publication-pdf/7.%202004-05_Medical_Assistants_in_California.pdf . Accessed 14 Aug 2020

Google Scholar  

Friedman JL, Neutze D. The financial cost of medical assistant turnover in an academic family medicine center. J Am Board Fam Med. 2020;33(3):426–30. https://doi.org/10.3122/jabfm.2020.03.190119 .

Taché S, Hill-Sakurai L. Medical assistants: the invisible “glue” of primary health care practices in the United States? J Health Organ Manage. 2010;24(3):288–305. https://doi.org/10.1108/14777261011054626 .

Seay-Morrison TP, Hirabayshi K, Malloy CL, Brown-Johnson C. Factors affecting burnout among medical assistants. J Healthc Manag. 2021;66(2):111–21. https://doi.org/10.1097/JHM-D-19-00265 .

Vilendrer S, Brown-Johnson C, Kling SMR, et al. Financial incentives for medical assistants: a mixed-methods exploration of Bonus structures, motivation, and population health quality measures. Ann Fam Med. 2021;19(5):427–36. https://doi.org/10.1370/afm.2719 .

Bodenheimer T, Willard-Grace R, Ghorob A. Expanding the roles of medical assistants: who does what in primary care? JAMA Intern Med. 2014;174(7):1025–6. https://doi.org/10.1001/jamainternmed.2014.1319 .

Chapman SA, Marks A, Dower C. Positioning Medical Assistants for a Greater Role in the Era of Health Reform. Acad Med. 2015;90(10):1347–52. https://doi.org/10.1097/ACM.0000000000000775 .

Ilbawi NM, Kamieniarz M, Datta A, Ewigman B. Reinventing the medical assistant staffing model at no cost in a large medical group. Ann Fam Med. 2020;18(2):180. https://doi.org/10.1370/afm.2468 .

Shekelle PG, Begashaw M. What are the effects of different team-based primary care structures on the quadruple aim of care? A Rapid Review: Department of Veterans Affairs (US); 2021. http://www.ncbi.nlm.nih.gov/books/NBK568333/ . Accessed 21 Jan 2022

Willard-Grace R, Najmabadi A, Araujo C, et al. “I Don’t See Myself as a Medical Assistant Anymore”: Learning to Become a Health Coach, in our Own Voices. i.e inquiry in education . 4(2).

Dill JS, Morgan JC, Weiner B. Frontline health care workers and perceived career mobility: do high-performance work practices make a difference? Health Care Manag Rev. 2014;39(4):318–28. https://doi.org/10.1097/HMR.0b013e31829fcbfd .

Dill J, Morgan JC, Chuang E. Career ladders for medical assistants in primary care clinics. J Gen Intern Med. 2021;36(11):3423–30. https://doi.org/10.1007/s11606-021-06814-5 .

Gray CP, Harrison MI, Hung D. Medical assistants as flow managers in primary care: challenges and recommendations. J Healthc Manag. 2016;61(3):181–91.

Dill JS, Chuang E, Morgan JC. Healthcare organization–education partnerships and career ladder programs for health care workers. Soc Sci Med. 2014;122:63–71. https://doi.org/10.1016/j.socscimed.2014.10.021 .

Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptualdistinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76. PMID: 20957426; PMCID: PMC3068522.  https://doi.org/10.1007/s10488-010-0319-7 .

United States Census Bureau: Urban and Rural. Published February 24, 2020. https://www.census.gov/programs-surveys/geography/guidance/geo-areas/urban-rural.html . Accessed 15 Jan 2020.

Brown-Johnson C, Safaeinili N, Zionts D, et al. The Stanford lightning report method: a comparison of rapid qualitative synthesis results across four implementation evaluations. Learn Health Syst Published online December 21. 2019. https://doi.org/10.1002/lrh2.10210 .

Kitzinger J. Qualitative research: introducing focus groups. BMJ. 1995;311(7000):299–302. https://doi.org/10.1136/bmj.311.7000.299 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Gill P, Stewart K, Treasure E, Chadwick B. Methods of data collection in qualitative research: interviews and focus groups. Br Dent J. 2008;204(6):291–5. https://doi.org/10.1038/bdj.2008.192 .

Article   CAS   PubMed   Google Scholar  

Glaser BG, Strauss AL. The discovery of grounded theory: strategies for qualitative research, vol. 5. Paperback print. Aldine Transaction; 2010.

Hallberg LRM. The “core category” of grounded theory: making constant comparisons. Int J Qual Stud Health Well Being. 2006;1(3):141–8. https://doi.org/10.1080/17482620600858399 .

Sbaraini A, Carter SM, Evans RW, Blinkhorn A. How to do a grounded theory study: a worked example of a study of dental practices. BMC Med Res Methodol. 2011;11(1):128. https://doi.org/10.1186/1471-2288-11-128 .

Miles MB, Huberman AM, Saldaña J. Qualitative data analysis: a methods sourcebook. 4th ed: SAGE; 2020.

Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57. https://doi.org/10.1093/intqhc/mzm042 .

Bates T, Hailer L, Chapman SA. Diversity in California’s health professions: current status and emerging trends: The Public Health Institute and the UC Berkeley School of Public Health; 2008. https://healthforce.ucsf.edu/sites/healthforce.ucsf.edu/files/publication-pdf/10.%20Diversity-in-Californias-Health-Professions-Current-Status-and-Emerging-Trends.pdf . Accessed 28 Jan 2021

Sex, Race, and Ethnic Diversity of U.S, Health Occupations (2011–2015). U.S. Department of Health and Human Services, Health Resources and Services Administration, National Center for Health Workforce Analysis; 2017.

Yohn DL. Design your employee experience as thoughtfully as you design your customer experience. Harvard Business Review. Published online December 8, 2016. https://hbr.org/2016/12/design-your-employee-experience-as-thoughtfully-as-you-design-your-customer-experience .

Duffield CM, Roche MA, Homer C, Buchan J, Dimitrelis S. A comparative review of nurse turnover rates and costs across countries. J Adv Nurs. 2014;70(12):2703–12. https://doi.org/10.1111/jan.12483 .

Zhang X, Tai D, Pforsich H, Lin VW. United States registered nurse workforce report card and shortage forecast: a revisit. Am J Med Qual. 2018;33(3):229–36. https://doi.org/10.1177/1062860617738328 .

Hayes LJ, O’Brien-Pallas L, Duffield C, et al. Nurse turnover: a literature review – an update. Int J Nurs Stud. 2012;49(7):887–905. https://doi.org/10.1016/j.ijnurstu.2011.10.001 .

Waldman JD, Kelly F, Aurora S, Smith HL. The Shocking Cost of Turnover in Health Care. Health Care Manage Rev. 2004;29(1):2–7. https://doi.org/10.1097/00004010-200401000-00002 .

Scheinman SJ, Ryu J. Why a teaching hospital offers an employment-based tuition waiver program. NEJM Catalyst. Published online June 26, 2019. https://catalyst.nejm.org/doi/full/10.1056/CAT.19.0646?casa_token=_ugA89cfTkgAAAAA:JcOwSI1AYL9wez4fMx0X8J0h_Fk9KEmNol7VLV3WGnREa9e8upBd8HPW20_HBWszDU0lv_jWDYEIVj8 .

Benson A, Rissing BA. Strength from within: internal mobility and the retention of high performers. Organ Sci. 2020;31(6):1475–96. https://doi.org/10.1287/orsc.2020.1362 .

Prendergast C. The role of promotion in inducing specific human capital acquisition. Q J Econ. 1993;108(2):523–34. https://doi.org/10.2307/2118343 .

Djuric Z, Segar M, Orizondo C, et al. Delivery of health coaching by medical assistants in primary care. J Am Board Fam Med. 2017;30(3):362. https://doi.org/10.3122/jabfm.2017.03.160321 .

Mengstie MM. Perceived organizational justice and turnover intention among hospital healthcare workers. BMC Psychol. 2020;8(1):19. https://doi.org/10.1186/s40359-020-0387-8 .

Bond W. Creating incentives for accountability in patient care. AMA J Ethics. 2013;15(6):522–8. https://doi.org/10.1001/virtualmentor.2013.15.6.pfor1-1306 .

Download references

Acknowledgments

The authors would like to acknowledge Dr. Raj Srivastava of Intermountain Healthcare, Catherine Krna of the University Healthcare Alliance and Dr. Bryan Bohman of Stanford School of Medicine for their support in facilitating this evaluation.

This work received indirect support through the Stanford-Intermountain Fellowship in Population Health, Primary Care and Delivery Science.

Author information

Authors and affiliations.

Division of Primary Care and Population Health, Stanford University School of Medicine, 1265 Welch Rd., Mail Code 5475, Stanford, CA, 94305, USA

Stacie Vilendrer, Alexis Amano, Cati Brown Johnson & Steve Asch

Stanford Health Care, 300 Pasteur Drive, Palo Alto, CA, 94304, USA

Timothy Morrison

You can also search for this author in PubMed   Google Scholar

Contributions

Author SV conceived of the concept; SV, SA and CB-J designed the evaluation and focus group protocol. Author SV collected focus group data with support from author TM. Authors SV, AA, CB-J analyzed the qualitative transcripts while author SV led the drafting of the manuscript. All authors contributed to the manuscript content and edits. The authors read and approved the final manuscript.

Corresponding author

Correspondence to Stacie Vilendrer .

Ethics declarations

Ethics approval and consent to participate.

The study was reviewed by the Stanford School of Medicine and Intermountain Healthcare Institutional Review Boards and, as quality improvement, did not meet the definition of human subjects research (Protocols #51945, #1051215, respectively). Informed consent was obtained from all participants. All methods were performed in accordance with the relevant guidelines and regulations.

Consent for publication

Not applicable.

Competing interests

No competing interests are reported.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Vilendrer, S., Amano, A., Johnson, C.B. et al. A qualitative assessment of medical assistant professional aspirations and their alignment with career ladders across three institutions. BMC Prim. Care 23 , 117 (2022). https://doi.org/10.1186/s12875-022-01712-z

Download citation

Received : 03 August 2021

Accepted : 14 April 2022

Published : 16 May 2022

DOI : https://doi.org/10.1186/s12875-022-01712-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Medical assistants
• Career ladders
• Health care workforce
• Primary care
• Team-based care

BMC Primary Care

ISSN: 2731-4553

• General enquiries: [email protected]

•   Library Hours Servicios in Español   Databases   Research Guides   Get Help My Account CCC Home myClackamas
• myClackamas
• Research Guides
• Servicios in Español
• CCC Library
• Health Professions

Medical Assistant

• APA (7th ed.) resources
• Course Reserves (textbooks)
• Article databases and eJournals
• eBook databases
• Streaming video databases
• CCC Library Catalog
• Search tips and strategies

APA 7th edition manual

Apa 7 citation examples, missing elements - apa 7, apa 7 paper formatting basics, apa 7 document templates, more apa 7th ed. resources.

• Job and career resources

This guide will introduce you to APA 7 citations, both for the References page of your paper and in-text citations. It is offered in multiple file formats below. 

• Citation Examples - APA 7 - Word Document
• Citation Examples - APA 7 - PDF

This guide will tell you exactly what to do if your resource is missing a citation element. Can't find the author, publication date, page numbers, or something else? Use this guide to find out what to do! This guide is offered in multiple formats below. ​​​​​​​

• Missing Elements - APA 7 - Word Document
• Missing Elements - APA 7 - PDF
• Typed, double-spaced paragraphs.
• 1" margins on all sides.
• Align text to the left.
• Choose one of these fonts: 11-point Calibri, 11-points Arial, 10-point Lucida Sans Unicode, 12-point Times New Roman, 11-point Georgia, 10-point Computer Modern.
• Include a page header (also known as the "running head") at the top of every page with the page number.
• APA papers are broken up into sections. Check with your instructor for their expectations.
• In general, headings and title are centered.

APA 7th edition recognizes two kinds of paper formats - student papers (undergraduate students) and professional research papers (graduate students and professionals). At Clackamas CC, you will use the student paper formatting conventions.

You don't have to format a paper from scratch! Download this APA-formatted document template as a Word document or Google document. Save it, erase the existing text, and type your text right into the template. Learn how to format a paper in APA format by reading the contents of the template. The References page has been formatted with hanging indents.

• Download & edit: APA Word document template Microsoft Word document template to save a copy of and type into. To edit it, save a copy to your desktop or Clackamas Office 365 account. Includes tips on how to format a paper in APA. Last updated Feb. 2020.
• Download & edit: Pages document template If you need this template in Pages, email [email protected]
• View Only: Sample APA student paper (7th ed.) This sample student paper includes descriptions of indentations, margins, headers, and other formatting conventions (APA, 2020).
• APA Style (APA.org) APA's site answers all the basic questions about APA 7th edition and gives sample "student" and "professional" papers. This will help you with document format, in-text citations, the References list, and various stylistics.
• << Previous: Search tips and strategies
• Next: Job and career resources >>
• Last Updated: Apr 18, 2024 11:39 AM
• URL: https://libguides.clackamas.edu/medical-assistant

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

•  We're Hiring!
•  Help Center

Medical Assistant

• Most Cited Papers
• Most Downloaded Papers
• Newest Papers
• Save to Library
• ECG analysis Follow Following
• Addiction Medicine Follow Following
• Management Information System Follow Following
• Knowledge Translation Follow Following
• Alternative Medicine Follow Following
• Rating Scales Follow Following
• Journalism Follow Following
• New Media Follow Following
• Media Studies Follow Following
• Beletristica Follow Following

Enter the email address you signed up with and we'll email you a reset link.

• Academia.edu Publishing
•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

• Find Books with Library SmartSearch
• Find Articles and eBooks with EBSCOhost
• All Library Resources
• Left Your Textbook At Home?
• APA Citations
• MLA Citations
• Get Research Help
• How-To Videos
• Research Guides
• Connect With A Librarian
• How do I get an ID?
• How do I get materials for classes?
• How do I borrow library items?
• How do I return library items?
• Where is my library account?
• Does the library have my textbooks?
• How do I visit the library space?
• How do I find a quiet place to study?
• Can I reserve a study room?
• Where can I use a desktop computer on campus?
• Where can I print on campus?

Service Alert

Library Services

Medical Assistant Research Guide

• Research Help
• Library SmartSearch
• Databases, Journal Articles & More
• Visible Body: Human Anatomy Atlas
• Reference Books
• Cite Your Sources
• Careers and Licensing Exam Prep

Citation Help

Need apa or mla help.

Select the appropriate Citation Style on the left.

• NoodleTools How To Guide & Video Tutorials NoodleTools is a software that streamlines the paper-writing process. Read this guide and learn more!
• NoodleTools For Faculty Instructions for Faculty to set up their NoodleTools Inbox

Citation Check

• If you'd like someone to look over your citations, the librarian can help. Email your Works Cited or References a few days before it's due to [email protected].
• Your instructor may have very specific citation expectations, so be sure to consult him or her as well!

Need help with your writing?

• Visit the Writing Lab, online or in-person in Library & Tutoring Services (A-1100).
• Writing tutors can help you with citations, grammar, and other writing skills.
• Writing Lab Make an appointment in Accudemia for assistance from our Writing Lab Tutors.

APA stands for American Psychological Association, a professional group that sets guidelines for citing sources in psychology, nursing, and other fields.

The guides below will help you with APA Style.

• Reference Source Examples and References Page Setup (APA 7th ed.) Examples of how to cite books, journal articles, and website articles. Sample References page and in-text citation format.
• APA 7th edition Tips and Notes Overview of major revisions in APA 7th edition
• Running head setup in Microsoft Word Step-by-step instructions on setting up a running head in Microsoft Word

Based on the MLA Handbook, 9th Edition. 

The guides below will help you with MLA Style citations.

• Citing books, articles, and web pages in MLA Style Examples of source citations, sample Works Cited page, and In-Text Citation examples.
• Books (MLA Style) Cite books and book chapters
• Articles from EBSCO Host (MLA Style) Cite magazine, journal, and newspaper articles from the Library's EBSCO Host system
• Web Pages (MLA Style) CIte web pages in MLA citation format
• In-Text Citations (MLA Style) Cite sources within the text of your research paper or essay using parenthetical references

Citing Articles from EBSCO Host and Online

• Use automatically-generated citations : Article database systems like EBSCO Host generate sample citations for their articles .
• Check your automatic citations : Automatically-generated citations often contain errors. Be sure to check them for accuracy using either your student writer's handbook, or an online tool like Purdue University Online Writing Lab (OWL) .  
• MLA Style Citation Guide (Purdue OWL)
• APA Style Citation Guide (Purdue OWL)
• << Previous: Books
• Next: Careers and Licensing Exam Prep >>
• Last Updated: Apr 12, 2024 9:03 AM
• URL: https://guides.kish.edu/medicalassistant

A Medical Diagnostic Assistant Based on LLM

• Conference paper
• First Online: 20 March 2024
• Cite this conference paper

• Chengyan Wu 16 ,
• Zehong Lin 16 ,
• Wenlong Fang 16 &
• Yuyan Huang 16 , 17  

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 2080))

Included in the following conference series:

• China Health Information Processing Conference

128 Accesses

With the advent of ChatGPT, large language models (LLMs) have received extensive attention because of their excellent instruction comprehension and generation capabilities. However, LLMs are not specifically designed for the healthcare domain and still lack accuracy in answering specialized healthcare-related questions. In this paper, we mainly used some approaches to improve the performance of large language models in the medical domain. First, we analyzed and processed data to ensure high quality and consistency. Second, we used the model’s excellent ability to generate inference process to the training data. Finally, the data with the explanation and inference process, which are helpful in guiding the thinking and improving the inference ability of the model, are used for training. In terms of model training, we used ChatGLM2-6B as the base model, and the large language model was fine-tuned using the QLoRA framework. To guide the model to generate compliant outputs better, we also explored and carefully constructed appropriate prompts. Overall, our approachs enable the model to achieve the F1 value of 0.433 in this task.

C. Wu, Z. Lin and W. Fang—These authors contributed equally to this work.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Zhang, H., et al.: HuatuoGPT, towards taming language model to be a doctor. arXiv preprint arXiv:2305.15075 (2023)

Xiong, H., et al.: DoctorGLM: fine-tuning your Chinese doctor is not a herculean task. arXiv preprint arXiv:2304.01097 (2023)

Brown, T., et al.: Language models are few-shot learners. In: Advances in Neural Information Processing Systems, vol. 33, pp. 1877–1901 (2020)

Google Scholar  

Touvron, H., et al.: LLaMA: open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)

Chowdhery, A., et al.: PaLM: scaling language modeling with pathways. arXiv preprint arXiv:2204.02311 (2022)

Du, Z., et al.: GLM: general language model pretraining with autoregressive blank infilling. arXiv preprint arXiv:2103.10360 (2021)

Zeng, A., et al.: GLM-130B: an open bilingual pre-trained model. arXiv preprint arXiv:2210.02414 (2022)

Singhal, K., et al.: Towards expert-level medical question answering with large language models. arXiv preprint arXiv:2305.09617 (2023)

Li, Y., Li, Z., Zhang, K., Dan, R., Zhang, Y.: ChatDoctor: a medical chat model fine-tuned on llama model using medical domain knowledge. arXiv preprint arXiv:2303.14070 (2023)

Hu, E.J., et al.: LoRA: low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021)

Wang, H., et al.: HuaTuo: tuning llama model with Chinese medical knowledge. arXiv preprint arXiv:2304.06975 (2023)

Pergola, G., Kochkina, E., Gui, L., Liakata, M., He, Y.: Boosting low-resource biomedical QA via entity-aware masking strategies. arXiv preprint arXiv:2102.08366 (2021)

Lee, J., et al.: BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 36 (4), 1234–1240 (2020)

Article   MathSciNet   Google Scholar  

Chen, Z., Li, G., Wan, X.: Align, reason and learn: enhancing medical vision-and-language pre-training with knowledge. In: Proceedings of the 30th ACM International Conference on Multimedia, pp. 5152–5161 (2022)

Pal, A., Umapathi, L.K., Sankarasubbu, M.: MedMCQA: a large-scale multi-subject multi-choice dataset for medical domain question answering. In: Conference on Health, Inference, and Learning, pp. 248–260. PMLR (2022)

Jin, Q., Dhingra, B., Liu, Z., Cohen, W.M., Lu, X.: PubMedQA: a dataset for biomedical research question answering. arXiv preprint arXiv:1909.06146 (2019)

Hendrycks, D., et al.: Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300 (2020)

Wei, J., et al.: Chain-of-thought prompting elicits reasoning in large language models. In: Advances in Neural Information Processing Systems, vol. 35, pp. 24824–24837 (2022)

Zhou, D., et al.: Least-to-most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205.10625 (2022)

Zhang, Z., Zhang, A., Li, M., Smola, A.: Automatic chain of thought prompting in large language models. arXiv preprint arXiv:2210.03493 (2022)

Kojima, T., Gu, S.S., Reid, M., Matsuo, Y., Iwasawa, Y.: Large language models are zero-shot reasoners. In: Advances in Neural Information Processing Systems, vol. 35, pp. 22199–22213 (2022)

Zelikman, E., Wu, Y., Mu, J., Goodman, N.: STaR: bootstrapping reasoning with reasoning. In: Advances in Neural Information Processing Systems, vol. 35, pp. 15476–15488 (2022)

Hu, Z., et al.: LLM-adapters: an adapter family for parameter-efficient fine-tuning of large language models. arXiv preprint arXiv:2304.01933 (2023)

Li, X.L., Liang, P.: Prefix-tuning: optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190 (2021)

Dettmers, T., Pagnoni, A., Holtzman, A., Zettlemoyer, L.: QLORA: efficient finetuning of quantized LLMs. arXiv preprint arXiv:2305.14314 (2023)

Download references

Author information

Authors and affiliations.

School of Electronics and Information Engineering, South China Normal University, Foshan, 528225, China

Chengyan Wu, Zehong Lin, Wenlong Fang & Yuyan Huang

Datastory, Guangzhou, China

Yuyan Huang

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Yuyan Huang .

Editor information

Editors and affiliations.

The University of Texas Health Science Center at Houston, Houston, TX, USA

Harbin Institute of Technology, Shenzhen, China

Qingcai Chen

Dalian University of Technology, Dalian, China

Hongfei Lin

Zhejiang University, Hangzhou, China

Fudan University, Shanghai, China

Buzhou Tang

South China Normal University, Guangzhou, China

Tianyong Hao

Zhengxing Huang

Medical Informatics Center of Peking University, Beijing, China

Takeda Co. Ltd, Shanghai, China

West China Hospital of Sichuan University, Chengdu, China

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Cite this paper.

Wu, C., Lin, Z., Fang, W., Huang, Y. (2024). A Medical Diagnostic Assistant Based on LLM. In: Xu, H., et al. Health Information Processing. Evaluation Track Papers. CHIP 2023. Communications in Computer and Information Science, vol 2080. Springer, Singapore. https://doi.org/10.1007/978-981-97-1717-0_12

Download citation

DOI : https://doi.org/10.1007/978-981-97-1717-0_12

Published : 20 March 2024

Publisher Name : Springer, Singapore

Print ISBN : 978-981-97-1716-3

Online ISBN : 978-981-97-1717-0

eBook Packages : Computer Science Computer Science (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Med Internet Res
• v.23(12); 2021 Dec

Improving User Experience of Virtual Health Assistants: Scoping Review

Rachel g curtis.

1 UniSA Allied Health and Human Performance, Alliance for Research in Exercise, Nutrition and Activity, University of South Australia, Adelaide, Australia

Bethany Bartel

Ty ferguson, henry t blake, celine northcott, rosa virgara, carol a maher, associated data.

Data charting form.

Characteristics of studies included in the scoping review.

Outcome categories.

Virtual assistants can be used to deliver innovative health programs that provide appealing, personalized, and convenient health advice and support at scale and low cost. Design characteristics that influence the look and feel of the virtual assistant, such as visual appearance or language features, may significantly influence users’ experience and engagement with the assistant.

This scoping review aims to provide an overview of the experimental research examining how design characteristics of virtual health assistants affect user experience, summarize research findings of experimental research examining how design characteristics of virtual health assistants affect user experience, and provide recommendations for the design of virtual health assistants if sufficient evidence exists.

We searched 5 electronic databases (Web of Science, MEDLINE, Embase, PsycINFO, and ACM Digital Library) to identify the studies that used an experimental design to compare the effects of design characteristics between 2 or more versions of an interactive virtual health assistant on user experience among adults. Data were synthesized descriptively. Health domains, design characteristics, and outcomes were categorized, and descriptive statistics were used to summarize the body of research. Results for each study were categorized as positive, negative, or no effect, and a matrix of the design characteristics and outcome categories was constructed to summarize the findings.

The database searches identified 6879 articles after the removal of duplicates. We included 48 articles representing 45 unique studies in the review. The most common health domains were mental health and physical activity. Studies most commonly examined design characteristics in the categories of visual design or conversational style and relational behavior and assessed outcomes in the categories of personality, satisfaction, relationship, or use intention. Over half of the design characteristics were examined by only 1 study. Results suggest that empathy and relational behavior and self-disclosure are related to more positive user experience. Results also suggest that if a human-like avatar is used, realistic rendering and medical attire may potentially be related to more positive user experience; however, more research is needed to confirm this.

Conclusions

There is a growing body of scientific evidence examining the impact of virtual health assistants’ design characteristics on user experience. Taken together, data suggest that the look and feel of a virtual health assistant does affect user experience. Virtual health assistants that show empathy, display nonverbal relational behaviors, and disclose personal information about themselves achieve better user experience. At present, the evidence base is broad, and the studies are typically small in scale and highly heterogeneous. Further research, particularly using longitudinal research designs with repeated user interactions, is needed to inform the optimal design of virtual health assistants.

Introduction

Advancements in machine learning and artificial intelligence offer promise for delivering automated, tailored, convenient health assistance with an unprecedented level of sophistication and personalization and are already contributing to the transformation of health care [ 1 ]. Virtual assistants can be broadly defined as digital services designed to simulate human conversation and provide personalized responses based on input from the user. They can be programmed with structured conversations or to answer the user’s questions. Capabilities range from simple menu or multiple choice–based assistants to more sophisticated virtual assistants with natural language processing that recognize free speech or text. At present, virtual assistants are widely deployed in web-based banking and service settings, reducing reliance on staff by being available to answer consumers’ questions about products and services on demand . Virtual assistants are also increasingly being designed for various health applications, such as delivering cognitive behavior therapy for depression and anxiety [ 2 ], improving diet and physical activity [ 3 ], and conducting remote patient monitoring [ 4 ]. Despite the exciting potential for using virtual assistants for health purposes, the use of virtual assistants in health could be ineffective or even have unintended negative consequences if the technology does not meet the user’s needs and preferences.

The user experience of a virtual health assistant can be defined as the user’s perceptions and responses (eg, emotions, beliefs, preferences, and behaviors) that result from its use or anticipated use [ 5 ]. User experience is influenced by a range of factors, including presentation, functionality, and interactive behavior [ 5 ]. It is important to optimize the design of virtual assistants to provide a positive user experience and promote engagement. A growing body of evidence suggests that design characteristics that influence the look and feel of the virtual assistant, such as visual appearance, communication method, and language features, are an important consideration for design, as such design characteristics can significantly influence users’ psychological and emotional responses and engagement with technology-based applications [ 6 , 7 ]. In addition, although some design decisions may not affect the cost (eg, whether an avatar should be male or female), other decisions may have a major impact on the cost of designing a virtual health assistant (eg, whether an avatar should be animated with facial expressions). Understanding how such design characteristics influence user experience will assist in using finite health software development budgets most effectively.

Previous literature has proposed general guidelines for designing voice user interfaces [ 8 ] and accessible conversational user interfaces for different disability groups [ 9 ], as well as virtual assistants for specific purposes such as teaching [ 10 ] and in-vehicle assistance [ 11 ]. Optimal design techniques are likely to depend on the purpose of the virtual assistant [ 12 , 13 ]; therefore, recommendations specifically in the context of health are needed. Although research has examined methods of assessing the usability of virtual assistants in the health domain [ 14 ], clear guidelines on maximizing the user experience of virtual health assistants are lacking.

An important first step toward constructing guidelines for the development of virtual health assistants was achieved by the literature review conducted by ter Stal et al [ 15 ] in 2018, which aimed to identify the researched design characteristics for embodied conversational agents (virtual assistants that have an animated avatar) in health. The review provided a comprehensive overview of the existing literature, with results suggesting that speech and/or textual output and facial and gaze expressions were the most commonly researched design characteristics. The secondary aims of ter Stal et al [ 15 ] were to identify the outcome variables used in the research and the effects of the design characteristics. The authors concluded that, based on the immature body of evidence at the time, there was no consensus on the optimal design characteristics for embodied conversational agents in health. Results highlighted key avenues for future research, including the fact that more research is needed on all design characteristics to advance the field. Notably, the review by ter Stal et al [ 15 ] included studies using any research design and studies where participants viewed stimuli but did not necessarily interact with a virtual assistant.

The evidence base for the use of interactive virtual health assistants is rapidly growing in both size and quality. In particular, experimental research designs with interactive virtual assistants are being reported increasingly, which should provide clearer evidence of the influence of design characteristics on user experience. A scoping review methodology offers an explicit, systematic means to overview this large and diverse body of literature using rigorous methods to minimize bias [ 16 ]. In this study, we seek to undertake the first scoping review of design characteristics of virtual health assistants, with a view to bring together the strongest evidence available regarding the effects of design characteristics on the user experience of interactive virtual health assistants. In particular, the aims of our scoping review are as follows:

• Provide an overview of all the experimental research examining how design characteristics of virtual health assistants affect user experience
• Summarize research findings of experimental research examining how design characteristics of virtual health assistants affect user experience
• Identify whether research supports making recommendations for the design of virtual health assistants

Bringing together the available evidence on how design characteristics affect the user experience of virtual health assistants will assist researchers and software developers in making decisions about the look and feel of their software and developing the most user-friendly and effective virtual health assistants.

This review is reported according to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) checklist [ 17 ].

Eligibility Criteria

Eligibility criteria were designed using the population, intervention, comparator, and outcome framework (population: adults; intervention: virtual health assistant; comparator: design characteristics; and outcome: user experience) [ 18 ]. Original research articles in peer-reviewed journals and full-length conference papers were included.

Studies with adult samples (aged ≥18 years) were included.

Intervention

Studies examining virtual health assistants were included. For this review, we considered virtual health assistants to be any virtual assistant aimed at the health consumer (general population or patient) relating to the prevention, management, or treatment of any physical or mental health condition, as well as clinical research. Virtual health assistants were included if they functioned on any electronic device (eg, smartphone, computer, and headset). Wizard of Oz virtual assistants (where the user believes they are interacting with a computer-automated virtual assistant, but the virtual assistant is operated by a human [ 19 ]) were included.

Studies comparing design characteristics between ≥2 versions of a virtual health assistant were included. For this review, we defined design characteristics as characteristics of the virtual assistant that influence its look and feel without affecting its core content, purpose, or function. Examples of design characteristics include visual cues such as whether the virtual health assistant has an avatar (ie, an image that represents the virtual assistant), language style, and interaction modality (ie, text or speech). Between- and within-subject experimental designs were included.

Studies evaluating user experience outcomes were included. For this review, we defined user experience to include self-reported evaluations of the virtual assistant or the user’s interaction with the virtual assistant that indicated a more positive or negative experience (eg, trustworthiness, likeability, enjoyment, and ease of use), affect, intentions to continue using the virtual assistant, and objective measures of user engagement (eg, frequency, duration, or nature of the interaction with the virtual health assistant). Only quantitative data were included.

Exclusion Criteria

Dissertations, review articles, conference abstracts, and studies with children were excluded. Virtual assistants used for training or educating medical professionals, as well as robots with a physical body, were excluded. Studies were excluded if participants did not interact with the virtual health assistant; that is, they did not provide any input into the system. Studies were also excluded if the virtual health assistant was not the main component of the health program. Studies were excluded if they evaluated only 1 version of a virtual assistant (ie, nonexperimental research design with no comparator) or if they compared a virtual assistant to a human. Dependent variables that were not associated with a more positive or negative user experience—for example, those used as manipulation checks (eg, where participants were asked to confirm whether a realistic-looking assistant was indeed more realistic looking than a cartoon-style assistant)—were excluded.

Information Sources and Search Strategy

A cross-disciplinary search of the literature was conducted on June 4, 2020, and included 5 electronic databases across the fields of health and information technology: Web of Science, MEDLINE, Embase, PsycINFO, and ACM Digital Library. Search terms for virtual assistant AND design characteristics were included in the search strategy ( Table 1 ). Eligibility specifying the virtual assistant related to health , user experience outcomes, and experimental study design was assessed at screening. Searches were limited to the English language with no limit on publication date. Reference lists of the included studies and other key papers in the field were searched to identify further studies (pearling).

Search terms.

Evidence Selection and Data Charting

Search results from each database were imported into EndNote (Clarivate) [ 20 ], in which duplicates were removed. Studies were screened based on title and abstract. Studies that met the eligibility criteria progressed to full-text screening. The full texts of the studies were then screened to determine final eligibility. Articles were screened by 1 of 2 raters. Raters screened a randomly generated selection of 20 articles in duplicate, and the agreement was 100%. A custom form was developed and used for data charting ( Multimedia Appendix 1 ). Extracted data included population, sample size, age, gender, study country, cultural background, health domain, purpose of the virtual assistant, name of the virtual assistant, Wizard of Oz design, device used, animated character, output modality, input modality, whether the interaction was scripted (whether participants were told what to say), duration of interaction, experimental design, and study results. If articles included multiple studies, data extraction was completed only for studies meeting the eligibility criteria. Where multiple eligible studies were included in an article, data were extracted separately. Where relevant outcomes were measured but not compared statistically between experimental conditions, authors were contacted to provide additional information.

Data Synthesis

Study characteristics were compiled for all the studies included in the review. Where a study was reported in multiple articles, articles were compiled as 1 study with a primary reference indicated, as well as an indication of additional references. To facilitate data synthesis across diverse research designs, overarching categories were constructed to describe the health domains, design characteristics, and outcomes. Retrospective thematic analysis was used to identify similar health domains, design characteristics, and outcomes to construct the relevant categories. After data extraction was completed, lists of all reported health domains, design characteristics, and outcomes were compiled. After familiarization with the data, the first author sorted them into similar categories using an inductive approach (ie, directed by the data with no preconceived categories). These categories were reviewed with the senior author, refined, and named.

Data were synthesized descriptively. Descriptive statistics were used to summarize the body of research. A matrix of the design characteristics and outcome categories was constructed to summarize the research findings. Results in the matrix were based on statistical results reported in the articles. Where interactions were examined (eg, in factorial designs or examining interactions with participant characteristics), main effects were included in the matrix. Studies could report results for 1 or multiple outcomes within a particular outcome category. Results were categorized as positive, negative, or no effect. Where studies reported multiple results in a single outcome category, they were categorized as positive if all multiple outcomes showed positive effects, mixed positive if multiple outcomes were reported with both positive and nonsignificant effects, negative if all multiple outcomes showed negative effects, mixed negative if multiple outcomes were reported with both negative and nonsignificant effects, and no effect if multiple outcomes showed no significant effects.

Authors from 2 studies provided additional data on measures that were not compared between experimental groups. Independent sample t tests (2-tailed) were conducted, and the results were included in the matrix. In total, 4 studies did not present a statistical analysis comparing relevant experimental conditions; therefore, these studies are included only in the text description.

The search identified 6879 articles after duplicates were removed. Of the 6879 articles, 6763 (98.31%) were deemed ineligible based on title and abstract screening. We identified 30 additional records through reference lists. In total, 146 articles (116/6879, 1.69% from the database search plus 30 from reference lists) were screened at full text. Of the 146 articles, 98 (67.1%) were deemed ineligible; 81 (55.5%) did not examine an interactive virtual health assistant, 8 (5.5%) did not compare design features between ≥2 virtual health assistants, 4 (2.7%) did not report user experience outcomes, 2 (1.4%) were not adult samples, 1 (0.7%) did not report original research, 1 (0.7%) was not a journal of conference paper, and 1 (0.7%) did not have the virtual health assistant as a main component of the program. Of the 146 articles, a final 48 (32.9%) articles were included in the scoping review ( Figure 1 ). From the 48 articles, 45 unique studies were identified (5 studies were reported in multiple articles, whereas 3 articles contained multiple studies).

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flowchart. VHA: virtual health assistant.

Multimedia Appendix 2 [ 21 - 68 ] provides an overview of the participant characteristics and study designs for all studies included in the review. Table 2 summarizes the characteristics of the body of research. The virtual assistants used in the research were categorized into 8 health domains: physical activity (aimed to increase exercise), nutrition (aimed to improve diet), alcohol consumption (aimed to reduce alcohol consumption), mental health (eg, aimed to improve mood), medical information or treatment (eg, discussed colorectal cancer screening), sexual health (eg, provided advice about sexually transmitted infections), multiple health behaviors (eg, aimed to improve both exercise and diet), and other (eg, aimed to prevent carpal tunnel). A total of 27 design characteristics were examined in the literature. These were categorized into 5 categories: visual design (eg, realism, age, and body shape of an animated avatar), interface design (eg, input modality), conversational style and relational behavior (eg, empathy and relational behavior and personality), combined visual and conversational design (eg, variability in language and background scene assessed simultaneously), and cultural and organizational affiliation (eg, culturally tailored argumentation and appearance; see Table 3 for the full list of design characteristics). We identified 140 outcome variables, which were categorized into nine categories: virtual assistant personality traits (eg, credible and intelligent), relationship (eg, intimacy and relationship closeness), ease of use (eg, cognitive load and ease of use), satisfaction (eg, enjoyment, satisfaction, and usefulness), emotion (eg, positive and negative affect), use intention (eg, intention to keep using the virtual assistant), engagement (eg, interaction duration), and disclosure (eg, self-disclosure detail and intimacy; see Multimedia Appendix 3 for a full list of outcomes by category). Outcome assessment most frequently used Likert scales, with server logs and conversation transcripts used to assess engagement and disclosure.

Summary of study characteristics (N=45).

a N sums to >45 studies and 100% because 2 studies examined design characteristics in multiple categories.

b Includes studies where at least one experimental condition used an animated avatar.

c Includes studies where at least one experimental condition used speech.

Summary of research findings (N=41).

a Results indicated for nonempathetic avatar only (empathetic avatar had additional dialog to the no avatar condition).

b No study examined the combination of design characteristic and outcome.

c Multiple studies were reported in the article with similar results.

d Similar results were additionally reported at a different time point in the study [ 39 ].

e Similar results were additionally reported at a different time point in the study [ 46 ].

f Indicates any effects of personality (no consistent comparator).

Most studies were conducted in the United States, with a greater number of studies conducted during more recent years (ie, between 2017 and 2020). Several authors led multiple studies (ie, Bickmore [ 25 - 27 , 29 , 31 ], Creed [ 34 , 35 ], Olaffsson, [ 53 , 54 ], Ring [ 56 - 58 ], and Zhou [ 67 , 68 ]). Most studies examined conversational style and relational behavior or visual design and assessed outcomes in the categories of personality, satisfaction, relationship, and use intention. Virtual assistants most frequently related to mental health and physical activity. Those addressing multiple health behaviors frequently examined physical health and nutrition together. Most virtual assistants had an animated avatar and used speech output and multiple-choice input. Most virtual assistants were automated (did not use a Wizard of Oz design), and participant input was not scripted. Studies were most frequently conducted with between 21 and 100 participants in a single session using a between-subjects design, where participants were allocated to evaluate 1 version of the virtual assistant. Participants were most frequently from the general population, with a larger proportion of females than males. Studies were most often published in conference proceedings in fields related to interdisciplinary research on intelligent virtual agents and human–computer interactions, with fewer published in health-related fields.

Table 3 summarizes research findings grouped according to the design characteristic examined and the categories of outcomes measured. Where identical outcomes of a study were reported in multiple articles, the primary reference listed in Multimedia Appendix 2 was used. Additional references were used for outcomes that were not reported in the primary study. In total, 4 studies did not present a statistical analysis comparing the relevant experimental conditions; therefore, these studies are not included in Table 3 .

The following paragraphs highlight key results from the studies presented in Table 3 and include a narrative synthesis of studies that were not presented in Table 3 .

Visual Design

Approximately 7% (3/41) of studies examined whether user experience differed using a virtual assistant with an animated avatar compared with using a text- or speech-only virtual assistant with no visual representation [ 48 , 51 , 62 ]. Findings were generally nonsignificant [ 48 , 51 , 62 ], with some mixed negative effects of using an animated avatar [ 48 , 62 ]. An additional study not included in Table 3 concluded that virtual assistants with an animated avatar were preferred over voice-only assistants; however, the analyses included both real and virtual assistants [ 45 ].

Approximately 22% (9/41) of studies examined the appearance of the animated avatar, and 10% (4/41) of studies examined whether user experience differed using a virtual assistant with a more realistic human avatar compared with a more cartoon human avatar [ 57 , 62 , 64 ]. Although some positive and mixed positive effects of using a more realistic avatar were found [ 64 ], more effects were nonsignificant [ 57 , 62 , 64 ], and 1 was negative [ 57 ]. The species of the avatar was examined by 2% (1/41) of studies, which found mixed negative effects of using a human avatar compared with using a robot avatar [ 62 ]. Age was examined by 2% (1/41) of studies, which found mixed positive effects of using an avatar with a younger appearance compared with using one with an older appearance on satisfaction but no significant effects on other outcomes [ 64 ]. Body shape was examined by 2% (1/41) of studies, which found a positive effect of a fat avatar compared with a slim avatar on personality traits but nonsignificant effects on other outcomes [ 63 ]. The familiarity of the avatar was examined by 2% (1/41) of studies, which found mixed negative and nonsignificant effects of using an avatar that looked like a health coach that participants met at the beginning of the session compared with using an unfamiliar avatar [ 64 ]. The avatar’s attire was examined by 2% (1/41) of studies, which found consistently positive effects of medical professional attire compared with casual attire [ 55 ].

The background scene behind the avatar was examined by 2% (1/41) of studies, which found mixed positive effects of representing a medical office compared with representing an empty room on personality but no significant effects on other measured outcomes [ 55 ]. Approximately 7% (3/41) of studies (all reported in 1 paper) examined whether variability in the camera position, distance, and focus was associated with user experience and found mostly nonsignificant effects [ 58 ].

Interface Design

Approximately 7% (3/41) of studies examined the effects of input modality—whether the user communicates using speech, text, or multiple choice—on user experience and found a combination of positive, mixed negative, and nonsignificant effects of speech input compared with other modalities [ 32 , 33 , 52 ]. A menu-based virtual assistant was examined by 1 further study not included in Table 3 , and it concluded that there were no differences in usability between speech and phone key press user input [ 42 ].

How the conversation between the virtual assistant and user was initiated was examined by 2% (1/41) of studies, which found positive and nonsignificant effects of automated motion initiation compared with user initiation [ 56 ]. The type of ringtone used to initiate a conversation with the user was examined by 2% (1/41) of studies, which found positive effects of more polite tones compared with less polite tones [ 25 ].

Conversational Style and Relational Behavior

Approximately 17% (7/41) of studies examined empathy and relational behavior—empathetic verbal feedback and nonverbal behavior such as facial expressions and gestures [ 27 , 31 , 39 , 40 , 48 , 49 , 51 ]. Although some effects were nonsignificant [ 27 , 40 , 48 , 49 , 51 ], more effects were positive or mixed positive, with 71% (5/7) of studies showing at least some positive effect [ 27 , 31 , 39 , 48 , 51 ]. Approximately 7% (3/41) of studies examined emotional expression—the use of facial expression and voice to express emotion—and found some mixed positive effects [ 34 , 43 ] but more nonsignificant effects [ 34 , 35 , 43 ]. Approximately 7% (3/41) of studies examined self-disclosure—whether the virtual assistant tells the user information about themselves—and found mostly positive effects [ 29 , 44 , 47 ]. Approximately 7% (3/41) of studies examined personality [ 36 , 60 , 61 ]. Although some positive and mixed positive effects were found [ 60 , 61 ], most effects were nonsignificant [ 36 , 60 ].

Approximately 5% (2/41) of studies examined conversation memory—whether the virtual assistant remembered information from earlier conversation—and found some mixed positive effects [ 36 ] but mostly nonsignificant effects [ 23 , 36 ]. An additional study not included in Table 3 compared users’ first interactions when the virtual assistant did not recall their previous session and when the virtual assistant did recall their previous session [ 38 ]. The authors concluded that users were more positive when the virtual assistant recalled their session; however, the conversations were less personal.

Humor was examined by 2% (1/41) of studies, which found mostly nonsignificant effects of including humor compared with not including humor [ 36 ]. Using emojis was examined by 2% (1/41) of studies, which found no significant effects of using emojis compared with not using emojis [ 37 ]. Rap was examined by 2% (1/41) of studies, which found a combination of mixed positive, mixed negative, and nonsignificant effects of including rap compared with not including rap [ 53 ]. Allowing participants to control the virtual assistant’s facial and vocal expression was examined by 2% (1/41) of studies, which found nonsignificant and mixed positive effects compared with not allowing such control [ 26 ]. Approximately 2% (1/41) of studies examined constraining users to respond only positively to questions about their confidence and motivation compared with also presenting negative multiple-choice response options [ 54 ]. It found a combination of mixed negative and neutral effects of constraining users to positive responses. A further study not included in Table 3 examined whether user evaluations were more positive for a virtual assistant that changed behavior based on the user’s eye contact compared with a virtual assistant that always appeared attentive or always bored or that changed behavior randomly [ 41 ]. The authors concluded that changing based on the user’s eye contact seemed more normal than changing behavior randomly but did not confirm the hypothesis that changing behavior is more normal than unchanging behavior.

Combined Visual and Conversational Design

Personification—the use of a name, static avatar, and conversational language—was examined by 2% (1/41) of studies, which found negative effects of personification on users’ disclosure [ 59 ]. Variability in dialog structure (the order of the conversation and the utterances used) and background scene was examined by 2% (1/41) of studies, which found consistently positive effects of variability compared with no variability [ 29 ].

Organizational and Cultural Affiliation

Approximately 10% (4/41) of studies examined cultural tailoring—matching the culture of the virtual assistant to that of the user [ 50 , 65 , 67 , 68 ]. Approximately 5% (2/41) of studies examined cultural tailoring of the virtual assistant’s argumentation (eg, discussed culturally relevant topics) [ 50 , 65 ], and 50% (1/2) of those found a positive effect [ 50 ]. Approximately 7% (3/41) of studies examined cultural tailoring of the virtual assistant’s appearance and the household setting and found predominantly nonsignificant effects [ 50 , 65 , 67 ]. Culturally tailored background scene and argumentation combined were examined by 2% (1/41) of studies, which found no significant effects [ 68 ]. The organizational affiliation of the virtual assistant—who the virtual assistant claimed to be and the context provided in the background scene—was examined by 2% (1/41) of studies, which found positive effects of the virtual assistant being a patient assistant compared with the virtual assistant being either a member of the medical team conducting the research or a government employee [ 66 ].

Principal Findings

This study aimed to provide an overview of experimental research examining how design characteristics of virtual health assistants affect user experience. This is a growing area of scientific endeavor with studies, taken together, examining highly diverse health domains, design characteristics, and outcomes. The most common health domains were physical activity and mental health, with relatively few virtual assistants related to specific health conditions. Approximately half of the studies were categorized as examining the design of conversational style and relational behavior, with the most common design characteristic researched being empathy and relational behavior. The most commonly measured outcomes were in the categories of personality traits, satisfaction, relationship, and use intention.

This study also aimed to summarize the research findings of experimental research examining how design characteristics of virtual health assistants affect user experience. Generally, research has been piecemeal, with few design characteristics having a sufficient body of evidence to draw conclusions about their effects on user experience. The 2 design characteristics that defy this are virtual assistants’ empathy and relational behavior and self-disclosure, which have been the focus of a good number of studies. Research suggests that all 3 (ie, empathy, relational behavior, and self-disclosure) are related to more positive user experience. Other design characteristics with emerging levels of evidence are having a more realistic human representation for an avatar and having medical attire for the avatar, both of which may potentially be related to more positive user experience. Finally, evidence to date suggests that using an animated avatar (compared with no avatar) and cultural tailoring may not affect user experience; however, more research is needed to explore these findings.

One of the clearest findings of this study was that the use of empathy and relational behavior in virtual health assistants appears to have positive effects on user experience. Empathy may help to build trust and rapport with the virtual assistant. The finding that empathy was associated with user satisfaction is in line with research indicating a positive association between empathy in real health care providers and patient satisfaction [ 69 , 70 ]. Results were not consistently positive; however, this may be related to differences between the virtual assistants. For example, for the outcome category personality traits, of the 5 studies examining empathy and relational behavior, 3 (60%) studies showing positive effects used animated avatars, including nonverbal relational behaviors [ 31 , 48 , 51 ]. In contrast, 40% (2/5) of studies showing no effects were text-only assistants [ 40 , 49 ]. It may be that users do not expect text-only assistants to show empathy; therefore, the presence or absence of empathy has no impact on the ratings of the virtual assistant. Alternatively, the effects of empathy may be diminished when nonverbal relational behaviors such as expression and gestures are not present.

Research suggests that virtual health assistants that use self-disclosure (ie, provide information about themselves) elicit a more positive user experience. Results were similar whether the autobiographical information was framed as being about the virtual assistant’s experience as a computer agent [ 44 ] or included human experiences that could not actually be true [ 29 , 47 ]. Self-disclosure is important for the formation of relationships [ 71 ], although research suggests that self-disclosure by a real counselor can have either positive or detrimental effects on a client’s perceptions of the counselor [ 72 ]. The finding that users respond positively to the autobiographical stories of a virtual health assistant supports the computers are social actors paradigm, where users display social responses to computers, although they know they are not human [ 73 , 74 ].

Research examining the realism of the animated avatar showed some positive effects; however, more were nonsignificant. The uncanny valley theory suggests that robots that appear almost but not quite human may elicit a negative emotional response and be less likable than those that are clearly nonhuman [ 75 ]. However, in this review, the study that used a photo-realistic representation in the realistic experimental condition [ 64 ] showed positive effects. More research is needed to examine how the realism of the avatar affects the user experience of virtual health assistants.

Results from 1 study suggest that dressing the avatar in medical attire results in a more positive user experience [ 55 ]. Although more research is needed to confirm this finding, this was a large study (n=308) with consistent results across all outcomes measures. Interestingly, the background setting for the avatar (medical office or empty room) had a mixed positive effect on only 1 out of 3 outcomes categories [ 55 ].

Research suggests that including an animated avatar has no effect or, in some cases, a negative effect on user experience. However, upon closer inspection, this may be because of the nature of the avatars used in the research and may also be affected by interactions between the animation and other virtual assistant characteristics. For example, Lisetti et al [ 48 ] showed that an animated avatar with a neutral facial expression and no empathetic dialog led to poorer user experience than a text-only virtual assistant, whereas an expressive and empathetic virtual assistant led to a better user experience than the text-only virtual assistant. Nguyen and Masthoff [ 51 ] reported similar findings; a nonempathetic animated virtual assistant and a nonempathetic text-only virtual assistant led to a similar user experience; however, an empathetic animated virtual assistant led to better user experience than an empathetic text-only virtual assistant. Taken together, it appears that users may expect a virtual assistant with a human-like representation to have empathy and human-like relational behaviors and have a poorer user experience when this expectation is not met.

Overall, the research did not show cultural tailoring to improve the user experience of virtual health assistants. Notably, although 75% (3/4) of studies included participants who were born overseas (in China [ 68 ], India [ 50 ], or a Spanish-speaking Latin-American country [ 65 ]), participants in all the studies lived in the United States. This may suggest that cultural tailoring is not required for different cultures living in the United States who have had exposure to Anglo-American culture, although more research could confirm this finding. Additional research is also needed to determine whether cultural tailoring affects user experience in other cultural contexts.

Strengths and Limitations

This scoping review is the most rigorous attempt at synthesizing the literature regarding the effects of design characteristics on the user experience of virtual health assistants. It followed the PRISMA-ScR guidelines for scoping reviews and searched a large number of databases. It examined a broad range of design characteristics using the highest level of evidence—experimental research using only interactive virtual health assistants where participants were able to input into the system. However, we acknowledge that the use of specific search terms to capture virtual assistants and design characteristics could have omitted some results. It is also possible that other literary sources may have been available in other databases. In addition, qualitative data were excluded. This enabled a structured approach to synthesizing the data based on statistical significance but may have omitted some important views on user experience.

Although the breadth of the review is a major strength, the heterogeneity of the included studies makes it difficult to synthesize and interpret the results. There was considerable heterogeneity in the purpose of the virtual assistants studied. Optimal design techniques may differ among different health domains. For example, although no overall effect of using emojis was found, the difference in ratings of confidence between using text-only and text with emojis depended on whether the virtual assistant was discussing physical or mental well-being [ 37 ]. In addition, some health conditions were not represented in the studies, for example, neurocognitive impairments such as dementia. There was also significant heterogeneity in the outcomes measured. The most commonly measured outcomes were in the categories of personality, satisfaction, relationship, and use intention. Few studies examined the ease of use, engagement, or disclosure. Although interface design may play a key role in determining the ease of use, other design characteristics such as the visual appearance of an avatar may not be expected to affect the ease of use. More research examining how users interact with the virtual assistant (engagement and disclosure), particularly using objective measures, may complement subjective ratings of the virtual assistant and interaction.

An additional limitation of the literature is that some studies combined a set of similar characteristics into 1 condition, making it difficult to ascertain which characteristic might be responsible for the effects on user experience. For example, research on empathy and relational behavior frequently included verbal empathy with nonverbal relational behaviors. In addition, in most studies, participants evaluated the virtual assistant after interacting during a single session. Programs that aim to promote health behavior change or provide support for a health condition are often designed for ongoing use. Additional research should examine how design characteristics affect user experience over time. Most virtual assistants had animated avatars and speech output; however, over half constrained user input to selecting from predefined response options. Constraining user input requires simpler programming and removes the risk of errors occurring when the virtual assistant misinterprets the user’s input or cannot formulate a response to a query that is outside the bounds of its programmed knowledge [ 76 ]. Natural language processing enables users to communicate using unconstrained text or speech and enables more natural user-directed communication. Virtual assistants using natural language processing have been commonly used in health care [ 77 ] and, with rapid advancements in artificial intelligence, are likely to become increasingly sophisticated. More research should examine the design and user experience of these types of virtual health assistants.

Recommendations

Research demonstrates that design characteristics affect the user experience of virtual health assistants; therefore, researchers and software developers should carefully consider the look and feel of a virtual health assistant during development and testing. On the basis of the results of this scoping review, the following recommendations for designing virtual health assistants and advancing the field of research may be useful for health researchers and software developers:

• Design virtual health assistants to express verbal empathy, for example, understanding of the user’s feelings
• Design virtual health assistants to disclose personal information about themselves to the user, for example, information about their past and personal preferences
• Consider designing a human avatar to be more realistic with medical professional attire
• If designing an animated virtual health assistant, it should display nonverbal relational behaviors, for example, emotional facial expressions, gestures, and mutual gaze
• If empathy and relational behaviors are unable to be incorporated, consider that an animated avatar may not be beneficial or cost-effective
• Engage in formative research with the target audience and adopt a user-centered design approach to ensure that the software meets the needs and preferences of the user
• Conduct further systematic research to replicate and extend previous findings, particularly with longitudinal research designs with repeated user interactions, objective engagement outcomes, and virtual assistants with natural language processing capabilities

Virtual health assistants can provide health information and support on demand and may be applied in the future to a wide variety of purposes such as providing public health information, health education, supporting patients with chronic health conditions, and assisting with healthy lifestyle behavior change. This scoping review examined experimental research assessing how design characteristics of virtual health assistants affect user experience. This is a rapidly growing field of research but is difficult to synthesize and interpret because of the heterogeneity of studies. Nonetheless, certain design characteristics have emerged as important for improving user experience. Preliminary recommendations suggest that programming virtual health assistants to show empathy, display nonverbal relational behaviors, and disclose personal information about themselves may result in a more positive user experience. The decision to include an animated avatar should consider whether the avatar can display empathy and nonverbal relational behaviors. Future research is required to improve our understanding of the relationship between design characteristics and user experience of virtual health assistants, particularly with longitudinal research designs with repeated user interactions.

Acknowledgments

CAM is supported by a Medical Research Future Fund Emerging Leader Grant (GNT1193862). HTB is supported by an Australian Government Research Training Program Scholarship. The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Abbreviations

Multimedia appendix 1, multimedia appendix 2, multimedia appendix 3.

Authors' Contributions: RGC and CAM conceived and designed the study. RGC conducted database searches. RGC and BB conducted screening, and RGC, BB, TF, CN, HTB, and RV conducted data extraction. RGC analyzed the data and wrote the manuscript. All authors revised the manuscript and reviewed and approved the final version.

Conflicts of Interest: None declared.

Russia (General)

Western countries approve deal to stop buying nuclear fuel from russia, after russia's invasion of ukraine, swedish utility vattenfall stops deliveries of russian nuclear fuel, russia to enrich reprocessed uranium for edf, russia to supply enriched uranium to argentina, prosecutors probe german utility enbw, russian lobbyist, russian-ukrainian uranium enrichment joint venture, downblending of russian heu for use in u.s. nuclear power plants, decommissioning of seversk conversion plant used for heu downblending now completed, transport company tli pays us$ 2 million to settle charges over bribes in connection with former shipments of downblended heu from russia to the u.s., russia sends last shipment of downblended heu for use in u.s. reactors, downblending program of russian heu for use in u.s. reactors 95 percent complete, russia not planning to extend heu-leu deal, companies amend deal for uranium from dismantled russian nuclear weapons, us, russia, agree on flexible pricing terms for megatons to megawatts program, russia (european part), tvel elektrostal nuclear fuel plant, rosatom shipments of fuel for china's cfr-600 fast-neutron reactor raise concern over new arms race, tvel elektrostal nuclear fuel plant introduces reduction pyrohydrolysis method for uo 2 powder production, german nuclear power plants silently use fuel made from recycled uranium and russian military heu, tvel set to ship fuel pellets to india, capacity increase at tvel elektrostal nuclear fuel plant, glazov conversion plant , udmurtia, russian uranium (hexafluoride) production to be concentrated in seversk (tomsk region), russia (asian part), iaea peer review team unable to confirm long-term safety of deep well injection of radioactive waste at russian nuclear fuel facilities, employee suffers burns in explosion at seversk nuclear fuel plant, russia to start direct supplies of enriched uranium to japan, tvel takes over uranium enrichment company, decommissioning of former krasnoyarsk nuclear fuel plant completed, russia and south africa eye cooperation in conversion and enrichment of uranium, tvel aims to sell nuclear fuel in the us market by 2014, japan to contract russia for enrichment of recycled uranium, kazatomprom to obtain share in russian enrichment plants in exchange for uranium deliveries, seversk conversion plant , tomsk region, seversk conversion plant becomes joint stock company, "public hearing" on new seversk conversion plant project held behind closed doors, public comment invited on environmental impact assessment for new seversk conversion plant project, new seversk plant to replace angarsk conversion plant in second quarter of 2014, new conversion plant project to end obsolete practice of underground disposal of liquid radioactive waste at seversk, angarsk conversion plant , irkutsk region, angarsk conversion plant to be closed in 2016, centrifuge enrichment, seversk enrichment plant (formerly tomsk-7) , tomsk region, in view of russian invasion of ukraine, robin des bois calls for halt of french exports of reprocessed uranium to russia, greenpeace blocks railway track used for transport of reprocessed uranium from france to russia, greenpeace disrupts shipment of reprocessed uranium from france to russia, greenpeace denounces shipments of reprocessed uranium from france to russia, old centrifuges decommissioned at seversk enrichment plant, seversk enrichment plant waste disposal scheme puts drinking water at risk, enrichment of french recycled uranium at seversk enrichment plant, novouralsk enrichment plant (formerly sverdlovsk-44) , sverdlovsk region, one worker killed and more than 100 hospitalized after breach of cylinder holding depleted uranium hexafluoride at novouralsk enrichment plant, kazatomprom sells its 12.5% share in novouralsk enrichment plant to joint venture partner tvel, tvel and kazatomprom sign key documents on uranium enrichment joint project in novouralsk, russia, kazakhstan to kick off uranium enrichment in novouralsk in 2013, zelenogorsk enrichment plant (formerly krasnoyarsk-45) , krasnoyarsk region, replacement of old gas centrifuges continues at zelenogorsk enrichment plant, operational life of zelenogorsk enrichment plant extended to 2048, new generation of gas centrifuges put into operation at zelenogorsk enrichment plant, ice-breaking on river kan with boats and explosions prevents floodings at zelenogorsk enrichment plant, licence renewal for zelenogorsk enrichment plant, capacity increase planned for zelenogorsk enrichment plant, angarsk enrichment plant , irkutsk region, angarsk enrichment plant to resume enrichment of fresh uranium, while continuing re-enrichment of depleted uranium stockholdings, angarsk uranium enrichment plant producing 2,500 t u annually by re-enriching depleted uranium tails, presidential council on human rights concerned about storage of depleted uranium hexafluoride in open air at angarsk, angarsk uranium enrichment plant re-enriching depleted uranium tails, properties and fate of radioactive waste to be removed from storage site at angarsk uranium enrichment plant unclear - hearing reveals; no explanation for presence of artificial nuclides given, activist demands deconversion of depleted uranium of european origin stored in angarsk, demonstration in angarsk against uranium enrichment and storage of urenco's depleted uranium, angarsk enrichment plant wants its premises excised from city territory to avoid legal conflict over depleted uranium hexafluoride (tails) storage, russo-kazakhstani joint-venture enterprise for uranium enrichment, fuel fabrication, novosibirsk nuclear fuel plant, new pwr fuel production line commissioned at novosibirsk nuclear fuel plant, new uo 2 powder production line commissioned at novosibirsk nuclear fuel plant, factories linked to novosibirsk nuclear fuel plant polluting major river in siberia - prosecutors, zelenogorsk mox fuel plant , krasnoyarsk region.