problem solving methodology in public health

• Discussions
• Certificates
• Collab Space
• Course Details
• Announcements

In this course, you will learn about the hypothesis-driven approach to problem-solving. This approach originated from academic research and later adopted in management consulting. The course consists of 4 modules that take you through a step-by-step process of solving problems in an effective and timely manner.

Course information

This course is also available in the following languages:

The hypothesis-driven approach is a problem-solving method that is necessary at WHO because the environment around us is changing rapidly. WHO needs a new way of problem-solving to process large amounts of information from different fields and deliver quick, tailored recommendations to meet the needs of Member States. The hypothesis-driven approach produces solutions quickly with continuous refinement throughout the research process.

What you'll learn

• Define the most important questions to address.
• Break down the question into components and develop an issue tree.
• Develop and validate the hypothesis.
• Synthesize findings and support recommendations by presenting evidence in a structured manner.

Who this course is for

• This course is for everyone. Whether your position is in administrative, operations, or technical area of work, you’re sure to run into problems to solve. Problem-solving is a key skill to continue developing and refining—the hypothesis-driven approach will surely be a great addition in your toolbox!

Course contents

Introduction: hypothesis-driven approach to problem solving:, module 1: identify the question:, module 2: develop & validate hypothesis:, module 3: synthesize findings & make recommendations:, enroll me for this course, certificate requirements.

• Gain a Record of Achievement by earning at least 80% of the maximum number of points from all graded assignments.
• Gain a Confirmation of Participation by completing at least 80% of the course material.

Fresh Perspectives

Want to Solve Problems in Public Health? Here's How

I have many loves in family medicine. I love delivering a newborn directly into a mother's arms. I love excisional biopsies of funny looking moles. I love giving someone hope after a chronic disease diagnosis.  

What I love most, however, is community-based preventive medicine. As such, I wrangled a contract directly out of residency that is 20 percent population medicine. (Lesson No. 1: Ask for what you want. You might actually get it).

As part of this endeavor, I am pursuing a master's degree in public health. I hope to use this training to make connections in the world of public health policy; to learn how to create, implement, message and evaluate programming; and perhaps to eventually break into creation of, or participation in, policy. I will have some required coursework in, for example, biostatistics and epidemiology, but I will also have myriad electives on topics like environmental public health and behavioral economics.

Throughout the course of my program, I hope to distill the most useful-to-the-family-medicine-doc public health pearls from my classes and pass them along. This post is the first in this series. Thus far, I have taken courses on problem-solving in public health and intro to persuasive communication.

The course on problem-solving in public health taught me two things: a remarkably egalitarian way to run a meeting and a systematic approach to solving problems.

The Nominal Group Technique (NGT)

In this setup for running a meeting, start by imagining a group of eight people. During each session, one of them is the moderator, one a notetaker and one a timekeeper. The moderator's job is to decide how long each part of the session ought to take, and the timekeeper's job is to cut people off once time is reached. The notetaker … takes notes.

Each meeting uses the following series of steps, and as participants get used to the process, they get faster and more efficient.

• Clarify the purpose and goals. The moderator reminds everyone about the specific question or questions for the session, reviews time limits for each ensuing step and allows for adjustments on each of these points.
• Brainstorm solutions. Group members brainstorm answers to the session's central question, a step that can take place before the meeting.
• Share ideas in a round robin. Going around in a circle, each person briefly shares one idea, adding more brief ideas -- avoiding duplicates -- when the circle comes back around until time runs out or all ideas have been voiced. In this manner, no one dominates the discussion and everyone is heard.
• Discuss as a group. Here the group focuses on clarifying, not debating. The goal is to add salient details or reasons for a certain suggestion. This is a time to ask questions rather than make arguments. Some time can also be spent discussing criteria for voting in the next step.
• Rank the suggestions. Each group member ranks the options based on the set criteria -- perhaps voting for their three favorites, using two votes however they want, or casting one vote each -- and the group ends up with two or three leading suggestions.
• Wrap up with conclusions and assignments. Participants are assigned roles or tasks to complete before the next meeting, and a new moderator, timekeeper and notetaker are assigned.

The NGT is delightfully efficient and focused. Moreover, it imposes a thoughtful, respectful and inclusive methodology to traditionally explosive or at least controversial topics. By using this technique, you can assure all members of the group that each of their voices will be heard with equal weight, as will also be the case in the problem-solving process below.

The Problem-solving Process

Usually applied to public health problems, this series of steps offers a framework through which one can approach just about any problem that involves groups of people. Whether your problem is developing a group visit program or decreasing smoking in pregnant women, you can approach the problem with success in this way. Notably, this process works well in combination with the NGT.

• Define the problem. A good problem definition has a specific group, timeframe and outcome of interest. For example, the definition could be, "Childhood obesity rates in the United States among school-aged children have been rising since the 1970s."
• Identify indicators of the problem. If your problem is childhood obesity, your direct indicators would be things like body mass index, waist circumference or waist-to-hip ratio. Indirect indicators -- things that give you a clue your endpoint might be happening -- would be rates of childhood hypertension, diabetes or obesity-related sleep apnea. Using the NGT would lead your group to brainstorm as many direct and indirect indicators as possible, then you vote on which ones to track and change.
• Find data for the indicators. Without data, you will have a hard time convincing others to do what you want.
• Identify stakeholders. Find out who cares about the outcome. A meeting held in the NGT style would come out of brainstorming and round robin with a diverse, inclusive and thorough list of potential stakeholders. For childhood obesity, the stakeholders could be parents, students, educators, elected officials, etc. The ranking step would narrow the list to the stakeholders that your group wants to work with.
• Identify key determinants. These are the things that might make the outcome of interest more or less likely. For childhood obesity, these factors might be diet, exercise, dangerous neighborhoods that prevent exercise, food deserts, genes, obesity in parents, television watching, school lunches and poverty.
• Identify intervention strategies. Here is when you brainstorm actions to change the outcome. Some of the group's ideas might be school lunch programs, educational programs for parents, active recess or adjusting food aid programs. All ideas are welcome for discussion and ranking. At the end of the meeting, your group will have decided on an intervention strategy to pursue.
• Identify implementation strategies. It is all well and good to have an intervention, but the next step is to figure out how to get it off the ground. You need to use all the resources you have -- friends in high places, friends in low places, grants, national organizations, local fundraising, city council meetings and more.
• Evaluate. All good interventions need to be evaluated. Be sure to figure out how to do so. Is it working? Is it costing too much? Does it have any unintended benefits or consequences?

I already have used each of these techniques to great effect. By addressing problems in this step-by-step fashion, I find myself suddenly more organized, and you know what that means: more time for more projects!

Just kidding. I get to read books for fun these days and go on long runs. It is amazing.

Stewart Decker, M.D., is a family physician practicing in southern Oregon. He focuses on the intersection of public health and primary care. You can follow him on Twitter at @drstewartdecker.

The opinions and views expressed here are those of the authors and do not necessarily represent or reflect the opinions and views of the American Academy of Family Physicians. This blog is not intended to provide medical, financial, or legal advice. All comments are moderated and will be removed if they violate our  Terms of Use .

Copyright © 2024 American Academy of Family Physicians. All Rights Reserved.

• Table of Contents

TOOLS AND TECHNIQUES

Tools for implementing an evidence-based approach in public health practice, navigate this article, introduction, the need for evidence-based public health, training programs, key elements, putting evidence to work, acknowledgments, author information, julie a. jacobs, mph; ellen jones, phd; barbara a. gabella, msph; bonnie spring, phd; ross c. brownson, phd.

Suggested citation for this article: Jacobs JA, Jones E, Gabella BA, Spring B, Brownson RC. Tools for Implementing an Evidence-Based Approach in Public Health Practice. Prev Chronic Dis 2012;9:110324. DOI: http://dx.doi.org/10.5888/pcd9.110324 .

PEER REVIEWED

Increasing disease rates, limited funding, and the ever-growing scientific basis for intervention demand the use of proven strategies to improve population health. Public health practitioners must be ready to implement an evidence-based approach in their work to meet health goals and sustain necessary resources. We researched easily accessible and time-efficient tools for implementing an evidence-based public health (EBPH) approach to improve population health. Several tools have been developed to meet EBPH needs, including free online resources in the following topic areas: training and planning tools, US health surveillance, policy tracking and surveillance, systematic reviews and evidence-based guidelines, economic evaluation, and gray literature. Key elements of EBPH are engaging the community in assessment and decision making; using data and information systems systematically; making decisions on the basis of the best available peer-reviewed evidence (both quantitative and qualitative); applying program-planning frameworks (often based in health-behavior theory); conducting sound evaluation; and disseminating what is learned.

Top of Page

An ever-expanding evidence base, detailing programs and policies that have been scientifically evaluated and proven to work, is available to public health practitioners. The practice of evidence-based public health (EBPH) is an integration of science-based interventions with community preferences for improving population health (1). The concept of EBPH evolved at the same time as discourse on evidence-based practice in the disciplines of medicine, nursing, psychology, and social work. Scholars in these related fields seem to agree that the evidence-based decision-making process integrates 1) best available research evidence, 2) practitioner expertise and other available resources, and 3) the characteristics, needs, values, and preferences of those who will be affected by the intervention (Figure) (2-5).

Figure. Domains that influence evidence-based decision making. Source: Satterfield JM et al (2). [A text description of this figure is also available.]

Public health decision making is a complicated process because of complex inputs and group decision making. Public health evidence often derives from cross-sectional studies and quasi-experimental studies, rather than the so-called “gold standard” of randomized controlled trials often used in clinical medicine. Study designs in public health sometimes lack a comparison group, and the interpretation of study results may have to account for multiple caveats. Public health interventions are seldom a single intervention and often involve large-scale environmental or policy changes that address the needs and balance the preferences of large, often diverse, groups of people.

The formal training of the public health workforce varies more than training in medicine or other clinical disciplines (6). Fewer than half of public health workers have formal training in a public health discipline such as epidemiology or health education (7). No single credential or license certifies a public health practitioner, although voluntary credentialing has begun through the National Board of Public Health Examiners (6). The multidisciplinary approach of public health is often a critical aspect of its successes, but this high level of heterogeneity also means that multiple perspectives must be considered in the decision-making process.

Despite the benefits and efficiencies associated with evidence-based programs or policies, many public health interventions are implemented on the basis of political or media pressures, anecdotal evidence, or “the way it’s always been done” (8,9). Barriers such as lack of funding, skilled personnel, incentives, and time, along with limited buy-in from leadership and elected officials, impede the practice of EBPH (8-12). The wide-scale implementation of EBPH requires not only a workforce that understands and can implement EBPH efficiently but also sustained support from health department leaders, practitioners, and policy makers.

Calls for practitioners to include the concepts of EBPH in their work are increasing as the United States embarks upon the 10-year national agenda for health goals and objectives that constitutes the Healthy People 2020 initiative. The very mission of Healthy People 2020 asks for multisectoral action “to strengthen policies and improve practices that are driven by the best available evidence and knowledge” (13).

Funders, especially federal agencies, often require programs to be evidence-based. The American Recovery and Reinvestment Act of 2009 allocated $650 million to “carry out evidence-based clinical and community-based prevention and wellness strategies . . . that deliver specific, measurable health outcomes that address chronic disease rates” (14). The Patient Protection and Affordable Care Act of 2010 mentions “evidence-based” 13 times in Title IV, Prevention of Chronic Disease and Improving Public Health, and will provide $900 million in funding to 75 communities during 5 years through Community Transformation Grants (15).

Federal funding in states, cities, and tribes, and in both urban and rural areas, creates an expectation for EBPH at all levels of practice. Because formal public health training in the workforce is lacking (7), on-the-job training and skills development are needed. The need may be even greater in local health departments, where practitioners may be less aware of and slower to adopt evidence-based guidelines than state practitioners (16) and where training resources may be more limited.

Core Competencies for Public Health Professionals (17) emerged on the basis of recommendations of the Institute of Medicine’s 1988 report The Future of the Public’s Health . Last updated in May 2010, these 74 competencies represent a “set of skills desirable for the broad practice of public health,” and they are compatible with the skills needed for EBPH (3). Elements of state chronic disease programs and competencies endorsed by the National Association of Chronic Disease Directors are also compatible with EBPH (18).

In addition to efforts to establish competencies and certification for individual practitioners, voluntary accreditation for health departments is now offered through the Public Health Accreditation Board (PHAB). Tribal, state, and local health departments may seek this accreditation to document capacity to deliver the 3 core functions of public health and the Ten Essential Public Health Services (19). One of 12 domains specified by the PHAB as a required level of achievement is “to contribute to and apply the evidence base of public health” (19). This domain emphasizes the importance of the best available evidence and the role of health departments in adding to evidence for promising practices (19).

Several programs have been developed to meet EBPH training needs, including free, online resources (Box 1).

In 1997, the Prevention Research Center in St. Louis (PRC-StL) developed an on-site training course, Evidence-Based Public Health. To date, the course has reached more than 1,250 practitioners and has been replicated by PRC-StL faculty in 14 US states and 6 other countries. The course aims to “train the trainer” to extend the reach of the course and build local capacity (Box 2). Course evaluations are positive, and more than 90% of attendees have indicated they will use course information in their work (20-23). Course slides are available online, and a textbook is in its second edition (8). Using a similar framework, the University of Illinois at Chicago developed an online EBPH course that includes short quizzes and additional resources.

In 2006, with support from National Institutes of Health, experts from the fields of medicine, nursing, public health, social work, psychology, and library sciences formed the Council for Training in Evidence-Based Behavioral Practice. This group produced a transdisciplinary model of evidence-based practice that facilitates communication and collaboration (Figure) (2,4,5,24) and launched an interactive website to provide web-based training materials and resources to practitioners, researchers, and educators. The EBBP Training Portal, available free with registration, offers 9 modules on both individual and population-based approaches. Users learn how to choose effective interventions, evaluate interventions that are not yet proven, engage in decision making with others, and balance the 3 domains of evidence-based decision making (Figure).

Key elements of EBPH have been summarized (3) as the following:

• Engaging the community in assessment and decision making;
• Using data and information systems systematically;
• Making decisions on the basis of the best available peer-reviewed evidence (both quantitative and qualitative);
• Applying program planning frameworks (often based in health behavior theory);
• Conducting sound evaluation; and
• Disseminating what is learned.

Data for community assessment

As a first step in the EBPH process, a community assessment identifies the health and resource needs, concerns, values, and assets of a community. This assessment allows the intervention (a public health program or policy) to be designed and implemented in a way that increases the likelihood of success and maximizes the benefit to the community. The assessment process engages the community and creates a clear, mutual understanding of where things stand at the outset of the partnership and what should be tracked along the way to determine how an intervention contributed to change.

Public health surveillance is a critical tool for understanding a community’s health issues. Often conducted through national or statewide initiatives, surveillance involves ongoing systematic collection, analysis, and interpretation of quantitative health data. Various health issues and indicators may be tracked, including deaths, acute illnesses and injuries, chronic illnesses and impairments, birth defects, pregnancy outcomes, risk factors for disease, use of health services, and vaccination coverage. National surveillance sources typically provide state-level data, and county-level data have become more readily available in recent years (Box 1). State health department websites can also be sources of data, particularly for vital statistics and hospital discharge data. Additionally, policy tracking and surveillance systems (Box 1) monitor policy interest and action for various health topics (25).

Other data collection methods can be tailored to describe the particular needs of a community, creating new sources of data rather than relying on existing data. Telephone, mail, online, or face-to-face surveys collect self-reported data from community members. Community audits involve detailed counting of factors such as the number of supermarkets, sidewalks, cigarette butts, or health care facilities. For example, the Active Living Research website (www.activelivingresearch.org) provides a collection of community audit tools designed to assess how built and social environments support physical activity.

Qualitative methods can help create a more complete picture of a community, using words or pictures to describe the “how” and “why” of an issue. Qualitative data collection can take the form of simple observation, interviews, focus groups, photovoice (still or video images that document community conditions), community forums, or listening sessions. Qualitative data analysis involves the verbatim creation of transcripts, the development of data-sorting categories, and iterative sorting and synthesizing of data to develop sets of common concepts or themes (26).

Each of these forms of data collection offers advantages and disadvantages that must be weighed according to the planning team’s expertise, time, and budget. No single source of data is best. Most often data from several sources are needed to fully understand a problem and its best potential solutions. Several planning tools are available (Box 1) to help choose and implement a data collection method.

Selecting evidence

Once health needs are identified through a community assessment, the scientific literature can identify programs and policies that have been effective in addressing those needs. The amount of available evidence can be overwhelming; practitioners can identify the best available evidence by using tools that synthesize, interpret, and evaluate the literature.

Systematic reviews (Box 1) use explicit methods to locate and critically appraise published literature in a specific field or topic area. The products are reports and recommendations that synthesize and summarize the effectiveness of particular interventions, treatments, or services and often include information about their applicability, costs, and implementation barriers. Evidence-based practice guidelines are based on systematic reviews of research-tested interventions and can help practitioners select interventions for implementation. The Guide to Community Preventive Services (the Community Guide ), conducted by the Task Force on Community Preventive Services, is one of the most useful sets of reviews for public health interventions (27,28). The Community Guide evaluates evidence related to community or population-based interventions and is intended to complement the Guide to Clinical Preventive Services (systematic reviews of clinical preventive services) (29).

Not all populations, settings, and health issues are represented in evidence-based guidelines and systematic reviews. Furthermore, there are many types of evidence (eg, randomized controlled trials, cohort studies, qualitative research), and the best type of evidence depends on the question being asked. Not all types of evidence (eg, qualitative research) are equally represented in reviews and guidelines. To find evidence tailored to their own context, practitioners may need to search resources that contain original data and analysis. Peer-reviewed research articles, conference proceedings, and technical reports can be found in PubMed (www.ncbi.nlm.nih.gov/pubmed). Maintained by the National Library of Medicine, PubMed is the largest and most widely available bibliographic database; it covers more than 21 million citations in the biomedical literature. This user-friendly site provides tutorials on topics such as the use of Medical Subject Heading (MeSH) terms. Practitioners can freely access abstracts and some full-text articles; practitioners who do not have journal subscriptions can request reprints from authors directly. Economic evaluations provide powerful evidence for weighing the costs and benefits of an intervention, and the Cost-Effectiveness Analysis Registry tool (Box 1) offers a searchable database and links to PubMed abstracts.

The “gray” literature includes government reports, book chapters, conference proceedings, and other materials not found in PubMed. These sources may provide useful information, although readers should interpret non–peer-reviewed literature carefully. The New York Academy of Medicine produces a bimonthly Grey Literature Report (Box 1), and the US government maintains a website (www.science.gov) that searches the databases and websites of federal agencies in a single query. Internet search engines such as Google Scholar (http://scholar.google.com) may also be useful in finding both peer-reviewed articles and gray literature.

Program-planning frameworks

Program-planning frameworks provide structure and organization for the planning process. Commonly used models include PRECEDE-PROCEED (30), Intervention Mapping (31), and Mobilizing for Action through Planning and Partnerships (Box 1). Public health interventions grounded in health behavior theory often prove to be more effective than those lacking a theoretical base, because these theories conceptualize the mechanisms that underlie behavior change (32,33). Developed as a free resource for public health practitioners, the National Cancer Institute’s guide Theory at a Glance concisely summarizes the most commonly used theories, such as the ecological model, the health belief model, and social cognitive theory, and it uses 2 planning models (PRECEDE-PROCEDE and social marketing) to explain how to incorporate theory in program planning, implementation, and evaluation (34). Logic models are an important planning tool, particularly for incorporating the concepts of health-behavior theories. They visually depict the relationship between program activities and their intended short-term objectives and long-term goals. The first 2 chapters of the Community Tool Box explain how to develop logic models, provide overviews of several program-planning models, and include real-world examples (Box 1).

Evaluation and dissemination

Evaluation answers questions about program needs, implementation, and outcomes (35). Ideally, evaluation begins when a community assessment is initiated and continues across the life of a program to ensure proper implementation. Four basic types of evaluation can achieve program objectives, using both quantitative and qualitative methods. Formative evaluation is conducted before program initiation; the goal is to determine whether an element of the intervention (eg, materials, messages) is feasible, appropriate, and meaningful for the target population (36). Process evaluation assesses the way a program is being implemented, rather than the effectiveness of that program (36) (eg, counting program attendees and examining how they differ from those not attending).

Impact evaluation assesses the extent to which program objectives are being met and may reflect changes in knowledge, attitudes, behavior, or other intermediate outcomes. Ideally, practitioners should use measures that have been tested for validity (the extent to which a measure accurately captures what it is intended to capture) and reliability (the likelihood that the instrument will get the same result time after time) elsewhere. The Behavioral Risk Factor Surveillance System (BRFSS) is the largest telephone health survey in the world, and its website offers a searchable archive of survey questions since the survey’s inception in 1984 (Box 1). New survey questions receive a technical review, cognitive testing, and field testing before inclusion. A 2001 review summarized reliability and validity studies of the BRFSS (37).

Outcome evaluation provides long-term feedback on changes in health status, morbidity, mortality, or quality of life that can be attributed to an intervention. Because it takes so long to observe effects on health outcomes and because changes in these outcomes are influenced by factors outside the scope of the intervention itself, this type of evaluation benefits from more rigorous forms of quantitative evaluation, such as experimental or quasi-experimental rather than observational study designs.

The Centers for Disease Control and Prevention (CDC) Framework for Program Evaluation, developed in 1999, identifies a 6-step process for summarizing and organizing the essential elements of evaluation (38). The related CDC website (Box 1) maintains links to framework-based materials, step-by-step manuals, and other evaluation resources. Within a detailed outline of the CDC framework’s steps, the Community Toolbox also provides tools and examples (Box 1).

After an evaluation, the dissemination of findings is often overlooked, but practitioners have an implied obligation to share results with stakeholders, decision makers, and community members. Often these are people who participated in data collection and can make use of the evaluation findings. Dissemination may take the form of formal written reports, oral presentations, publications in academic journals, or placement of information in newsletters or on websites.

An increasing volume of scientific evidence is now at the fingertips of public health practitioners. Putting this evidence to work can help practitioners meet demands for a systematic approach to public health problem solving that yields measurable outcomes. Practitioners need skills, knowledge, support, and time to implement evidence-based policies and programs. Many tools exist to help efficiently incorporate the best available evidence and strategies into their work. Improvements in population health are most likely when these tools are applied in light of local context, evaluated rigorously, and shared with researchers, practitioners, and other stakeholders.

Preparation of this article was supported by the National Association of Chronic Disease Directors; cooperative agreement no. U48/DP001903 from CDC, Prevention Research Centers Program; CDC grant no. 5R18DP001139-02, Improving Public Health Practice Through Translation Research; and National Institutes of Health Office of Behavioral and Social Sciences Research contract N01-LM-6-3512, Resources for Training in Evidence-Based Behavioral Practice.

We thank Dr Elizabeth Baker, Dr Kathleen Gillespie, and the late Dr Terry Leet for their roles in developing the PRC-StL EBPH course. We thank the Colorado pilot portfolio teams Erik Aakko, Linda Archer, Gretchen Armijo, Mandy Bakulski, Renee Calanan, Julie Davis, Julie Graves, Indira Gujral, Rebecca Heck, Ashley Juhl, Kyle Legleitner, Flora Martinez, Kristin McDermott, Jessica Osborne, Kerry Thomson, Jason Vahling, and Stephanie Walton. We acknowledge the Mississippi EBPH team, Dr Victor Sutton, Dr Rebecca James, Dr Thomas Dobbs, Cassandra Dove, and State Health Officer Dr Mary Currier, for its commitment to the pilot and implementation of EBPH. We also thank Molly Ferguson, MPH (coordinator), and Drs Ed Mullen, Robin Newhouse, Steve Persell, and Jason Satterfield, members of the Council on Evidence-Based Behavioral Practice.

Corresponding Author: Ross C. Brownson, PhD, Washington University in St. Louis, Kingshighway Building, 660 S Euclid, Campus Box 8109, St. Louis, MO 63110. Telephone: 314-362-9641. E-mail: [email protected] .

Author Affiliations: Julie A. Jacobs, Prevention Research Center in St. Louis, Brown School, Washington University in St. Louis, St. Louis, Missouri; Ellen Jones, School of Health Related Professions, University of Mississippi Medical Center, Jackson, Mississippi; Barbara A. Gabella, Colorado Department of Public Health and Environment, Denver, Colorado; Bonnie Spring, Northwestern University Feinberg School of Medicine, Chicago, Illinois.

• Kohatsu ND, Robinson JG, Torner JC. Evidence-based public health: an evolving concept. Am J Prev Med 2004;27(5):417-21. CrossRef PubMed
• Satterfield JM, Spring B, Brownson RC, Mullen EJ, Newhouse RP, Walker BB, et al. Toward a transdisciplinary model of evidence-based practice. Milbank Q 2009;87(2):368-90. CrossRef PubMed
• Brownson RC, Fielding JE, Maylahn CM. Evidence-based public health: a fundamental concept for public health practice. Annu Rev Public Health 2009;30:175-201. CrossRef PubMed
• Spring B, Hitchcock K. Evidence-based practice. In: Weiner IB, Craighead WE, editors. Corsini encyclopedia of psychology. 4th edition. New York (NY): Wiley; 2009. p. 603-7.
• Spring B, Neville K, Russell SW. Evidence-based behavioral practice. In: Encyclopedia of human behavior. 2nd edition. New York (NY): Elsevier; 2012.
• Gebbie KM. Public health certification. Annu Rev Public Health 2009;30:203-10. CrossRef PubMed
• Turnock BJ. Public health: what it is and how it works. Sadbury (MA): Jones and Bartlett; 2009.
• Brownson RC, Baker EA, Leet TL, Gillespie KN, True WR. Evidence-based public health. 2nd edition. New York (NY): Oxford University Press; 2011.
• Dodson EA, Baker EA, Brownson RC. Use of evidence-based interventions in state health departments: a qualitative assessment of barriers and solutions. J Public Health Manag Pract 2010;16(6):E9-15. PubMed
• Baker EA, Brownson RC, Dreisinger M, McIntosh LD, Karamehic-Muratovic A. Examining the role of training in evidence-based public health: a qualitative study. Health Promot Pract 2009;10(3):342-8. CrossRef PubMed
• Brownson RC, Ballew P, Dieffenderfer B, Haire-Joshu D, Heath GW, Kreuter MW, et al. Evidence-based interventions to promote physical activity: what contributes to dissemination by state health departments. Am J Prev Med 2007;33(1 Suppl):S66-73. CrossRef PubMed
• Jacobs JA, Dodson EA, Baker EA, Deshpande AD, Brownson RC. Barriers to evidence-based decision making in public health: a national survey of chronic disease practitioners. Public Health Rep 2010;125(5):736-42. PubMed
• Healthy People 2020 framework: the vision, mission and goals of Healthy People 2020. US Department of Health and Human Services, Office of Disease Prevention and Health Promotion. http://www.healthypeople.gov/2020/Consortium/HP2020Framework.pdf. Accessed March 7, 2012.
• American Recovery and Reinvestment Act of 2009, Pub L No 111-5, 123 Stat 233 (2009).
• Patient Protection and Affordable Care Act of 2010, Pub L No 111-148, 124 Stat 119 (2010).
• Brownson RC, Ballew P, Brown KL, Elliott MB, Haire-Joshu D, Heath GW, et al. The effect of disseminating evidence-based interventions that promote physical activity to health departments. Am J Public Health 2007;97(10):1900-7. CrossRef PubMed
• Core competencies for public health professionals. Council on Linkages Between Academia and Public Health Practice. http://www.phf.org/resourcestools/Pages/Core_Public_Health_Competencies.aspx. Accessed March 7, 2012.
• Slonim A, Wheeler FC, Quinlan KM, Smith SM. Designing competencies for chronic disease practice. Prev Chronic Dis 2010;7(2). http://www.cdc.gov/pcd/issues/2010/mar/08_0114.htm. Accessed March 7, 2012. PubMed
• Standards and measures. Public Health Accreditation Board. http://www.phaboard.org/accreditation-process/public-health-department-standards-and-measures/. Accessed March 7, 2012.
• Brownson RC, Diem G, Grabauskas V, Legetic B, Poternkina R, Shatchkute A, et al. Training practitioners in evidence-based chronic disease prevention for global health. Promot Educ 2007;14(3):159-63. PubMed
• O’Neall MA, Brownson RC. Teaching evidence-based public health to public health practitioners. Ann Epidemiol 2005;15(7):540-4. CrossRef PubMed
• Dreisinger M, Leet TL, Baker EA, Gillespie KN, Haas B, Brownson RC. Improving the public health workforce: evaluation of a training course to enhance evidence-based decision making. J Public Health Manag Pract 2008;14(2):138-43. PubMed
• Franks AL, Brownson RC, Bryant C, Brown KM, Hooker SP, Pluto DM, et al. Prevention Research Centers: contributions to updating the public health workforce through training. Prev Chronic Dis 2005;2(2). http://www.cdc.gov/pcd/issues/2005/apr/04_0139.htm. Accessed March 7, 2012. PubMed
• Newhouse RP, Spring B. Interdisciplinary evidence-based practice: moving from silos to synergy. Nurs Outlook 2010;58(6):309-17. CrossRef PubMed
• Chriqui JF, O’Connor JC, Chaloupka FJ. What gets measured, gets changed: evaluating law and policy for maximum impact. J Law Med Ethics 2011;39 (Suppl 1)21-6. CrossRef PubMed
• Hesse-Biber S, Leavy P. The practice of qualitative research. Thousand Oaks (CA): Sage; 2006.
• Mullen PD, Ramirez G. The promise and pitfalls of systematic reviews. Annu Rev Public Health 2006;27:81-102. CrossRef PubMed
• Zaza S, Briss PA, Harris KW, editors. The guide to community preventive services: what works to promote health? New York (NY): Oxford University Press; 2005.
• U.S. Preventive Services Task Force. http://www.uspreventiveservicestaskforce.org/. Accessed March 9, 2012.
• Green LW, Kreuter MW. Health promotion planning: an educational and ecological approach. 4th edition. New York (NY): McGraw-Hill; 2004.
• Bartholomew LK, Parcel GS, Kok G, Gottlieb NH, Fernandez ME. Planning health promotion programs: an Intervention Mapping approach. 3rd edition. San Francisco (CA): Jossey-Bass; 2011.
• Glanz K, Bishop DB. The role of behavioral science theory in the development and implementation of public health interventions. Annu Rev Public Health 2010;31:399-418. CrossRef PubMed
• Glanz K, Rimer BK, Viswanath K. Health behavior and health education: theory, research, and practice. 4th edition. San Francisco (CA): Jossey-Bass; 2008.
• Glanz K, Rimer BK. Theory at a glance: a guide for health promotion practice. National Cancer Institute, National Institutes of Health; 2005. (NIH publication 05-3896). http://www.cancer.gov/cancertopics/cancerlibrary/theory.pdf. Accessed March 7, 2012.
• Shadish WR. The common threads in program evaluation. Prev Chronic Dis 2006;3(1). http://www.cdc.gov/pcd/issues/2006/jan/05_0166.htm. Accessed March 7, 2012. PubMed
• Thompson N, Kegler M, Holtgrave D. Program evaluation. In: Crosby RA, DiClemente RJ, Salazar LF, editors. Research methods in health promotion. San Francisco (CA): Jossey-Bass; 2006. p. 199-225.
• Nelson DE, Holtzman D, Bolen J, Stanwyck CA, Mack KA. Reliability and validity of measures from the Behavioral Risk Factor Surveillance System (BRFSS). Soz Praventivmed 2001;(46 Suppl 1):S3-42. PubMed
• Centers for Disease Control and Prevention. Framework for program evaluation in public health. MMWR Recomm Rep 1999;(48 RR-11):1-40. http://www.cdc.gov/mmwr/preview/mmwrhtml/rr4811a1.htm. Accessed March 7, 2012. PubMed

File Formats Help:

• PCD podcasts
• PCD on Facebook
• Page last reviewed: July 26, 2012
• Page last updated: July 26, 2012
• Content source: National Center for Chronic Disease Prevention and Health Promotion
• Using this Site
• Contact CDC

• Open access
• Published: 29 March 2022

A framework of evidence-based decision-making in health system management: a best-fit framework synthesis

• Tahereh Shafaghat 1 , 2   na1 ,
• Peivand Bastani   ORCID: orcid.org/0000-0002-0412-0267 1 , 3   na1 ,
• Mohammad Hasan Imani Nasab 4 ,
• Mohammad Amin Bahrami 1 ,
• Mahsa Roozrokh Arshadi Montazer 5 ,
• Mohammad Kazem Rahimi Zarchi 2 &
• Sisira Edirippulige 6  

Archives of Public Health volume  80 , Article number:  96 ( 2022 ) Cite this article

10k Accesses

6 Citations

6 Altmetric

Metrics details

Scientific evidence is the basis for improving public health; decision-making without sufficient attention to evidence may lead to unpleasant consequences. Despite efforts to create comprehensive guidelines and models for evidence-based decision-making (EBDM), there isn`t any to make the best decisions concerning scarce resources and unlimited needs . The present study aimed to develop a comprehensive applied framework for EBDM.

This was a Best-Fit Framework (BFF) synthesis conducted in 2020. A comprehensive systematic review was done via six main databases including PUBMED, Scopus, Web of Science, Science Direct, EMBASE, and ProQuest using related keywords. After the evidence quality appraisal, data were extracted and analyzed via thematic analysis. Results of the thematic analysis and the concepts generated by the research team were then synthesized to achieve the best-fit framework applying Carroll et al. (2013) approach.

Four thousand six hundred thirteen studies were retrieved, and due to the full-text screening of the studies, 17 final articles were selected for extracting the components and steps of EBDM in Health System Management (HSM). After collecting, synthesizing, and categorizing key information, the framework of EBDM in HSM was developed in the form of four general scopes. These comprised inquiring, inspecting, implementing, and integrating, which included 10 main steps and 47 sub-steps.

Conclusions

The present framework provided a comprehensive guideline that can be well adapted for implementing EBDM in health systems and related organizations especially in underdeveloped and developing countries where there is usually a lag in updating and applying evidence in their decision-making process. In addition, this framework by providing a complete, well-detailed, and the sequential process can be tested in the organizational decision-making process by developed countries to improve their EBDM cycle.

Peer Review reports

Globally, there is a growing interest in using the research evidence in public health policy-making [ 1 , 2 ]. Public health systems are diverse and complex, and health policymakers face many challenges in developing and implementing policies and programs that are required to be efficient [ 1 , 3 ]. The use of scientific evidence is considered to be an effective approach in the decision-making process [ 3 , 4 , 5 ]. Due to the lack of sufficient resources, evidence-based decision-making ( EBDM) is regarded as a way to optimize costs and prevent wastes [ 6 ]. At the same time, the direct consequence of ignoring evidence is poorer health for the community [ 7 ].

Evidence suggests that health systems often fail to exploit research evidence properly, leading to inefficiencies, death or reduced quality of citizens’ lives, and a decline in productivity [ 8 ]. Decision-making in the health sector without sufficient attention to evidence may lead to a lack of effectiveness, efficiency, and fairness in health systems [ 9 ]. Instead, the advantages of EBDM include adopting cost-effective interventions, making optimal use of limited resources, increasing customer satisfaction, minimizing harm to individuals and society, achieving better health outcomes for individuals and society [ 10 , 11 ], as well as increasing the effectiveness and efficiency of public health programs [ 12 ].

Using the evidence in health systems’ policymaking is a considerable challenging issue that many developed and developing countries are facing nowadays. This is particularly important in the latter, where their health systems are in a rapid transition [ 13 ]. For instance, although in 2012, a study in European Union countries showed that health policymakers rarely had necessary structures, processes, and tools to exploit research evidence in the policy cycle [ 14 ], the condition can be worse among the developing and the underdeveloped ones. For example, evidence-based policy-making in developing countries like those located in the Middle East can have more significant impacts [ 15 , 16 ]. In such countries resources are generally scarce, so the policymakers' awareness of research evidence becomes more important [ 17 ]. In general, low and middle-income countries have fewer resources to deal with health issues and need quality evidence for efficient use of these resources [ 7 ].

Since the use of EBDM is fraught with the dilemma of most pressing needs and having the least capacity for implementation especially in developing countries [ 16 ], efforts have been made to create more comprehensive guidelines for EBDM in healthcare settings, in recent years [ 18 ]. Stakeholders are significantly interested in supporting evidence-based projects that can quickly prioritize funding allocated to health sectors to ensure the effective use of their financial resources [ 19 , 20 , 21 ]. However, it is unlikely that the implementation of EBDM in Health System Management (HSM) will follow the evidence-based medicine model [ 10 , 22 ]. On the other hand, the capacity of organizations to facilitate evidence utilization is complex and not well understood [ 22 ], and the EBDM process is not usually institutionalized within the organizational processes [ 10 ]. A study in 2005 found that few organizations support the use of research evidence in health-related decisions, globally [ 23 ]. Weis et al. (2012) also reported there is insufficient information on EBDM in local health sectors [ 12 ]. In general, it can be emphasized that relatively few organizations hold themselves accountable for using research evidence in developing health policies [ 24 ]. To the best of our knowledge, there isn`t any comprehensive global and practical model developed for EBDM in health systems/organizations management. Accordingly, the present study aimed to develop a comprehensive framework for EBDM in health system management. It can shed the light on policymakers to access a detailed practical model and enable them to apply the model in actual conditions.

This was a Best Fit Framework (BFF) synthesis conducted in 2020 to develop a comprehensive framework for EBDM in HSM. Such a framework synthesis is achieved as a combination of the relevant framework, theory, or conceptual models and particularly is applied for developing a priori framework based on deductive reasoning [ 25 ]. The BFF approach is appropriate to create conceptual models to describe or express the decisions and behaviors of individuals and groups in a particular domain. This is distinct from other methods of evidence synthesis because it employs a systematic approach to create an initial framework for synthesis based on existing frameworks, models, or theories [ 25 ] for identifying and adapting theories systematically with the rapid synthesis of evidence [ 25 , 26 ]. The initial framework can be derived from a relatively well-known model in the target field, or be formed by the integration of several existing models. The initial framework is then reduced to its key components that have shaped its concepts [ 25 ]. Indeed, the initial framework considers as the basis and it can be rebuilt, extended, or reduced based on its dimensions [ 26 ]. New concepts also emerge based on the researchers' interpretation of the evidence and ongoing comparisons of these concepts across studies [ 25 ]. This approach of synthesis possesses both positivist and interpretative perspectives; it provides the simultaneous use of the well-known strengths of both framework and evidence synthesis [ 27 ].

In order to achieve this aim the following methodological steps were conducted as follows:

Searching and selection of studies

In this step, we aimed to look for the relevant models and frameworks related to evidence-based decision-making in health systems management. The main research question was “what is the best framework for EBDM in health systems?” after defining the research question, the researchers searched for published studies on EBDM in HSM in different scientific databases with relevant keywords and constraints as inclusion and exclusion criteria from 01.01.2000 to 12.31.2020 (Table 1 ).

Inclusion and exclusion criteria

Inclusion criteria were determined as the studies that identify the components or develop a model or framework of EBDM in health organization in the form of original or review articles or dissertations, which were published in English and had a full text. The studies like book reviews, opinion articles, and commentaries that lacked a specific framework for conducting our review were excluded. During the search phase of the study, we attempted as much as possible to access studies that were not included in the search process or gray literature by reviewing the references lists of the retrieved studies or by contacting the authors of the articles or experts and querying them, as well as manually searching the related sites (Fig.  1 ).

The PRISMA flowchart for selection of the studies in scoping review

Quality appraisal

The quality of the obtained studies was investigated using three tools for assessing the quality of various types of studies considering types and methods of the final include studies in systematic review. These tools were including Critical Appraisal Skills Program (CASP) for assessing the quality of qualitative researches [ 28 ], Scale for the Assessment of Narrative Review Articles (SANRA) [ 29 ], and The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers [ 30 ] (Table 3- Appendix ).

Data extraction

After searching the studies from all databases and removing duplicates, the studies were independently reviewed and screened by two members (TS and MRAM) of the research team in three phases by the title, abstract, and then the full text of the articles. At each stage of the study, the final decision to enter the study to the next stage was based on agreement and, in case of disagreement, the opinion of the third person from the research team was asked (PB). Mendeley reference manager software was used to systematically search and screen relevant studies. The data from the included studies were extracted based on the study questions and accordingly, a form of the studies’ profile including the author's name, publication year, country, study title, type of study, and its conditions were prepared in Microsoft Excel software (Table 4- Appendix ).

Synthesis and the conceptual model

In this step, a thematic analysis approach was applied to extract and analyze the data. For this purpose, first, the texts of the selected studies were read several times, and the initial qualitative codes or thematic concepts, according to the determined keywords and based on the research question, were found and labeled. Then these initial thematic codes were reviewed to achieve the final codes and they were integrated and categorized to achieve the final main themes and sub-themes, eventually. The main and the sub-themes are representative of the main and sub-steps of EBDM. At the last stage of the synthesis, the thematic analysis was finalized with 8 main themes and all the main and the sub-themes were tabulated (Table 5- Appendix ).

Creation of a new conceptual framework

For BFF synthesis in the present study, we compared the existing models and tried to find a model that fits the best. Three related models that appeared to be relatively well-suited to the purpose of this study to provide a complete, comprehensive, and practical EBDM model in HSM were found. According to the BFF instruction in Carroll et al. (2013) study [ 25 ], we decided to use all three models as the basis for the best fit because any of those models were not complete enough and we could give no one an advantage over others. Consequently, the initial model or the BFF basis was formed and the related thematic codes were classified according to the category of this basis as the main themes/steps of EBDM in HSM (Table 5- Appendix ). Then, the additional founded thematic codes were added and incorporated to this basis as the other main steps and the sub-steps of the EBDM in HSM according to the research team and some details in the form of sub-steps were added by the research team to complete the synthesized framework. Eventually, a comprehensive practical framework consisting of 10 main steps and 47 sub-steps was created with the potentiality of applying and implementing EDBM in HSM that we categorized them into four main phases (Table 6- Appendix ).

Testing the synthesis: comparison with the a priori models, dissonance and sensitivity

In order to assess the differences between the priori framework and the new conceptual framework, the authors tried to ask some experts’ opinions about the validity of the synthesized results. The group of experts has included eight specialists in the field of health system management or health policy-making. These experts have been chosen considering their previous research or experience in evidence-based decision/policy making performance/management (Table 2 ). This panel lasted in two three-hour sessions. The finalized themes and sub-themes (Table 6- Appendix ) and the new generated framework (Fig.  3 ) were provided to them before each session so that they could think and then in each meeting they discussed them. Finally, all the synthesized themes and sub-themes resulted were reviewed and confirmed by the experts.

Ethical considerations

To prevent bias, two individuals carried out all stages of the study such as screening, data extraction, and data analysis. The overall research project related to this manuscript was approved by the medical ethics conceal of the research deputy of Shiraz University of Medical Sciences with approval number IR.SUMS.REC.1396–01-07–14184, too.

The initial search across six electronic databases and the Cochrane library yielded 4613 studies. After removing duplicates, 2416 studies were assessed based on their titles. According to the abstract screening of the 1066 studies that remained after removing the irrelevant titles, 291 studies were selected and were entered into the full-text screening phase. Due to full-text screening of the studies, 17 final studies were selected for extracting the components and steps of EBDM in HSM (Fig.  1 ). The features of these studies were summarized in Table 4- Appendix (see supplementary data). Furthermore, according to the quality appraisal of the included studies, the majority of them had an acceptable level of quality. These results have been shown in Table 3- Appendix .

Results of the thematic analysis of the evidence (Table 5- Appendix ) along with the concepts proposed and added by the research team according to the focus-group discussion of the experts were shown in Table 6- Appendix . Accordingly, the main steps and related sub-steps of the EBDM process in HSM were defined and categorized.

After collecting, synthesizing, and categorizing thematic concepts, incorporating them with the initial models, and adding the additional main steps and sub-steps to the basic models, the final synthesized framework as a best-fit framework for EBDM in HSM was developed in the form of four general phases of inquiring, inspecting, implementing, and integrating and 10 main steps (Fig.  2 ). For better illustration, this framework with all the main steps and 47 sub-steps has been shown in Fig.  3 , completely.

The final synthesized framework of evidence-based decision-making in health system management

The main steps and sub-steps of the framework of EBDM in health system management

In the present study, a comprehensive framework for EBDM in HSM was developed. This model has different distinguishing characteristics than the formers. First of all, this is a comprehensive practical model that combined the strengths and the crucial components of the limited number of previous models; second, the model includes more details and complementary steps and sub-steps for full implementation of EBDM in health organizations and finally, the model is benefitted from a cyclic nature that has a priority than the linear models. Concerning the differences between the present framework and other previous models in this field, it must be said that most of the previous models related to EBDM were presented in the scope of medicine (that they were excluded from our SR according to the study objectives and exclusion criteria). A significant number of those models were proposed for the scope of public health and evidence-based practice, and only a limited number of them focused exactly on the scope of management and policy/decision making in health system organizations.

Given that the designed model is a comprehensive 10-step model, it can be used in some way at all levels of the health system and even in different countries. However, there will be a difference here, given that this framework provides a practical guide and a comprehensive guideline for applying evidence-based decision-making approach in health systems organizations, at each level of the health system in each country, this management approach can be applied depending on their existing infrastructure and the processes that are already underway (such as capacity building, planning, data collection, etc.), and at the same time, with a general guide, they can provide other infrastructure as well as the prerequisites and processes needed to make this approach much more possible and applicable.

It is true that evidence-based management is different from evidence-based medicine and even more challenging (due to lack of relevant data, greater sensitivity in data collection and their accuracy, lack of consistency and lack of transparency in the implementation of evidence-based decision-making in management rather than evidence-based medicine, etc.). Still, the general framework provided in this article can be used to help organizations that really want to act and move forward through this approach.

Furthermore, based on the findings, most of the previous studies only referred to some parts of the components and steps of the EBDM in health organizations and neglected the other parts or they were not sufficiently comprehensive [ 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 ]. Most of the previous models did not mention the necessary sub-steps, tools, and practical details for accurate and complete implementation of the EBDM, which causes the organizations that want to use these models, will be confused and cannot fully implement and complete the EBDM cycle. Among the studies that have provided a partly complete model than the other studies, were the studies by Brownson (2009), Yost (2014), and Janati (2018) [ 3 , 41 , 42 ]. Consequently, the combination of these three studies has been used as the initial framework for the best-fit synthesis in the present study.

Likewise, the models presented by Brownson (2009) and Janati (2018) were only limited to the six or seven key steps of the EBDM process, and they did not mention the details required for doing in each step, too [ 3 , 4 , 42 ]. Also, the model presented in the study of Janati (2018) was linear, and the relationships between the EBDM components were not well considered [ 42 , 43 ]; however, the model presented in this study was recursive. Also, in Yost's study (2014), despite the 7 main steps of EBDM and some details of each of the steps, the proposed process was not schematically drawn in the form of a framework and therefore the relationships between steps and sub-steps were not clear [ 41 ]. According to what was discussed, the best-fit framework makes the possibility of concentrating the fragmented models to a comprehensive one that can be fully applied and evaluated by the health systems policymakers and managers.

In the present study, the framework of EBDM in HSM was developed in the form of four general scopes of inquiring, inspecting, implementing, and integrating including 10 main steps and 47 sub-steps. These scopes were discussed as follows:

In the first step, “situation analysis and priority setting”, the most frequently cited sub-step was identifying and prioritizing the problem. Accordingly, Falzer (2009), emphasized the importance of identifying the decision-making conditions and the relevant institutions and determining their dependencies as the first steps of EBDM [ 44 ]. Aas (2012) has also cited the assessment of individuals and problem status and problem-finding as the first steps of EBDM [ 34 ]. Moreover, the necessity of identifying the existing situation and issues and prioritizing them has been emphasized as the initial steps in most management models such as environmental analysis in strategic planning [ 45 ].

Despite considering the opinions and experience of experts and managers as one of the important sources of evidence for decision-making [ 42 , 46 , 47 , 48 , 49 , 50 ], many studies did not mention this sub-step in the EBDM framework. Hence, the present authors added the acquisition of experts’ opinions as a sub-step of the first step because of its important role in achieving a comprehensive view of the overall situation.

In the second step, “quantifying the issue and developing a statement”, “Developing the conceptual model for the issue” was more addressed [ 37 , 41 , 47 ]. In addition, the authors to complete this step added the fourth sub-step, “Defining the main statement of issue”. This is because that most of the problems in health settings may have a similar value for managers and decision-makers and quantifying them can be used as a criterion for more attention or selecting the problem as the main issue to solve.

The third step, “Capacity building and setting objectives”, was not seen in many other included studies as a main step in EBDM, however, the present authors include this as a main step because without considering the appropriate objectives and preparing necessary capacities and infrastructures, entering to the next steps may become problematic. Moreover, in numerous studies, factors such as knowledge and skills of human resources, training, and the availability of the essential structures and infrastructures have been identified as facilitators of EBDM [ 51 , 52 , 53 , 54 , 55 ]. According to this justification, they are included in the present framework as sub-steps of the third step.

Considering the third step and based on the knowledge extracted from the previous studies, the three sub-steps of “understanding context and Building Culture” [ 56 , 57 ], “gaining the support and commitment of leaders” [ 39 , 57 , 58 ], and “identifying the capabilities required by employees and their skills weaknesses” [ 58 , 59 , 60 ] were the most important sub-steps in this step of EBDM framework. In this regard, Dobrow (2004) has also stated that the two essential components of any EBDM are the evidence and context of its use [ 32 ]. Furthermore, Isfeedvajani (2018) stated that to overcome barriers and persuade hospital managers and committees to apply evidence-based management and decision-making, first and foremost, creating and promoting a culture of "learning through research" was important [ 61 ].

The present findings showed that in the fourth main step, “evidence acquisition and integration”, the most important sub-step was “finding the sources for seeking the evidence” [ 39 , 40 , 41 , 60 , 62 , 63 ]. Concerning the sources for the use of evidence in decision-making in HSM, studies have cited numerous sources, most notably scientific and specialized evidence such as research, articles, academic reports, published texts, books, and clinical guidelines [ 39 , 64 , 65 ]. After scientific evidence, using the opinions and experiences of experts, colleagues, and managers [ 42 , 46 , 49 , 66 ] as well as the use of census and local level data [ 49 , 66 , 67 ], and other sources such as financial [ 67 ], political [ 42 , 49 ] and evaluations [ 49 , 68 ] data were cited.

The fifth step of the present framework, “evidence appraising”, was emphasized by previous literature; for instance, Pierson (2012) pointed to the use of library services in EBDM [ 69 ]. Appraising and selecting the evidence according to appropriate appraisal tools/methods was cited the most. International and local evidence is confirmed that ignoring these criteria can lead to serious faults in the process of decision and policy-making [ 70 , 71 ].

Furthermore, the sixth step, “analysis, synthesis, and interpretation of data”, was mentioned in many included studies [ 36 , 39 , 41 , 42 , 57 , 59 , 72 ]. This step emphasized the role of analysis and synthesis of data in the process of generation applied and useful information. It is obvious that the local interpretation according to different contexts may lead to achieving such kind of knowledge that can be used as a basis for local EBDM in HSM.

Implementing

The third scope consisted of the seventh and eighth steps of the EBDM process in HSM. In the seventh step, “developing evidence-based alternatives”, the issue of involving stakeholders in decision-making and subsequently, planning to design and implementation of the process and evaluation strategies had been focused by the previous studies [ 58 , 60 , 62 , 63 , 73 ]. Studies by Belay (2009) and Armstrong (2014) had also emphasized the need to use stakeholder and public opinion as well as local and demographic data in decision-making [ 49 , 67 ].

“Pilot-implementation of selected alternatives” was the eighth step of the framework. Some key sub-steps of this step were resources allocation [ 58 ], Pre-implementation and pilot change in practice and assessing barriers and enablers for implementation [ 40 ] that indicated the significance of testing the strategies in a pilot stage as a pre- requisition of implementing the whole alternatives. It is obvious that without attention to the pilot stage, adverse and unpleasant outcomes may occur that their correction process imposes many financial, organizational, and human costs on the originations. In addition, a study explained that one of the strategies of the decision-makers to measure the feasibility of the policy options was piloting them, which had a higher chance of being approved by the policymakers. Also, pilot implementation in smaller scales has been recommended in public health in cases of lack of sufficient evidence [ 74 ].

Integrating

This last scope consists of the ninth and tenth steps. The main sub-step of the ninth step, “evaluating alternatives”, was to evaluating process and outcomes and revise. After a successful implementation of the pilot, this step can be assured that the probable outcomes may be achieved and this evaluation will help the decision and policymakers to control the outcomes, effectively. Also, it impacts the whole target program and proposes some correcting plans through an accurate feedback process, too. Pagoto (2007) explained that a facilitator for EBDM would be an efficient and user-friendly system to assess utilization, outcomes, and perceived benefits [ 55 ].

Also, the tenth step, “integrating and maintaining change in practice”, was not considered as a major step in previous models, too, while it is important to maintain and sustain positive changes in organizational performance. In this regard, Ward (2011) also suggested several steps to maintain and sustain the widespread changes in the organization, including increasing the urgency and speed of action, forming a team, getting the right vision, negotiating for buy-in, empowerment, short-term success, not giving up and help to make a change stick [ 35 ]. Finally, the most important sub-steps that could be mentioned in this step were the dissemination of evidence results to decision-makers and the integration of changes made to existing standards and performance guidelines. Liang (2012) had also emphasized the importance of translating existing evidence into useful practices as well as disseminating them [ 47 ]. In addition, the final sub-step, “feedback and feedforward towards the EBDM framework”, was explained by the authors to complete the framework.

Some previous findings showed that about half and two-thirds of organizations do not regularly collect related data about the use of evidence, and they do not systematically evaluate the usefulness or impact of evidence use on interventions and decisions [ 75 ]. The results of a study conducted on healthcare managers at the various levels of an Iranian largest medical university showed that the status of EBDM is not appropriate. This problem was more evident among physicians who have been appointed as managers and who have less managerial and systemic attitudes [ 76 ]. Such studies, by concerning the shortcomings of current models for EBDM in HSM or even lack of a suitable and usable one, have confirmed the necessity of developing a comprehensive framework or model as a practical guide in this field. Consequently, existing and presenting such a framework can help to institutionalize the concept of EBDM in health organizations.

In contrast, results of Lavis study (2008) on organizations that supported the use of research evidence in decision-making reported that more than half of the organizations (especially institutions of health technology assessment agencies) may use the evidence in their process of decision-making [ 75 ], so applying the present framework for these organizations can be recommended, too.

Limitations

One of the limitations of the present study was the lack of access to some studies (especially gray literature) related to the subject in question that we tried to access them by manual searching and asking from some articles’ authors and experts. In addition, most of the existing studies on EBDM were limited to examining and presenting results on influencing, facilitating, or hindering factors or they only mentioned a few components in this area. Consequently, we tried to search for studies from various databases and carefully review and screen them to make sure that we did not lose any relevant data and thematic code. Also, instead of one model, we used four existing models as a basis in the BFF synthesis so that we can finally, by adding additional codes and themes obtained from other studies as well as expert opinions, provide a comprehensive model taking into account all the required steps and details. Also, the framework developed in this study is a complete conceptual model made by BFF synthesis; however, it may need some localization, according to the status and structure of each health system, for applying it.

The present framework provides a comprehensive guideline that can be well adapted for implementing EBDM in health systems and organizations especially in underdeveloped and developing countries where there is usually a lag in updating and applying evidence in their decision-making process. In addition, this framework by providing a complete, well-detailed, sequential and practical process including 10 steps and 56 sub-steps that did not exist in the incomplete related models, can be tested in the organizational decision-making process or managerial tasks by developed countries to improve their EBDM cycle, too.

Availability of data and materials

All data in a form of data extraction tables are available from the corresponding author on a reasonable request.

Abbreviations

• Evidence-based decision-making

Health System Management

Best-Fit Framework

Rychetnik L, Bauman A, Laws R, King L, Rissel C, Nutbeam D, et al. Translating research for evidence-based public health: Key concepts and future directions. J Epidemiol Community Health. 2012;66(12):1187–92.

Article   Google Scholar  

Nutbeam D, Boxall AM. What influences the transfer of research into health policy and practice? Observations from England and Australia. Public Health. 2008;122(8):747–53.

Brownson RC, Fielding JE, Maylahn CM. Evidence-Based Public Health: A Fundamental Concept for Public Health Practice. Annu Rev Public Health [Internet]. 2009;30(1):175–201. https://doi.org/10.1146/annurev.publhealth.031308.100134 .

Brownson RC, Gurney JG, Land GH. Evidence-based decision making in public health. J Public Heal Manag Pract [Internet]. 1999 [cited 2018 May 26];5(5):86–97. Available from: https://books.google.com/books?hl=en&lr=&id=yxRgAwAAQBAJ&oi=fnd&pg=PA133&dq=+criteria+OR+health+%22evidence+based+decision+making%22&ots=hiqVNQtF24&sig=jms9GsBfw6gz1cN2FQXzCBZvmMQ

McGinnis JM. ‘Does Proof Matter? Why Strong Evidence Sometimes Yields Weak Action.’ Am J Heal Promot. 2001;15(5):391–396. Available from:   https://doi.org/10.4278/0890-1171-15.5.391 .

Majdzadeh R, Yazdizadeh B, Nedjat S, Gholami J, Ahghari S, R. M, et al. Strengthening evidence-based decision-making: Is it possible without improving health system stewardship? Health Policy Plan [Internet]. 2012 Sep 1 [cited 2018 May 15];27(6):499–504. Available from: https://doi.org/10.1093/heapol/czr072

WHO EIPNET. Using evidence and innovation to strengthen policy and practice. 2008.

Ellen ME, Léon G, Bouchard G, Lavis JN, Ouimet M, Grimshaw JM. What supports do health system organizations have in place to facilitate evidence-informed decision-making? A qualitative study. Implement Sci. 2013;8(1):84.

Oxman AD, Lavis JN, Lewin S, Fretheim A. What is evidence-informed policymaking? Health Res Policy Sys. 2009;7:1–7.

Armstrong R, Waters E, Dobbins M, Anderson L, Moore L, Petticrew M, et al. Knowledge translation strategies to improve the use of evidence in public health decision making in local government: Intervention design and implementation plan. Implement Sci [Internet]. 2013;8(1):1. Available from: https://www.scopus.com/inward/record.uri?eid=2-s2.0-84885076767&doi=10.1186%2F1748-5908-8-121&partnerID=40&md5=dddaf4029205c63877a3e6ecf28f7762

Waters E, Armstrong R, Swinburn B, Moore L, Dobbins M, Anderson L, et al. An exploratory cluster randomised controlled trial of knowledge translation strategies to support evidence-informed decision-making in local governments (The KT4LG study). BMC Public Health [Internet]. 2011;11(1):34. Available from: http://www.biomedcentral.com/1471-2458/11/34

Imani-Nasab MH, Yazdizadeh B, Salehi M, Seyedin H, Majdzadeh R. Validity and reliability of the Evidence Utilisation in Policymaking Measurement Tool (EUPMT). Heal Res Policy Syst. 2017;15(1):1–11.

El-Jardali F, Lavis JN, Ataya N, Jamal D, Ammar W, Raouf S. Use of health systems evidence by policymakers in eastern mediterranean countries: Views, practices, and contextual influences. BMC Health Serv Res. 2012;12(1):200.

Ettelt S, Mays N. Health services research in Europe and its use for informing policy. J Health Serv Res Policy [Internet]. 2011 Jul 29 [cited 2019 Jun 29];16(2_suppl):48–60. Available from: https://doi.org/10.1258/jhsrp.2011.011004

Sutcliffe S. Evidence-Based Policymaking: What is it? How does it work? What relevance for developing countries? 2005.

Campbell DM, Redman S, Jorm L, Cooke M, Zwi AB, Rychetnik L. Increasing the use of evidence in health policy: Practice and views of policy makers and researchers. Aust New Zealand Health Policy. 2009;6(1):1–11.

WHO. Supporting the Use of Research Evidence (SURE) for Policy in African Health Systems. 2007:1–72.

Health Public Accreditation Board. Public Health Accreditation Board STANDARDS : AN OVERVIEW. 2012.

Riley WJ, Bender K, Lownik E. Public health department accreditation implementation: Transforming public health department performance. Am J Public Health. 2012;102(2):237–42.

Liebman JB. Building on Recent Advances in Evidence-Based Policymaking. 2013:36. Available from: http://www.americaachieves.org/9F42769C-3A65-4CCB-8466-E7BFD835CFE7/FinalDownload/DownloadId-0AB31816953C7B8AFBD2D15DFD39D5A8/9F42769C-3A65-4CCB-8466-E7BFD835CFE7/docs/RFA/THP_Liebman.pdf

Jacobs J a, Jones E, Gabella B a, Spring B, Brownson C. Tools for Implementing an Evidence-Based Approach in Public Health Practice. Prev Chronic Dis [Internet]. 2012;9(1):1–9. Available from: https://doi.org/10.5888/pcd9.110324

Kothari A, Edwards N, Hamel N, Judd M. Is research working for you? validating a tool to examine the capacity of health organizations to use research. Implement Sci. 2009;4(1):1–9.

Oxman AD, Bjørndal A, Becerra-Posada F, Gibson M, Block MAG, Haines A, et al. A framework for mandatory impact evaluation to ensure well informed public policy decisions. Lancet. 2010;375(9712):427–31. Available from: https://doi.org/10.1016/S0140-6736(09)61251-4

Oxman AD, Vandvik PO, Lavis JN, Fretheim A, Lewin S. SUPPORT Tools for evidence-informed health Policymaking (STP) 2: Improving how your organisation support the use of research evidence to inform policymaking. Chinese J Evidence-Based Med. 2010;10(3):247–54.

Google Scholar  

Carroll C, Booth A, Leaviss J, Rick J. “ Best fit ” framework synthesis : refining the method. BMC Med Res Methodol. 2013;13(1):1 Available from: BMC Medical Research Methodology.

Carroll C, Booth A, Cooper K. A worked example of “ best fit ” framework synthesis : A systematic review of views concerning the taking of some potential chemopreventive agents. BMC Med Res Methodol. 2011;11:29.

Barnett-page E, Thomas J. Methods for the synthesis of qualitative research : a critical review. BMC Med Res Methodol 9. 2009;59:1–26.

Public Health Resource Unit. Critical Appraisal Skills Programme (CASP) making sense of evidence;10 questions to help you make sense of qualitative research. Public Health. 2006.

Baethge C, Goldbeck-Wood S, Mertens S. SANRA—a scale for the quality assessment of narrative review articles. Res Integr Peer Rev. 2019;4(1):2–8.

Hong Q, Pluye P, Fàbregues S, Bartlett G, Boardman F, Cargo M, et al. Mixed Methods Appraisal Tool (MMAT), Version 2018. User guide. McGill [Internet]. 2018;1–11. Available from:  http://mixedmethodsappraisaltoolpublic.pbworks.com/w/file/fetch/127916259/MMAT_2018_criteria-manual_2018-08-01_ENG.pdf%0Ahttp://mixedmethodsappraisaltoolpublic.pbworks.com/ 

Titler MG. The Iowa Model of evidence-based practice to promote quality care. 2001.

Dobrow MJ, Goel V, Upshur REG. Evidence-based health policy: context and utilisation. Soc Sci Med [Internet]. 2004;58(1):207–17. Available from: http://www.sciencedirect.com/science/article/pii/S0277953603001667

Kohatsu ND, Robinson JG, Torner JC. Evidence-based public health: An evolving concept. Am J Prev Med. 2004;27(5):417–21.

PubMed   Google Scholar  

Aas RW, Alexanderson K. Challenging Evidence-based Decision-making: A Hypothetical Case Study about Return to Work. Occup Ther Int. 2012 [cited 2018 May 28];19(1):28–44. Available from: https://doi.org/10.1002/oti.326

Ward M. Evidence-informed decision making in a public health setting. Healthc Manag Forum. 2011;24(1 SUPPL):S8.

Champagne F, Lemieux-Charles L, Duranceau M-F, MacKean G, Reay T. Organizational impact of evidence-informed decision making training initiatives: A case study comparison of two approaches. Implement Sci [Internet]. 2014;9(1). Available from: https://www.scopus.com/inward/record.uri?eid=2-s2.0-84900004312&doi=10.1186%2F1748-5908-9-53&partnerID=40&md5=df5573f17e19ef002d67cf669059aef4

Wright AL, Zammuto RF, Liesch PW, Middleton S, Hibbert P, Burke J, et al. Evidence-based Management in Practice: Opening up the Decision Process, Decision-maker and Context. Br J Manag. 2016;27(1):161–78. Available from: https://doi.org/10.1111/1467-8551.12123

Stamatakis KA, Ferreira Hino AA, Allen P, McQueen A, Jacob RR, Baker EA, et al. Results from a psychometric assessment of a new tool for measuring evidence-based decision making in public health organizations. Eval Program Plann [Internet]. 2017 Feb [cited 2018 May 16];60:17–23. Available from: http://linkinghub.elsevier.com/retrieve/pii/S014971891630009X

Harris C, Allen K, Waller C, Dyer T, Brooke V, Garrubba M, et al. Sustainability in Health care by Allocating Resources Effectively (SHARE) 7: supporting staff in evidence-based decision-making, implementation and evaluation in a local healthcare setting. BMC Health Serv Res [Internet]. 2017 Dec 21 [cited 2018 May 19];17(1):430. Available from:  https://doi.org/10.1186/s12913-017-2388-8

Harris C, Garrubba M, Melder A, Voutier C, Waller C, King R, et al. Sustainability in Health care by Allocating Resources Effectively (SHARE) 8: Developing, implementing and evaluating an evidence dissemination service in a local healthcare setting. BMC Health Serv Res [Internet]. 2018;18(1). Available from: https://www.scopus.com/inward/record.uri?eid=2-s2.0-85042873576&doi=10.1186%2Fs12913-018-2932-1&partnerID=40&md5=2daa5bbd3ccddf299f7b9d527b6105af

Yost J, Dobbins M, Traynor R, DeCorby K, Workentine S, Greco L. Tools to support evidence-informed public health decision making. BMC Public Health [Internet]. 2014 Dec 18 [cited 2018 May 28];14(1):728. Available from: https://bmcpublichealth.biomedcentral.com/articles/ https://doi.org/10.1186/1471-2458-14-728

Janati A, Hasanpoor E, Hajebrahimi S, Sadeghi- H, Khezri A. An Evidence-Based Framework for Evidence-Based Management in Healthcare Organizations : A Delphi Study. Ethiop J Heal Sci. 2018;28(3):305–14.

Rosswurm MA, Larrabee JH. A Model for Change to Evidence-Based Practice. J Nurs Scholarsh. 1999;31(4):317–22.

Article   CAS   Google Scholar  

Falzer PR, Garman MD. A conditional model of evidence-based decision making. J Eval Clin Pract [Internet]. 2009 Dec [cited 2018 May 19];15(6):1142–51. Available from: http://doi.wiley.com/ https://doi.org/10.1111/j.1365-2753.2009.01315.x

Ginter PM, Duncan WJ, Swayne LE. Strategic management of health care organizations. 8th Edition. Hoboken: John Wiley & Sons, Inc. 2018. p. 40.

Humphries S, Stafinski T, Mumtaz Z, Menon D. Barriers and facilitators to evidence-use in program management: a systematic review of the literature. BMC Health Serv Res [Internet]. 2014 Dec 14 [cited 2018 May 16];14(1):171. Available from: http://bmchealthservres.biomedcentral.com/articles/ https://doi.org/10.1186/1472-6963-14-171

Liang Z, Howard PF, Leggat SG, Murphy G. A framework to improve evidence-informed decision-making in health service management. Aust Heal Rev [Internet]. 2012;36(3):284–9. Available from: https://www.scopus.com/inward/record.uri?eid=2-s2.0-84865450745&doi=10.1071%2FAH11051&partnerID=40&md5=c42d56774cd486b8df8a5e4afec33a6a

Kohn MK, Berta W, Langley A, Davis D. Evidence-based decision making in health care settings: From theory to practice [Internet]. Vol. 11, Advances in Health Care Management. Emerald Group Publishing Ltd; 215–234 p. Available from: 2011. https://doi.org/10.1108/S1474-8231(2011)0000011012 .

Armstrong R, Waters E, Moore L, Dobbins M, Pettman T, Burns C, et al. Understanding evidence: a statewide survey to explore evidence-informed public health decision-making in a local government setting. Implement Sci [Internet]. 2014;9(1):188. Available from: http://implementationscience.biomedcentral.com/articles/ https://doi.org/10.1186/s13012-014-0188-7

McDiarmid M, Sandra K, Binns M. Evidence-based administrative decision making and the Ontario hospital CEO: information needs, seeking behaviour, and access to sources. JCHLA / JABSC [Internet]. 2007 [cited 2018 May 28];28:63–72. Available from: https://ejournals.library.ualberta.ca/index.php/jchla/article/viewFile/24060/17884

Brownson RC, Allen P, Duggan K, Stamatakis KA, Erwin PC. Fostering more-effective public health by identifying administrative evidence-based practices: A review of the literature. Am J Prev Med [Internet]. 2012;43(3):309–19. Available from: https://www.scopus.com/inward/record.uri?eid=2-s2.0-84865688888&doi=10.1016%2Fj.amepre.2012.06.006&partnerID=40&md5=a843f04e58b619ab9355a98bbfeef51a

Moussata CO. Evidence-Based Management and its Influence on the Practices of Senior Leaders of Hospitals in the Denver Metropolitan Area [Internet]. ProQuest Dissertations and Theses. [Ann Arbor]: Colorado Technical University; 2017 [cited 2018 May 19]. Available from: https://search.proquest.com/docview/1967189422?accountid=41313

Ward M, Mowat D. Creating an organizational culture for evidence-informed decision making. Healthc Manag Forum [Internet]. 2012;25(3):146–50. Available from: https://www.scopus.com/inward/record.uri?eid=2-s2.0-84867778644&doi=10.1016%2Fj.hcmf.2012.07.005&partnerID=40&md5=42f06b7236a4a1cd6a4739c7e1900764

Sosnowy CD, Weiss LJ, Maylahn CM, Pirani SJ, Katagiri NJ. Factors affecting evidence-based decision making in local health departments. Am J Prev Med [Internet]. 2013 Dec [cited 2018 May 19];45(6):763–8. Available from: http://dx.doi.org/ https://doi.org/10.1016/j.amepre.2013.08.004

Pagoto SL, Spring B, Coups EJ, Mulvaney S. Barriers and Facilitators of Evidence-Based Practice Perceived by Behavioral Science Health Professionals. 2007;63(7):695–705.

Weiss L, Sosnowy C, Maylahn C, Katagiri N, Pirani S. Evidence-Based Decision Making in Local Health Departments Evidence-Based Decision Making in Local Health Departments. Front Public Heal Serv Syst Res. 2012;1(3):1–7.

Lester L, Haby MM, Chapman E, Kuchenmüller T. Evaluation of the performance and achievements of the WHO Evidence-informed Policy Network (EVIPNet) Europe. Heal Res policy Syst. 2020;18(1):109.

Aas RW, Alexanderson K. Challenging evidence-based decision-making: A hypothetical case study about return to work. Occup Ther Int [Internet]. 2012 Mar [cited 2018 May 28];19(1):28–44. Available from: https://doi.org/10.1002/oti.326

Daouk-Öyry L, Sahakian T, van de Vijver F. Evidence-Based Management Competency Model for Managers in Hospital Settings. Br J Manag. 2020;00:1–20.

Isfeedvajani MS. Evidence-Based Management and its Application in the Hospital Management Process. Hosp Pract Res [Internet]. 2018 [cited 2018 May 28];3(2):35–6. Available from: http://jhpr.ir/article_60533_8313dd6afa580ef2afa689e6984c8844.pdf

Armstrong R, Waters E, Moore L, Dobbins M, Pettman T, Burns C, et al. Understanding evidence: a statewide survey to explore evidence-informed public health decision-making in a local government setting. Implement Sci [Internet]. 2014 Dec 14 [cited 2018 May 28];9(1):188. Available from: https://doi.org/10.1186/s13012-014-0188-7

Uneke CJ, Sombie I, Johnson E, Uneke BI, Okolo S. Promoting the use of evidence in health policymaking in the ECOWAS region: the development and contextualization of an evidence-based policymaking guidance. Global Health [Internet]. 2020;16(1):73. Available from: http://www.embase.com/search/results?subaction=viewrecord&id=L632558076&from=export

Janati A, Hasanpoor E, Hajebrahimi S, Sadeghi-Bazargani H. Health care managers’ perspectives on the sources of evidence in evidence-based hospital management: A qualitative study in Iran. Ethiop J Health Sci [Internet]. 2017;27(6):659–68. Available from: https://www.ajol.info/index.php/ejhs/article/view/162555

Spiri WC, Kurcgant P, Pereira MV. Perception Of Nursing Middle Managers About The Evidence-Based Management. Int Arch Med [Internet]. 2017;10(February):11. Available from: http://imedicalsociety.org/ojs/index.php/iam/article/view/2311

Harris C, Allen K, Waller C, Green S, King R, Ramsey W, et al. Sustainability in Health care by Allocating Resources Effectively (SHARE) 5: developing a model for evidence-driven resource allocation in a local healthcare setting. BMC Health Serv Res. 2017;17(1):1–18.

Belay T, Mbuya N, Rajan V. Data utilization and evidence-based decision making in the health sector: survey of three Indian states [Internet]. 2009 [cited 2018 May 26]. Available from: https://openknowledge.worldbank.org/handle/10986/3161

Niedzwiedzka BM. Barriers to evidence-based decision making among Polish healthcare managers. Heal Serv Manag Res [Internet]. 2003 May 21 [cited 2018 May 19];16(2):106–15. Available from: https://doi.org/10.1258/095148403321591429

Peirson L, Ciliska D, Dobbins M, Mowat D. Building capacity for evidence informed decision making in public health: A case study of organizational change. BMC Public Health [Internet]. 2012;12(1):137. Available from: https://www.scopus.com/inward/record.uri?eid=2-s2.0-84857132983&doi=10.1186%2F1471-2458-12-137&partnerID=40&md5=68e18844316881fb3f2662477825d441

Buffett C, Ciliska D, Thomas H. Can I Use This Evidence in my Program Decision? Assessing Applicability and Transferability of Evidence. National Collaborating Centre for Methods and Tools (NCCMT) School of Nursing, McMaster University. 2007:1–22.

Buffet C, Ciliska D, Thomas H. It worked there. Will it work here? A tool for assessing applicability and transferability of evidence (A: When considering starting a new program). National Collaborating Centre for Methods and Tools (NCCMT) School of Nursing, McMaster University. 2011;(905):1–2.

Dobbins M, Jack S, Thomas H, Kothari A. Public health decision-makers’ informational needs and preferences for receiving research evidence. Worldviews Evidence-Based Nurs. 2007;4(3):156–63.

Wright AL, Zammuto RF, Liesch PW, Middleton S, Hibbert P, Burke J, et al. Evidence-based Management in Practice: Opening up the Decision Process, Decision-maker and Context. Br J Manag [Internet]. 2016 Jan [cited 2018 May 28];27(1):161–78. Available from: https://doi.org/10.1111/1467-8551.12123

Imani-Nasab MH, Seyedin H, Yazdizadeh B, Majdzadeh R. A Qualitative Assessment of the Evidence Utilization for Health Policy-Making on the Basis of SUPPORT Tools in a Developing Country. Int J Heal Policy Manag. 2017;6(8):457–65.

Lavis JN, Paulsen EJ, Oxman AD, Moynihan R. Evidence-informed health policy 2-Survey of organizations that support the use of research evidence. Implement Sci. 2008;3:54.

Bastani P, Kavosi Z, Alipoori S, Imani-Nasab MH. Evidence- based Policy and Decision-Making among Health Managers: A Case of Shiraz Univer- sity of Medical Sciences. GMJ. 2017;6(1):30–8.

Greaves DE. Evidence-based management of Caribbean health systems: barriers and opportunities. Int J Health Governance. 2017;22(2):104–117.

Download references

Acknowledgements

This research, derived from Proposal No. 96-01-07-14184, was conducted by Mrs. Tahereh Shafaghat as part of the activities required for a Ph.D. degree in health care management at the Shiraz University of Medical Sciences. The authors wish to express their sincere gratitude to the research administration of Shiraz University of Medical Sciences for its financial and administrative support and to the English editorial board of Research Editor Institution for improving the native English language of this work.

As the overall study was an approved research project of Shiraz University of Medical Sciences and it was conducted by Mrs. Tahereh Shafaghat as part of the activities required for a Ph.D. degree in the health care management field, the Shiraz University of Medical Sciences supported this study. This study was sponsored by Shiraz University of Medical Sciences under code (96‑01‑07‑14184). The funding body was not involved in the design of the study, data collection, analysis, and interpretation, as well as in writing the manuscript.

Author information

Tahereh Shafaghat and Peivand Bastani have equal participation as first authors.

Authors and Affiliations

School of Management and Medical Informatics, Health Human Recourses Research Center, Shiraz University of Medical Sciences, Shiraz, Iran

Tahereh Shafaghat, Peivand Bastani & Mohammad Amin Bahrami

Department of Health Care Management, School of Public Health, Health Policy and Management Research Center, Shahid Saoughi University of Medical Sciences, Yazd, Iran

Tahereh Shafaghat & Mohammad Kazem Rahimi Zarchi

Faculty of Health and Behavioral Sciences, School of Dentistry, University of Queensland, QLD, 4072, Brisbane, Australia

Peivand Bastani

Social Determinants of Health Research Center, Lorestan University of Medical Sciences, Khorramabad, Iran

Mohammad Hasan Imani Nasab

Student Research Committee, School of Management and Medical Informatics, Shiraz University of Medical Sciences, Shiraz, Iran

Mahsa Roozrokh Arshadi Montazer

Faculty of Medicine, Center for Health Services Research, The University of Queensland, Brisbane, Australia

Sisira Edirippulige

You can also search for this author in PubMed   Google Scholar

Contributions

PB and TSH designed the study and its overall methodology. BP also edited and finalized the article. TSH searched all the databases, with the help of MRAM retrieved the sources, scanned, and screened all the articles in 3 phases. TSH also prepared the draft of the article. MAB and MKRZ contributed to data analysis and synthesis. Also, the study was under consultation and supervision by ZK and MHIN as advisors. All the authors have read and approved the final manuscript.

Corresponding author

Correspondence to Peivand Bastani .

Ethics declarations

Ethics approval and consent to participate.

Since at this study a scoping review was conducted and then the best-fit framework synthesis was used for developing a comprehensive EBDM framework in HSM, there was no human or animal participant in this study. However, the overall research project related to this manuscript was approved by the medical ethics conceal of the research deputy of Shiraz University of Medical Sciences with approval number IR.SUMS.REC.1396–01-07–14184.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Tables 3 , 4 , 5 and 6 .

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Shafaghat, T., Bastani, P., Nasab, M.H.I. et al. A framework of evidence-based decision-making in health system management: a best-fit framework synthesis. Arch Public Health 80 , 96 (2022). https://doi.org/10.1186/s13690-022-00843-0

Download citation

Received : 30 October 2021

Accepted : 08 March 2022

Published : 29 March 2022

DOI : https://doi.org/10.1186/s13690-022-00843-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Health system
• Best-fit framework synthesis

Archives of Public Health

ISSN: 2049-3258

• General enquiries: [email protected]

• Search Menu
• Advance articles
• Editor's Choice
• 100 years of the AJE
• Collections
• Author Guidelines
• Submission Site
• Open Access Options
• About American Journal of Epidemiology
• About the Johns Hopkins Bloomberg School of Public Health
• Journals Career Network
• Editorial Board
• Advertising and Corporate Services
• Self-Archiving Policy
• Dispatch Dates
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

Epidemiology: an information science, epidemiology: working with complex systems, epidemiology: a professional practice area, concluding remarks, epidemiology: a problem-solving journey.

• Article contents
• Figures & tables
• Supplementary Data

Haroutune K. Armenian, Epidemiology: A Problem-solving Journey, American Journal of Epidemiology , Volume 169, Issue 2, 15 January 2009, Pages 127–131, https://doi.org/10.1093/aje/kwn311

• Permissions Icon Permissions

As a scientific discipline, epidemiology has helped liberate the practice of public health and medicine from dogmatic thinking over the past century. This commentary highlights some integrating principles to explain why epidemiology is a problem-solving discipline. The first of these is that epidemiology is an information science. Epidemiology generates information for decision-making at all levels of the health-care system, including information for both individuals and the general public. Although all scientific disciplines produce data that may be used for decision-making, there is more immediacy for the decisions in epidemiology. The second principle is that epidemiology operates within an environment of complex systems. Etiologic factors operate in complex systems, and the use of a systems analysis approach in investigating health problems must be considered. The third principle is that epidemiology is not just a scientific discipline but a professional practice area as well. Epidemiology has a solid disciplinary scientific base, and its practice requires well-grounded academic preparation. Its objectives are very much within the public–social domain, and a well-defined, outcome-oriented, prevention-based philosophy steers its practice. In a number of universities, epidemiology is taught today as a research discipline rather than as an operational, problem-solving one. In conclusion, this commentary emphasizes the need to accept the fact that epidemiology has as much of a social role as a scientific one. Public health action, problem solving, and a sense of mission are what brings many students to epidemiology. A problem-solving, action-oriented epidemiology is consistent with that sense of mission.

Epidemiology has been and continues to be a journey through time, place, and with persons. There are different pathways that have led people to choose epidemiology as a career. The initial excitement that draws many to epidemiology is a medical or public health problem that needs to be addressed or solved on a broader level—going beyond the individual person experiencing the problem.

Epidemiology is a wonderful expression of our civilization of the past 2 centuries. Civilizations are in constant flux. The search for solutions that address problems of the past, present, and future marks the most dynamic of societies. As a discipline, epidemiology has faced the problems of the present. It has tried to learn from the past in order to deal with the present but also to redirect the future. Although it deals mainly with the problems of the present, its raison d’être is very much dictated by the positive impact it can make on the future of persons without distinction of geography or demography.

The uses of epidemiology can not be selective, and openness is what helps us to both investigate a problem and look for its solutions. One wonders why epidemiology has really blossomed in countries like the United States and the United Kingdom more than in other countries over the past century. The frame of comparison includes such countries as Germany, the Soviet Union, and Japan—countries where physical and other sciences have done very well. Does it have something to do with the liberal environment of the United States with an intellectual milieu that challenges established schools of thought? As a scientific discipline, epidemiology is liberating because it does not accept dogma. It has helped liberate the practice of public health and medicine from dogmatic thinking over the past century. One has to get concerned whenever there is a colleague who embraces a finding or method too religiously. The last 2 decades have highlighted the need for Popperian thinking in epidemiology ( 1 ). Refutation of the existing way of thinking should very much be in our focus of attention. A hypothesis can never be proven. However, there are hypotheses that have never been rejected—so far. Being inquisitive and challenging existing paradigms have been very much a part of epidemiology.

I have been professing the following integrating principles in my teaching over the years to explain why epidemiology is a problem-solving discipline.

The first integrating principle is that epidemiology is an information science. Epidemiology generates information for decision-making at all levels of the health-care system, including information for both individuals and the general public. Although all scientific disciplines produce data that may be used for decision-making, there is more immediacy for some of the decisions stemming from epidemiologic problem solving. There needs to be some immediate or long-term utility for the data generated in epidemiology. Such a perspective encourages us to target information to the appropriate audience, as well as to present it with a clear definition of the value and potential applications of the data generated. Therefore, epidemiology is purposive . Whether it is prevention of disease, disability, or death, there is a societal responsibility in epidemiology.

Our starting point is the problem, and that is why we need to continue emphasizing the need for the case-control and other case-based methods, because they deal with the problem at hand: people who have disease, disability, or distress. The case-control method is a good starting point; there is immediacy in its approach. We start by identifying the problem; defining it, we investigate it and try to link it to its antecedents. Then hopefully, as a result of our investigation, we can propose some rational solutions to the problem.

Thus, addressing the problem is our number 1 public health and moral responsibility as health professionals working in epidemiology. We need to identify the best tools that help us deal with the problem at hand. There is a continuous evolution of methodologies and approaches that will be integrated in epidemiology, enriching its practice in the process. A few decades ago, medicine and microbiology were very much at the core of epidemiologic investigation, whereas today the presence of statistics in the practice of epidemiology has been overwhelming. We should not allow one set of tools or methods from another disciplinary area to dominate epidemiology. For almost 100 years, that was the case for microbiology, which so dominated epidemiologic investigation that, in many countries and during long decades, one could not differentiate between the epidemiologist and the microbiologist by what they did. We should not let that happen with statistics in epidemiology.

Epidemiology aims at a synthesis of knowledge and a meeting place of multiple scientific disciplines and tools. It is through epidemiology that the clinical disciplines, social sciences, statistics, genetics, microbiology, biochemistry, environmental and management sciences, and so on are brought under the same roof for solving the complex public health and health services problems facing us. Information from a variety of these disciplines needs to be culled together for decision-making.

Thus, it is essential to have good epidemiologic intelligence prior to engaging in the direction of intervention. Epidemiology is about asking the right questions and knowing the different ways by which you can get answers to your questions. Some 30 years ago, as an epidemiologist working in the Ministry of Health of Bahrain at a time when there was active international discussion about implementing primary health care within the framework of Health for All by the Year 2000, I asked myself about what constituted the evidence that primary health care works. The database for such a worldwide policy recommendation was very meager. At the time, an example that documented the effectiveness of primary health care was the Kavar Village Health Worker Project in Iran. In this project, a positive impact on infant mortality and crude death rates was demonstrated in the villages that were recipients of the primary health-care worker program compared with other villages that did not have such a program ( 2 ). Thus, in a way, the authors of that report from Iran saved the day for a major international policy decision that was being implemented with little supporting information.

The second integrating principle is that epidemiology operates within an environment of complex systems. The simple models of one major etiology and one outcome have allowed us to target a number of public health prevention efforts to these specific etiologies. Examples abound of these simple models from infectious diseases and cancer. Our etiologic investigations continue to have the focus of simple models even if we use multivariate analyses. Most multivariate analyses are done for adjustments or control of confounding rather than for how variables interact in more complex models.

Etiologic factors operate in complex systems, and we need to consider the use of a systems-analysis approach in investigating etiology. In such an approach, the focus of attention is sets of interacting entities or subsystems. Over the past few years, there have been a number of attempts at incorporating a systems approach to broader research in epidemiology. One of the most interesting articles I have come across on the subject was published 28 years ago in a phytopathology journal. The paper, entitled “Systems Analysis in Epidemiology” ( 3 ), described the use of systems analysis methods to control diseases in plant populations.

Systems are situated within different levels of organization. Epidemiologists, sometimes, need to work at multiple levels to make the appropriate inferences. Epidemiology has worked primarily at the level of comparison between persons and not between populations or within persons. The individual person or case is a level of organization in its own right, and our investigation of the individual case may also give us leads to new knowledge in epidemiology ( 4 ). Epidemiology needs to navigate and operate at multiple systems levels.

As a physician working in this health center, I am not just interested in the trends and distribution of the disease, but I want first to know individually who are my diabetic patients and what is being done to them.

In dealing with these complex systems, we need to innovate our methods. We strive for simple approaches when, in spite of the infinite number of configurations that combinations of time, place, and persons provide us, we continue to be fixated on 1 subgroup of the population for decades to study a problem or sometimes multiple problems within the same data pool. Are we losing something of the richness of expressions of the problems as they are evolving all around us? With the exponential growth of databases at the personal and group levels, are we losing every day and every hour opportunities of problem solving through potential innovation by using alternative methods and databases?

It is not necessary to have the ultimate in technology for us to innovate. Much of the world will continue for decades not to have access to such sophisticated technology. One of the problems in public health program development, planning, and policy setting that we face in many of the areas outside Western Europe and North America is the absence of valid and dependable records that will allow definition of priorities and review of time trends of demographic, mortality, and morbidity indicators. Some alternatives to state-run databases can be identified that could provide such information. One such alternative is the records of church parishes. For example, Armenian churches have recorded information about deaths, marriages, and baptisms in their parishes for over 300 years from about 24 countries of the diaspora.

Beginning in the early 1980s, we conducted a series of studies of patterns of infant mortality, general mortality trends, widowhood and mortality, and epidemics using these parish records as our primary source of data ( 5–7 ). The small communities of the diaspora served as a microcosm that reflected what was occurring in the larger societies that surrounded them. Thus, we were able to identify 2 epidemics of what was probably influenza in the small Armenian parish of Belgrade in the early 18th century and 3 epidemics of cholera in the Armenian Catholic parish records of Kutahya, Turkey, in the mid-19th century—the same time period that Snow was investigating for cholera in London. We were able to get an estimate of the speed with which the great influenza pandemic progressed from Dakka, Bengal, to Cairo, Egypt, in 1918 using, again, the Armenian parish records ( 6 ).

My third integrating principle is that epidemiology is not just a scientific discipline but a professional practice area. While we philosophize about our discipline in our academic ivory towers, let us remember that our colleagues in various health services positions practice epidemiology on the ground floor.

But what do we mean by “professional practice”? Definitions of the word “profession” emphasize the need for specialized knowledge, academic preparation, and an altruistic spirit. One of my medical school teachers, John Racy, wrote an article on the subject decades later. He defined a profession as “a socially sanctioned activity whose primary object is the well-being of others above the professional's personal gain” (8, p. 139). A profession is much more than a job; it is an identity where the giving supersedes and precedes the gain. A profession also uses well-defined paradigms or common approaches for practice.

Epidemiology has a solid, disciplinary scientific base, and its practice requires well-grounded academic preparation. Its objectives are very much within the public–social domain, and a well-defined, outcome-oriented, prevention-based philosophy steers its practice. Further, epidemiology uses well-defined paradigms of problem investigation, analysis, and inferences.

This professional practice orientation of epidemiology, which has helped us not just to address real life problems but also to engage in problem solving, has sharpened our tools and helped us develop new methods. There is a lot that can be said as to how the human immunodeficiency virus epidemic helped introduce new methods in infectious disease epidemiology over the past 25 years.

As a rotating intern in pediatrics and as a resident in internal medicine, I had to deal with a recurrent episodic condition that affects primarily people of Middle Eastern origin: familial paroxysmal polyserositis or familial Mediterranean fever. Within a decade of our getting involved in research about familial Mediterranean fever, the genes of the disease were identified. Our knowledge of the genes was really not very helpful to the patients who were suffering from the painful and debilitating episodes of illness. Thus, it was important to consider that, in addition to the genes, one may be able to prevent the attacks if there are external precipitants for these attacks to occur. It was very logical to think of the atopic model for the etiology of this disease where precipitants are needed for the phenotypic expression of the genetic predisposition. In order to investigate such a hypothesis, we worked on a research proposal in 1975 that would have studied antecedents and exposures in the same patient during the attack and at a random day when the patient was attack free. This was the first time we were thinking of a research design similar to what Malcolm Maclure later described as the case-crossover model ( 9 ). Unfortunately, with the eruption of the civil war in Lebanon that same year, we had to abandon that project. This project is currently being pursued by Dr. Gayane Yenokyan in Armenia for her doctoral thesis research at Johns Hopkins University.

The use of epidemiology may help us redirect our problems toward simpler solutions. A lot of effort has been invested over the past 100 years in trying to develop effective interventions against trachoma. We have a clear understanding of the pathobiology of the agent and its transmission. For years, efforts were made to test vaccines against the disease, as well as other medicinal based interventions. The disease continues to be an endemic, chronic problem for hundreds of millions around the world, causing blindness in thousands. However, the public health problem of trachoma was solved on its own in a number of communities in the Middle East and elsewhere by providing clean running water to everyone and improving sanitary habits. A review of the epidemiology of trachoma could have led epidemiologists to recommend these most effective of public health interventions—clean water and sanitary habits—to yet another major endemic disease several decades ago.

One of the essential characteristics of good professional practice is good management. Epidemiology has relevance beyond disease etiology, and epidemiologic thinking can be critical for good operational management and not just for planning and evaluation ( 10 ). Beyond efficacy and effectiveness, epidemiology has a lot to offer for efficient management. The discipline of thinking and analysis that are part of the core of epidemiologic methods can be very useful for professional practice.

Although, currently, epidemiology has developed into several divisions or specialized areas, there is a definite place for generalists. As a professional, one should not be selective as to what problems we will accept to tackle. The problems dictate what we practice.

Epidemiology is taught today as a research discipline rather than as an active problem-solving one. None of the introductory textbooks of epidemiology spends an adequate amount of space in elaborating on the professional practice areas of epidemiology. Professional practice is no more at the core but at the periphery. If you want to learn about the practice of epidemiology, you need to read some “applied epidemiology” texts rather than the main textbooks or take a course in the applications of epidemiology.

There are relatively fewer young physicians and other health professionals who are joining epidemiology today compared with a couple of decades ago. Is this because the discipline has strayed away from its core practice areas and is presented more and more as a research discipline? Why do a number of our graduates feel the need to spend a few years with the Epidemic Intelligence Service at the Centers for Disease Control and Prevention after spending 4–6 years in our academic departments for a doctoral degree?

Epidemiology cannot exist without being engaged in the broader public health and health services mission. We have to believe in not just the future of epidemiology as a discipline but also our common capacity of achieving our objectives of a humanity living in dignity. When we are inspired and driven by the potential impact of what we can achieve, then we can make a great leap forward for the discipline.

We need to accept and be comfortable with the fact that epidemiology has as much of a social role as a scientific one.

The great founder of the Institute of the History of Medicine at Johns Hopkins, Henry Siegerist, has stated that one of the problems of medicine through the ages has been that technology has always outpaced sociology ( 11 ). We have new knowledge and technology that take decades and sometimes centuries to make a part of the social fabric of the profession. Is that what is happening in epidemiology? I believe, today, that epidemiology is in need of sociology more than additional technology. Sociology in epidemiology is in the context of its uses and its practice within the framework of health services. This is not a plea for more studies on social factors in health and disease but for a redirection of epidemiology toward its social–professional roles within the health services and at all levels.

We need to define not an overarching set of methods but overarching social impact objectives for the discipline. Without its social dimension, epidemiology loses its soul.

Over the past century or so, we have aimed at preventing disease, disability, and death (DDD), and we have been quite successful in minimizing the load of the DDDs. Whether as individuals or as a society, we need to realize the inevitability of DDD for most people. Our immediate and long-term concern has been the minimizing of pain and suffering for our fellow human beings, because those with pain and suffering lose so much of their integrity and dignity as human beings. Thus, the ultimate objective of our aggregate effort is human dignity. We aim at helping the individual and the group in improving their dignity. Ultimately, this is every profession's aim.

We agree that human beings are not just a collection of cells or molecules and that they also have spirituality that binds the molecules and cells with an integrative purpose and resultant direction. Thus, in every culture and with every individual, there is this search for dignity that elevates us out of our biologic complexity. Dignity is very much an important dimension of our understanding of health.

Public health action, problem solving, and a sense of mission are what brings many of our students to epidemiology. They come to us in search of that soul of epidemiology, and that soul is where the action is. Today, the best expression of that soul is in the global dimension of epidemiology and the local engagement in our communities. Eliminate these, and epidemiology ends up becoming soulless numbers and technology.

It will be interesting to ask: Who are the heroes of our current students? I have a sense that they are mostly the theoreticians who have had very little to do with the practice of epidemiology. These are also the same people who today dictate the direction in which epidemiology develops and is taught.

In the final analysis, the question of why bad things happen to good people translates itself into some very different questions, no longer asking why something happened, but asking how we will respond, what we intend to do now that it has happened.

Abbreviation

disease, disability, and death

Author affiliation: Department of Epidemiology, the Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland.

This commentary was part of a talk at a symposium entitled, “Beyond Investigation: Epidemiology as a Problem Solving Discipline,” organized by the Department of Epidemiology of the Johns Hopkins Bloomberg School of Public Health on December 14, 2007.

Conflict of interest: none declared.

Google Scholar

Google Preview

• epidemiology
• problem solving
• public health medicine

Email alerts

Citing articles via, looking for your next opportunity.

• Recommend to your Library

Affiliations

• Online ISSN 1476-6256
• Print ISSN 0002-9262
• Copyright © 2024 Johns Hopkins Bloomberg School of Public Health
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Methodological innovations in public health education: transdisciplinary problem solving

Affiliation.

• 1 Edward F. Lawlor, Matthew W. Kreuter, Anne K. Sebert-Kuhlmann, and Timothy D. McBride are with the Brown School, Washington University, St. Louis, MO.
• PMID: 25706031
• PMCID: PMC4340002
• DOI: 10.2105/AJPH.2014.302462

In 2008, the faculty of the Brown School at Washington University in St. Louis designed a Master of Public Health program centered on transdisciplinary problem solving in public health. We have described the rationale for our approach, guiding principles and pedagogy for the program, and specific transdisciplinary competencies students acquire. We have explained how transdisciplinary content has been organized and delivered, how the program is being evaluated, and how we have demonstrated the feasibility of this approach for a Master of Public Health degree.

• Competency-Based Education*
• Education, Public Health Professional / methods*
• Education, Public Health Professional / organization & administration*
• Interdisciplinary Communication*
• Models, Educational
• Organizational Culture
• Problem-Based Learning*
• Program Development
• Schools, Public Health / organization & administration
• Staff Development

CURRICULUM, INSTRUCTION, AND PEDAGOGY article

Health design thinking: an innovative approach in public health to defining problems and finding solutions.

• 1 Jefferson College of Population Health, Thomas Jefferson University, Philadelphia, PA, United States
• 2 Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, PA, United States

Current trends in the United States health care landscape call for innovative and adaptive approaches to improve outcomes and reduce inefficiencies. Design Thinking is an innovative approach to problem-solving that leverages insights from the end-users of new products, services, and experiences in order to develop best-fit solutions that are rapidly prototyped and iteratively refined. When compared to traditional problem-solving methods in health care and other public health adjacent fields, Design Thinking leads to more successful and sustainable interventions. Design Thinking has facilitated improvements in patient, provider, and community satisfaction, and in public health, has increased efficiency and collaboration in intervention development. Given the promising nature of Design Thinking as an effective problem-solving method, it follows that Design Thinking training would prove a beneficial addition to public health education. The integration of Design Thinking in public health education may equip public health leaders with essential skills necessary to understand and more effectively approach historically intractable challenges. This article describes the development and evaluation of a hands-on Design Thinking workshop, piloted with Master of Public Health (MPH) students in April, 2019 at Thomas Jefferson University. Preceding and following the workshop, evaluation forms were used to assess participants' knowledge about Design Thinking concepts and attitudes towards the workshop experience. Metrics were aligned with established learning objectives related to process, impact, and outcomes of the workshop. We hypothesized that the workshop intervention would increase participants understanding of Design Thinking concepts and applications in public health. Evaluations demonstrated that after attending the workshop, participants were able to understand and apply Design Thinking concepts in a public health context. Following the evaluation of pilot data, the workshop was refined and embedded in the MPH curriculum at Thomas Jefferson University in Philadelphia, PA.

Introduction

Despite rapid advancements in research and technology, the United States health care system continues to endure pervasive inefficiencies including inequitable access, inconsistent quality, and high costs relative to comparable nations ( 1 ). Evidence suggests that innovation is an essential competency among the health care workforce to increase productivity and address these inefficiencies ( 2 ). Innovation is uniquely challenging in public health, as problems tend to be complex, dynamic, and context-specific, and can at times arise quickly and unpredictably, raising the urgency for rapid and efficient responses ( 3 ). However, current educational models in health care and public health provide limited training in creative thinking and innovation skills ( 2 , 4 ). Thus, traditional public health education may be augmented through the inclusion of innovative, non-linear, adaptive, and cost-effective tools ( 5 – 8 ).

Design Thinking is one such tool; it is an innovative approach to problem solving that leverages insights from the end-users of new products, services, and experiences in order to develop best-fit solutions that are rapidly prototyped and iteratively refined so they can be deployed quickly and cost-effectively. It is a “human-centric” approach that involves collaboratively generating solutions alongside intended audiences, empathizing, developing a clear and succinct problem definition, creative ideation, and low-fidelity prototyping ( 9 , 10 ). Design Thinking guides the early phases of innovation through deep empathy for users and a clear understanding of the problems facing them ( 11 ).

Frequently applied in industries such as business and technology, a Design Thinking approach recognizes that only an approximate 10% of new products or services successfully identify and respond to end-users' needs, meaning that the other 90% result in wasted time, funding, and other resources ( 11 ). In health care, Design Thinking has facilitated improvements in patient, provider, and community satisfaction, and can increase the efficiency and collaborative nature of intervention development ( 9 , 12 ). When compared to traditional problem-solving methods in health care and public health, Design Thinking has demonstrated greater empathy for the needs of a community, a clearer understanding of the problem, more resource-efficient and cost-effective processes, and solutions with greater end-user satisfaction ( 12 – 15 ). For example, this approach has translated to increased use of public park spaces and improved efficacy of app-based behavior change interventions ( 12 , 15 ). Furthermore, Design Thinking is well-suited to problem solving in the built environment, serving as a tool to combat health inequities and issues rooted in social determinants of health ( 9 , 16 ). In partnership with the Center for Social Design at the Maryland Institute College of Art (MICA), the Baltimore City Health Department applied Design Thinking methods to support families with smoking cessation. They were trained in Design Thinking methods and interviewed families to understand barriers and gain empathy, which enabled them to develop an interactive pop-up event providing interventions informed by the needs of the target community and health behavior change theory, and supporting families in creating smoke-free zones in their homes ( 9 ). In another collaboration with MICA's Center for Social Design, Johns Hopkins Children's Center tackled pediatric asthma, which affects twice as many Baltimore children as the national average and disproportionately impacts black children. Their process involved observations and ethnographic research, followed by open-ended interviews with adolescent asthma patients, along with caregivers, advocates, and providers. Their ideation resulted in over 200 potential solutions ( 9 ).

Given the promising nature of Design Thinking applications in public health, it follows that Design Thinking training would prove a beneficial addition to public health curricula. Design Thinking fits neatly into formal public health education, which already emphasizes communication, teamwork, and qualitative approaches such as Community-Based Participatory Research (CBPR) ( 17 ). Design Thinking is well aligned with competencies outlined by the Council on Education for Public Health (CEPH), can be used to complement traditional public health methods, and prepares students to apply innovative and creative problem-solving methods to address challenges related to quality, cost, and access ( 18 , 19 ).

This article describes the development and evaluation of a hands-on Design Thinking workshop, piloted with Master of Public Health (MPH) students at Thomas Jefferson University. This cross-disciplinary initiative aimed to provide Design Thinking training to public health students, preparing them to tackle complex problems efficiently and effectively as future professionals and providers. It challenged participants to engage directly with and apply material to public health problems. The goal of the endeavor was to integrate Design Thinking training as a core component of public health education in order to inspire widespread use of more systematic and effective approaches, and equip future leaders with innovative tools to improve the health of individuals and communities. The workshop provided an initial exposure to Design Thinking tools and applications, and aimed to increase participants' knowledge about Design Thinking while inspiring further curiosity about the pursuit of innovative methodologies in public health.

Design and Development

The workshop was developed through a collaboration between the Health Design Lab and Colleges of Medicine and Population Health at Thomas Jefferson University to supplement traditional public health education. It was created by drawing upon the curriculum used in the Health Design Lab at Thomas Jefferson University and adapting the content for a public health audience. The participant worksheets and substantial content in the slide deck were adapted from the Design Thinking “Crash Course” and other materials openly accessible online from the Hasso Plattner Institute of Design at Stanford University ( 10 ). Both the Health Design Lab and Hasso Plattner Institute of Design granted permission for their content to be used.

The evidence-based workshop aims to teach Design Thinking within the context of public health. Principles from Health Design Thinking ( Figure 1 ) and Stanford University's Hasso Plattner Institute of Design were used as conceptual structures ( 9 , 10 ). The workshop occurs in person over the course of approximately two hours. It includes a brief, didactic introduction to Design Thinking methodology and reviews case studies demonstrating real-world applications of Design Thinking in public health, such as improving public park spaces and addressing social determinants of infant mortality inequities ( 12 , 20 ). The remainder of the session involves hands-on activities centered on a design challenge related to health behavior change. Participants are paired into groups of two and guided through each of five stages in the Design Thinking process, beginning with interviewing one another to gain empathy into the other's barriers and facilitators to change, and concluding with prototyping and testing their innovative ideas. The learning objectives include several key concepts of Design Thinking, identified in Table 1 . See Supplementary Materials for workshop agenda.

Figure 1 . Fundamental principles and conceptual structure from Health Design Thinking ( 9 ).

Table 1 . Key concepts included in Design Thinking workshop ( 9 , 10 ).

Implementation

The workshop was piloted and evaluated on April 19, 2019 at Thomas Jefferson University from 12 p.m. to 3 p.m. with current public health graduate students. The pilot took place in the Health Design Lab space at Thomas Jefferson University. The lab space has several rectangular tables, each of which accommodated two individuals, therefore giving each participant ample space to sit and work creatively. Additionally, the Health Design Space offered use of their “prototyping cart,” which contained ample crafts supplies and facilitated low-fidelity prototyping.

Participants

Ten current Thomas Jefferson University Master of Public Health (MPH) students participated in the workshop pilot. Participants were recruited from all “tracks” to MPH completion at Jefferson, including an accelerated one-year track, a part-time track, a full-time track, and a dual degree (MD/MPH) track. All students pursuing an MPH who expressed interested were included. A small sample size (ten participants) was chosen because experts suggest small groups are best for an effective Design Thinking workshop ( 9 ). MPH students were recruited via email, newsletter, messenger application, and word of mouth. They were compensated with lunch and clerkship credit hours that counted towards their degree requirements. In addition, several faculty members were present during the pilot to support, observe, and offer feedback for future iterations of the workshop.

Evaluation Methods

Evaluations were collected using a paper survey that participants completed immediately before and after the workshop in order to assess their knowledge about Design Thinking and attitudes about the workshop experience. All survey data were anonymous and no identifying information was collected. The attitudes measured included interactivity, engagement, enjoyment, and learning. Interactivity, engagement and enjoyment were operationalized by asking participants to self-report their opinions of the workshop across each dimension using a Likert scale. For example, enjoyment was measured using the survey item: “Please rank your agreement with the following statement (1 = Strongly Disagree, 5 = Strongly Agree): I enjoyed the Design Thinking workshop.”

The attitudes items were only assessed after the workshop. Knowledge was assessed directly with a pre and post “quiz” style survey. Questions included: “Please describe three aspects of Design Thinking” and “What is one way that Design Thinking could be used in your career as a public health professional?” In addition, participants were asked to open-endedly report any comments or suggestions they had following the workshop. See Supplementary Materials for all survey questions. The workshop was ultimately evaluated for its adherence to several objectives related to process, impact, and outcomes, which built off of the survey items and are identified in Table 2 .

Table 2 . Workshop objectives.

Curriculum Integration

Findings from the pilot were synthesized and the workshop delivery was refined accordingly prior to its integration into the curriculum at Thomas Jefferson University in the Fall of 2019. It is now offered as part of the Introduction to Public Health course, which is taken by all MPH students in the College of Population Health.

The survey evaluations generated both qualitative and quantitative findings. These data were analyzed, and despite the small sample size, proved to be valuable in refining the workshop prior to its integration into Jefferson's MPH curriculum.

Quantitative Findings

Findings from the quantitative items on participant surveys are summarized here. The results indicated that participants' familiarity with Design Thinking and its applications increased meaningfully and that the workshop was received positively ( Tables 3 , 4 ).

Table 3 . Participants' knowledge about Design Thinking.

Table 4 . Participants' attitudes about workshop experience.

Overall, responses to the knowledge items indicate that participants were unfamiliar with Design Thinking prior to the workshop and were able to identify key concepts and potential applications for these concepts immediately following the workshop. On average, participants' self-reported familiarity with Design Thinking increased from 2.3 to 4.4 on a Likert Scale of 1–5. Number of correctly identified Design Thinking concepts increased from 0.4 to 2.9 on a scale from 0 to 3. An increase from 50 to 100% was observed among participants who were able to identify a meaningful public health application of Design Thinking relating to their own academic or professional careers.

Additionally, responses indicate positive attitudes towards the workshop experience. All workshop attendants indicated “5 (Strongly Agree)” to each question regarding workshop attitudes, indicating that the workshop was clear, enjoyable, and interactive.

Qualitative Findings

Open-ended questions eliciting participants' comments and qualitative feedback indicated that the workshop was positively received. When asked what participants would change about the workshop, many said “nothing,” or “it was great.” Suggestions for improvements were limited. Participants were given approximately 10 minutes to create low-fidelity prototypes of their ideas. One participant requested additional time for prototyping a solution. Ideas prototyped included devices to assist with schedule management and mobile application interfaces to facilitate physical activity and dietary changes. Two participants indicated they would have liked an opportunity to share their prototypes. All other participants left spaces designated for constructive feedback and suggestions for improvements blank. Faculty feedback indicated that certain aspects of the directions and content needed clarification, including the roles of each partner during the design challenge, and the concept of “fail fast.” Following the pilot, all feedback was taken into consideration and workshop agenda, slides, and worksheets were refined accordingly.

This workshop module represents the first integration of Design Thinking training into the public health curriculum at Jefferson. The objective of the pilot was to demonstrate the feasibility and efficacy of a Design Thinking workshop in teaching MPH students key principles of Design Thinking. Our findings indicate that a workshop intervention can increase participants' knowledge of Design Thinking and its applications, and demonstrate the feasibility of integrating Design Thinking training into public health education. In addition, surveys revealed positive attitudes toward the workshop experience. This serves as a promising indication that students are likely to engage with and retain concepts, making them more likely to apply innovative approaches in public health. In order to build upon the success of this pilot and inspire widespread use of more innovative and empathy-driven approaches to improve the health of individuals and communities, this workshop was integrated into the core MPH curriculum at Jefferson.

There is additional potential to expand the content delivered in this workshop and further enhance its impact. The success of this pilot indicates that a full course on Design Thinking and other innovation tools, if developed within a public health program such as Jefferson's, might promote greater active learning and innovation among MPH students and future public health professionals. The Gillings School of Global Public Health at the University of North Carolina offers such a course, where students are taught creative prototyping, adopting an entrepreneurial mindset, and learning from failure ( 22 , 23 ). Additionally, it is likely that the education provided through this workshop would benefit other health professional audiences, including public health students at other universities, public health practitioners, and educators.

Ultimately, the findings from this pilot should encourage other public health programs and educators to consider implementing a Design Thinking framework within public health education. Applying Design Thinking to public health challenges can help students, practitioners and educators to creatively and collaboratively problem solve. Integrating Design Thinking within public health pedagogy has the potential to increase use of creative approaches to develop more innovative ideas and interventions.

Limitations

Although a small group size (10 participants in the pilot workshop) lent itself well to an interactive Design Thinking session, the amount of data collected was minimal and thus, statistical analyses could not be meaningfully performed. Additionally, the long-term impact of Design Thinking training on public health professionals' interventions and population level outcomes remains unclear. Finally, participants volunteered to participate in the session. It is possible that selection bias played a role in the responses to survey items, although we do not believe the findings to have been significantly influenced by bias.

Future Research

Current and future offerings of this workshop within Jefferson's MPH program will continue to generate data using the same pre- and post-session evaluations that were used in the pilot. These data are currently used to refine the workshop, iterating such that future sessions adapt to the observations, feedback, and needs of participants. Additional data collection also provides the opportunity for a more robust statistical analysis, which will generate further evidence around the effectiveness of this intervention.

One of the goals of this pilot was to inspire more widespread use of Design Thinking in public health education and practice; as more Design Thinking trained professionals enter the public health workforce, opportunities to compare the effectiveness of intervention impact between those who did and did not receive Design Thinking training will become increasingly feasible. These comparisons will determine whether or not the integration of Design Thinking training into public health education truly serves to address the many intractable health and health care challenges faced nationally, and globally, today.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author Contributions

SA developed and implemented the workshop and accompanying materials and drafted the manuscript. BK contributed to content of workshop and generated the idea for the manuscript. CP contributed to development and refinement of the workshop and materials and including pre- and post-workshop evaluations. RF contributed to workshop refinement and edited the final version of the manuscript. CP and RF provided feedback and guided revision and iteration of workshop following pilot and facilitated curriculum integration process. All authors contributed to drafting and revisions, approved the final version for publication, and agreed to be accountable for the content of the work.

Publication made possible in part by support from the Thomas Jefferson University Open Access Fund.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors would like to thank Robert Pugliese, Geoffrey Hayden, Kristy Shine, and the Health Design Lab team for their contributions to workshop content and for welcoming our workshops in their lab space. We would also like to thank Pankti Acharya, Rachel Caulkins, Emily Cziraky, Hannah Derwick, Valerie Guerrero, Sean McKnight, McKenzie Prunier, Fahmida Sarmin, Emerson Trimble, and Jonathan Zuckerman for their participation in the pilot workshop.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2020.00459/full#supplementary-material

Supplementary Table 1. Workshop agenda, concepts and activities.

Supplementary Table 2. Pre-workshop survey.

Supplementary Table 3. Post-workshop survey.

Supplementary Data Sheet 1. Participant worksheets.

1. Davis K, Stremikis K, Squires D, Schoen C, Mirror, Mirror on the Wall, 2014 Update: How the U.S. Health Care System Compares Internationally . Washington, DC: The Commonwealth Fund (2014).

Google Scholar

2. Bridging Health Care's Innovation-Education Gap. Available online at: https://hbr.org/2014/11/bridging-health-cares-innovation-education-gap (accessed July 19, 2020).

3. Neuhauser L, Kreps GL. Integrating design science theory and methods to improve the development and evaluation of health communication programs. J Health Commun. (2014) 19:1460–71. doi: 10.1080/10810730.2014.954081

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Pillay R, Morris M. Changing healthcare by changing the education of its leaders: an innovation competence model. J Health Admin Educ . (2016) 33:393–410.

5. DeSalvo KB, O'Carroll PW, Koo D, Auerbach JM, Monroe JA. Public health 3.0: time for an upgrade. Am J Public Health. (2016) 106:621–2. doi: 10.2105/AJPH.2016.303063

6. Lister C, Payne H, Hanson CL, Barnes MD, Davis SF, Manwaring T. The public health innovation model: merging private sector processes with public health strengths. Front Public Health. (2017) 5:192. doi: 10.3389/fpubh.2017.00192

7. Durovich CJ, Roberts PW. Designing a community-based population health model. Popul Health Manag. (2018) 21:13–9. doi: 10.1089/pop.2017.0015

8. Wingood GM, DiClemente RJ. Accelerating the evolution of health promotion research: broadening boundaries and improving impact. Am J Public Health. (2019) 109:S116. doi: 10.2105/AJPH.2019.304991

9. Ku B, Lupton E. Health Design Thinking: Creating Products and Services for Better Health . Cambridge: The MIT Press (2020).

10. An Introduction to Design Thinking - Process Guide. Hasso Plattner Institute of Design (2010) Available online at: https://s3-eu-west-1.amazonaws.com/ih-materials/uploads/Introduction-to-design-thinking.pdf (accessed December 15, 2018).

11. Rottenberg S, Hartung H. “Human-Centered Design: Understanding Customers' Needs Through Discovery and Interviewing,” Academic Entrepreneurship for Medical and Health Scientists . (2019). Available online at: https://repository.upenn.edu/ace/vol1/iss4/5

12. Huang TTK, Aitken J, Ferris E, Cohen N. Design thinking to improve implementation of public health interventions: an exploratory case study on enhancing park use. Design Health. (2019) 2:1–17. doi: 10.1080/24735132.2018.1541047

13. Altman M, Huang TTK, Breland JY. Design thinking in health care. Prev Chronic Dis. (2018) 15:E117. doi: 10.5888/pcd15.180128

14. Scholten H, Granic I. Use of the principles of design thinking to address limitations of digital mental health interventions for youth: viewpoint. J Med Internet Res. (2019) 21:e11528. doi: 10.2196/11528

15. Fjeldsoe BS, Miller YD, O'Brien JL, Marshall AL. Iterative development of MobileMums: a physical activity intervention for women with young children. Int J Behav Nutr Phys Act. (2012) 9:151. doi: 10.1186/1479-5868-9-151

16. Neuhauser L. Integrating participatory design and health literacy to improve research and interventions. Inf Serv Use. (2017) 37:153–76. doi: 10.3233/ISU-170829

CrossRef Full Text | Google Scholar

17. Israel BA, Coombe CM, Cheezum RR, Schulz AJ, McGranaghan RJ, Lichtenstein R, et al. Community-based participatory research: a capacity-building approach for policy advocacy aimed at eliminating health disparities. Am J Public Health. (2010) 100:2094–102. doi: 10.2105/AJPH.2009.170506

18. Accreditation Criteria – Schools of Public Health & Public Health Programs. Council on Education for Public Health (CEPH) (2016). Available online at: https://media.ceph.org/documents/2016.Criteria.pdf

19. Mummah SA, Robinson TN, King AC, Gardner CD, Sutton S. IDEAS (integrate, design, assess, and share): a framework and toolkit of strategies for the development of more effective digital interventions to change health behavior. J Med Internet Res. (2016) 18:e317. doi: 10.2196/jmir.5927

20. Vechakul J, Shrimali BP, Sandhu JS. Human-centered design as an approach for place-based innovation in public health: a case study from Oakland, California. Matern Child Health J. (2015) 19:2552–9. doi: 10.1007/s10995-015-1787-x

21. Leifer L, Plattner H, Meinel C editors. Design Thinking Research: Building Innovation Eco-Systems . Springer Science & Business Media (2013).

22. New Course at UNC Taught by Glenn Walters Combines Design Thinking and Experiential Making. Chapel Hill,NC: UNC Gillings School of Global Public Health (2018) Available online at: https://sph.unc.edu/sph-news/new-course-at-unc-taught-by-glenn-walters-combines-design-thinking-and-experiential-making/ (accessed June 4, 2019).

23. Ramaswamy R, Mosnier J, Reed K, Powell BJ, Schenck AP. Building capacity for Public Health 3.0: introducing implementation science into an MPH curriculum. Implement Sci. (2019) 14:18. doi: 10.1186/s13012-019-0866-6

Keywords: design thinking, human-centered design, education, public health, innovation

Citation: Abookire S, Plover C, Frasso R and Ku B (2020) Health Design Thinking: An Innovative Approach in Public Health to Defining Problems and Finding Solutions. Front. Public Health 8:459. doi: 10.3389/fpubh.2020.00459

Received: 04 May 2020; Accepted: 22 July 2020; Published: 28 August 2020.

Reviewed by:

Copyright © 2020 Abookire, Plover, Frasso and Ku. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Sylvie Abookire, spa010@jefferson.edu

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

• MyU : For Students, Faculty, and Staff

Advances Magazine

The magazine of the University of Minnesota School of Public Health

Solving Problems the Minnesota Way

The School of Public Health’s Master of Healthcare Administration (MHA) is perennially one of the top programs in the country and currently ranked #2 by U.S. News and World Report . One reason for its success is the method students are taught to solve major organizational problems. Called the Minnesota MHA Problem Solving Method, it is now being offered in a new textbook titled, Applied Problem-Solving in Healthcare Management .

The Minnesota Way

MHA program founder James A. Hamilton created the method in the 1940s. Hamilton was a renowned hospital operations consultant and one of the first healthcare administration educators. He realized that in order to solve problems, you need to define them well, so he developed a 14-step approach to define problems and identify realistic solutions.

“The Minnesota Problem Solving Method has been taught to generations of MHA students,” says MHA Program Director and Professor Jean Abraham . “It is an integral part of our curriculum and a point of distinction for us.”

Justine Mishek , co-author of the new textbook, is a senior lecturer for the MHA program and the lead instructor for teaching the approach. 

“The Minnesota Method teaches students to break down complicated problems and ask the right questions to properly define underlying issues,” says Mishek. “It also provides a common language that students, our alumni, and industry leaders can use to study and act upon problems.”

The modern iteration of the method is conducted in three major phases:

Define : Work with stakeholders, define problems, group them into categories, and develop key questions to ask and answer.

Study: Search for the root causes of problems, develop alternative solutions, and lead decision making to draw conclusions.

Act : Articulate final recommendations, determine key audiences to support solutions, gain acceptance of the remedies, and prepare for implementation. 

Learning from — and through — experience

An essential component to understanding the method is learning how it works with real-world problems. 

When Rachel Kuhnly (MHA ’19) was a student, Allina Health in Minneapolis asked her problem-solving team to evaluate if and how a hospital-at-home program would be helpful and feasible for the organization to offer. In such programs, patients who are mildly sick with certain conditions are cared for at home, virtually, and are monitored through electronic devices and telehealth visits, freeing up hospital beds for more critically ill patients. 

Kuhnly and her team used the Minnesota Problem Solving Method and proposed a hospital-at-home care model and specific implementation considerations to Allina.

“With the method, we were able to understand the problems associated with a hospital-at-home program and what to recommend that would be advantageous for Allina as well as patients and families,” says Kuhnly. 

Allina leadership was happy with the plan, with a final decision and implementation projected far into the future. Then COVID-19 struck. A hospital-at-home program turned out to be a good solution to keep patients safely cared for at home and keep hospital beds available for COVID-19 patients. Allina asked Kuhnley to lead the implementation less than a month after the first COVID-19 case in Minnesota.

Getting it in Writing

Given the success of the method, alumni and instructors in other MHA programs around the country wanted to use it in their own courses and training programs. The textbook was born. 

To write the book, Mishek teamed with longtime instructor Gregory Hart (MHA ’76) and lead author and Associate Professor Emerita Sandra Potthoff . The 300-page book formalizes the process and the mindset around using the method; addresses how to eliminate various forms of decision bias; and offers opportunities to practice the method through tools and cases. 

Related Posts

From the Dean

Welcoming a New Dean

Launching the BA in Public Health

Begin typing your search above and press return to search. Press Esc to cancel.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Am J Public Health
• v.105(Suppl 1); Mar 2015

Methodological Innovations in Public Health Education: Transdisciplinary Problem Solving

E. F. Lawlor and M. W. Kreuter outlined the article. All authors contributed to writing and editing the article.

In 2008, the faculty of the Brown School at Washington University in St. Louis designed a Master of Public Health program centered on transdisciplinary problem solving in public health.

We have described the rationale for our approach, guiding principles and pedagogy for the program, and specific transdisciplinary competencies students acquire.

We have explained how transdisciplinary content has been organized and delivered, how the program is being evaluated, and how we have demonstrated the feasibility of this approach for a Master of Public Health degree.

The argument for improving public health education through case studies and blending disciplines has been made for the past decade, 1,2 setting the stage for interdisciplinary and transdisciplinary education that will build workforce capacity in science and practice to solve complex public health problems.

In 2008 the faculty of the Brown School at Washington University in St. Louis embarked on the design of a Master of Public Health (MPH) degree program that would operationalize this mandate for public health education. 3 The charge to the faculty was to begin with a blank slate and be innovative yet be sure to integrate all the requirements of the Council for Public Health Education for an accredited program. The Council for Public Health Education was highly supportive of this approach to program and curriculum development from the outset.

Working across disciplines to address a common challenge can take different forms. Multidisciplinary work is typically characterized by a sequential or additive combination of ideas or methods. Interdisciplinary approaches involve sharing and coordination across fields but with participants still anchored in the models and methods of their own discipline. By contrast, transdisciplinary approaches involve developing shared new frameworks that integrate and extend concepts and methods from among different disciplines, thereby transcending disciplinary boundaries. 4–6 Our focus on transdisciplinary problem solving had its roots in a broader institutional philosophy of public health.

In 2008, Washington University launched a university-wide public health initiative dedicated to involving all seven schools—architecture, arts and sciences, engineering, business, law, medicine, and social work—in collaborative public health research and teaching. The integrating structure, the Institute for Public Health, appointed 165 public health scholars from across these seven schools. The Brown School, a longstanding and premier school of social work with a long history of contributions in mental health research, also committed itself to the ideal of transdisciplinary professional education in public health focused on breaking down academic silos and integrating nonacademic and nonpublic health-focused community partners into the educational model.

The initial concept of the MPH program was explicitly transdisciplinary. Student recruitment materials, curricular design, community partnerships, and faculty recruitment all reflected this priority. The uptake of this idea was immediate and enthusiastic: the school recruited faculty and students who were looking for this approach to public health education and research.

The public health faculty of the Brown School reviewed the latest thinking and key concepts from the literature on transdisciplinary science, 7–15 consulted with leaders of the transdisciplinary movement, and invited numerous faculty from other public health schools and programs to learn about models and challenges in providing transdisciplinary education. Especially influential in this process was the work of Daniel Stokols on transdisciplinarity, team science, and evaluation, which is captured in this recent definition of transdisciplinarity:

An integrative process whereby scholars and practitioners from both academic disciplines and non-academic fields work jointly to develop and use novel conceptual and methodological approaches that synthesize and extend discipline-specific perspectives, theories, methods, and translational strategies to yield innovative solutions to particular scientific and societal problems. 16 (p6)

Several central ideas emerged from the process of bringing transdisciplinary approaches into the curriculum planning process:

• Teaching and student work should engage the relevant communities, leaders, and organizations necessary for problem understanding and solutions.
• Public health education needs to encompass the entire arc from underlying science to the organizational, social, and policy challenges of interventions.
• Public health education should be intensely applied and problem driven, with students pushed to design and implement solutions to important real-world public health problems that draw on evidence when it is available but also are innovative and feasible.
• The courses should integrate team-based learning.

The core innovation that resulted from this process was the creation of required transdisciplinary problem-solving courses, or, as they came to be known, TPS courses.

TRANSDISCIPLINARY COMPETENCIES

We followed guidance from the Council for Public Health Education to develop competencies that span the five core disciplines of public health and the seven crosscutting thematic areas. 17 However, neither Council for Public Health Education guidance nor the Council on Linkages Between Academia and Public Health Practice’s Core Competencies for Public Health Professionals 18 includes competencies focused on transdisciplinary education. So we developed a set of competencies for transdisciplinary problem solving in public health. In an iterative process among faculty members, we established seven such competencies that conveyed the base of knowledge and skills we expected students to master. 19 These are as follows:

• Explaining why the complex, multifactorial nature of problems in public health and health disparities requires a transdisciplinary approach
• Describing how social, economic, behavioral, environmental, and biological conditions contribute to health outcomes using theoretical approaches drawn from diverse disciplines
• Distinguishing the features of transdisciplinary collaboration
• Defining problems in a transdisciplinary way and developing shared conceptual frameworks from discipline-specific theories and models
• Developing and applying processes that integrate and promote transdisciplinary perspectives, contributions, and collaborations
• Applying transdisciplinary solutions to public health problems using appropriate analytical tools drawn from public health or other disciplines
• Demonstrating the ability to communicate transdisciplinary research evidence to key stakeholders to influence policy and practice

As is typical in any MPH curriculum, the competencies to be developed in each course were spelled out in the syllabus. In addition, we mapped TPS competencies by course in a matrix that indicates which courses are responsible for the primary development of each competency and which reinforce or extend each competency. This process flagged course–competency gaps and helped us adapt courses accordingly. The goal was to have each competency developed in more than one TPS course.

CONCEPTUAL FRAMEWORK

The conceptual framework for this educational program, drawing from Stokols, 20 looks to evaluate the quality, appropriateness, and implementation of problem-solving approaches; the competencies acquired by students; and the long-term capacity of graduates to advance innovative public health solutions. Because this approach is embedded in professional education, our framing and criteria for evaluation are somewhat different from those applied to transdisciplinary science or transdisciplinary collaboration. 15 These distinctions are important because much of the literature on transdisciplinarity focuses on the behavior of scientific teams, in which intellectual products are the outcome of interest, or collaboration, in which the outcomes of interests are the dynamics and behavior of groups or teams. In public health graduate education, we are interested in both the competencies and skills of students to perform this style of public health practice as well as the innovation and utility of solutions that are produced for improving public health.

Several evaluative domains guide this curriculum and approach to public health education. First, does the intermixing of disciplines, scientists, practitioners, and community produce original and productive reframing of public health problems and generate effective solutions? Second, does the transdisciplinary approach help develop skills, competencies, habits of mind, and the propensity to work effectively with others to solve public health problems? Third, does this educational approach produce distinctive lifelong career choices, transdisciplinary work styles, and an innovative impact in public health?

The first domain is assessed by the work products themselves, including judging performed by clients and external organizations. Ultimately, the outcome test of these courses is whether the proposed solutions are implemented in policy or practice. The second domain is assessed by evaluating competencies for transdisciplinary work, most directly by reviewing students’ culminating experiences. The formal and rigorous assessment of transdisciplinary skills and competencies is still evolving in our program. Deeper questions about the capacity of students to engage in effective group work, thoroughly integrate disciplines, and incorporate the arc of cells to society in formulating solutions will need more careful conceptualization, measurement, and evaluation. Although we have plans to longitudinally assess the lifetime application of this transdisciplinary approach in graduates’ careers, it will take time for this measurement to occur.

Every TPS course begins with the identification of the multiple factors that influence a specific public health problem. Students are coached to recognize how the problem is defined and addressed by different science and practice disciplines as well as how it may be seen from the perspective of those with different political ideologies. Students must demonstrate the ability to integrate these ideas into a more comprehensive understanding of problem and solution.

Non–public-health-focused community partners play an important role in many TPS courses and can be engaged in a variety of ways. For example, a course may be structured to have students apply transdisciplinary thinking to priority needs of a community partner. In a TPS course on criminal justice, students work on policies addressing the occupational experiences of corrections officers—a need identified by the Department of Public Safety. At semester’s end, the Department of Public Safety invited students to present their solutions to the Division of Corrections personnel department and the mayor’s office at the city hall.

Community partners also can provide insight into problems from a nontraditional public health perspective. For example, horticulturalists from the Missouri Botanical Garden have instructed students in a TPS course on the built environment on sustainability practices such as rain gardens and the use of plants in and around homes. Neighborhood stabilization team members from the city of St. Louis have described their role in improving housing and safety for residents in urban neighborhoods. Finally, external partners can be engaged in evaluating the transdisciplinary solutions that students generate. This often occurs through participation on review panels at the end of a TPS course. In one such course on health disparities, agency leaders from Social Services and Tobacco Control in Missouri selected and actually implemented a student proposal to connect food stamp recipients in the state with the Missouri Tobacco Quitline. 21

TRANSDISCIPLINARY PROBLEM-SOLVING COURSES

In the first five years of our MPH program, we developed 16 TPS courses that collectively were offered 34 times. Three additional courses codeveloped with or taught by faculty from the School of Law, the School of Design, and the Department of Anthropology also were approved as TPS courses. Our courses tackled public health challenges such as child obesity, global undernutrition, pregnancy outcomes, sexual health, tobacco, and child maltreatment.

Although all TPS courses adhered to the requirements, goals, and competencies spelled out previously, faculty were encouraged to innovate how their TPS course was structured and delivered. As a result, a variety of approaches emerged, with courses varying in (1) the selection and organization of transdisciplinary content, (2) the relative emphasis placed on transdisciplinary understanding versus solutions, and (3) the instructional model for delivering transdisciplinary learning.

An example of this framework and outcome of a TPS approach is the Healthy Families Initiative Implementation student team project in remote rural India. 22 This project was part of an eight-week TPS course on implementing public health interventions in developing countries that was jointly sponsored by the Brown School and the IKP Centre for Technologies in Public Health. 23 Student teams collaborated with practitioners, scientists in India, nongovernmental organizations, and community residents to design and evaluate novel interventions for stubbornly challenging public health problems. Projects included improving cardiovascular health, reducing oropharyngeal cancer, reducing diarrheal disease, addressing dental disease, and reducing high levels of anemia among women. In Healthy Families, the reduction of maternal anemia was conceptualized not as the delivery of appropriate micronutrients but rather as a complex sequence of understanding the epidemiology of maternal anemia (which affects 59.3% of pregnant women in the Thanjavur district), diagnostics; cultural and social determinants of nutrition, the evidence about effectiveness of educational and provider strategies, compliance behavior, appropriate and innovative evaluation criteria, and systems design for implementation.

Organizing Trandisciplinary Content

Our TPS courses have used three approaches. The first, cells to society, organizes disciplinary perspectives by levels or pathways of influence on a public health problem. This approach was popularized in biomedical and health science research communities and embraced by the National Institutes of Health and other institutions. 21,24 It considers how biological, behavioral, environmental, policy, economic, and other processes and factors—as well as the interplay among them—contribute to a public health problem. As an example, a TPS course on obesity examined how bacteria in the gut microbiome, biological mechanisms that regulate metabolism, lifestyle behaviors such as diet and activity, agricultural and economic policies, and food marketing and labeling regulations can influence obesity. Considering these perspectives together brings a richer understanding of obesity causes and challenges students to generate a more integrative model of obesity prevention.

A variant of this approach focuses on spheres of influence that are not inherently hierarchical as are the levels of an ecological model but rather that operate within a single level, such as the social environment. For example, a TPS course on child maltreatment explores how youths affected by violence or neglect are likely to be involved in many health and social service systems including child welfare, law enforcement, courts, medical care, mental health care, and housing shelters. Understanding these touch points and the goals of each system prepares students to think more broadly about potential opportunities for a coordinated and effective response.

Other courses have used a strategic blending of selected disciplinary perspectives. By contrast to TPS courses that examine a problem from many different angles, this approach focuses on a few handpicked disciplines that may be ripe for integration, with promising but untapped synergy. It reflects a purposeful exploration of the intersection between disciplines to solve a public health problem. For instance, one TPS course sought to integrate health and social services systems to reduce the prevalence and impact of smoking in low-income populations. The course focused on practice disciplines in public health (tobacco control) and social services (food security) as well as theory and research from the field of organizational behavior, specifically within public agencies. 25 Students in this course worked in teams to develop integration strategies and presented these as proposals to agency leaders.

Understanding vs Solutions

Each TPS course covers the continuum from understanding to solving a public health problem, with corresponding goals of helping students think and act differently on the basis of a transdisciplinary perspective. But the relative emphasis on understanding versus solving varies from course to course. For example, a TPS course on pregnancy outcomes strives for a deeper understanding of the myriad exposures that contribute to adverse maternal and child outcomes. Others, such as a health disparities course in which students learn to communicate effectively about social determinants of health to policy makers and the public, emphasize generating promising solutions.

These differences reflect the background, interests, and orientation of the instructor but also the maturity of the knowledge base on a particular public health challenge. They are often reflected in the kinds of assignments or products students complete in a course. For example, TPS courses focused on understanding might culminate in the development of integrative causal models, whereas solution-focused courses could generate specific strategies for prevention.

Instructional Models

Most instructors have delivered their TPS courses using one or more of four instructional approaches, each with its strengths and limitations. Team teaching pairs instructors from different disciplines central to the course content. Besides the obvious advantages of broader expertise, this model also tends to draw students from different disciplines across campus (usually those of the participating instructors), which can create valuable interactions among students with different training and perspectives. But we have found it can also create administrative challenges assigning or dividing teaching credit for instructors as well as allocating limited seats in the class across schools.

Some TPS courses consider so many disciplines that no two instructors could reasonably be expected to have deep knowledge of all content to be covered. In these cases, some TPS courses have opted for an expert-of-the-week model. Like team teaching, its main advantage is the breadth of expertise students are exposed to. However, we have found this guest speaker–dependent model hard to sustain because leading experts may not be consistently available to contribute every semester or year, especially to a course outside their school or department. Recording their live contributions for future use seems a reasonable solution to this challenge.

Several TPS courses use a design studio approach modeled after architectural training. Design studio work focuses on time-limited projects (usually one semester) addressing “complex and open-ended problems” through “rapid iteration of design solutions.” 26 (p349) It is informed by frequent formal and informal critique from peers and instructors as well as by precedent—examining past solutions to problems deemed relevant to the current project. In some TPS courses, students spend up to half of each class session (60–90 min) working on transdisciplinary solutions to a public health problem. During these periods, instructors provide students with feedback and suggestions. Students typically spend much more time working on their solutions outside class than in class, but the design studio approach ensures that they receive regular feedback throughout the development process. In our experience, this type of learning is initially unfamiliar to many public health students and requires coaching (and coaxing) to get meaningful peer critiques early on. It is a better fit for solution-focused TPS courses.

Finally, some TPS courses have been taught in situ, in environments as diverse as rural India, Haiti, and Washington, DC, and have relied much more on experiential learning, faculty supervision, and field applications. These courses are so popular among students we have had to adopt a competitive application process for enrollment because demand exceeds course capacity. On the down side, they are extremely time-consuming to plan, logistically complicated, and expensive to execute.

FACULTY, CULTURE, AND INSTITUTIONAL SUPPORT FOR TPS

A critical element in creating the environment for transdisciplinary public health education, reinforcing its importance, and promoting a distinctive style of education is recruiting and retaining faculty who exemplify this approach. The school’s faculty recruitment process purposely selects talented faculty with the training and aspirations to engage in team science and transdisciplinary learning. A faculty committee investigated research environments—such as the Bell Labs—that leveraged transdisciplinary scientific approaches.

An internal network analysis of faculty research and teaching investigated cross-disciplinary faculty research, teaching, and service activities. The Promotion and Tenure Committee took up the issue of how transdisciplinary research and education could be credited and recognized in promotion and tenure reviews. School administrators recognized that many TPS courses would require resources beyond what is needed in a typical course and created a specific budget to support them. In short, institutional support and engagement at all levels of the school have been critical to building this approach.

CONCLUSIONS

Transdisciplinary public health is a concept that is compelling in principle but difficult in practice. As schools and programs of public health continue to innovate with new approaches to address complex public health problems, the experiences we have described show that it is feasible to provide transdisciplinary graduate education in public health, with significant innovation in teaching methods, partnerships, and even the recruitment and promotion of faculty. We do not presume that our approach is the best or only way to organize and deliver transdisciplinary public health education, but we hope that sharing it will contribute to growing discussion and action on this important topic and help public health educators think about ways to integrate transdisciplinary learning activities.

Although we have not yet conducted a formal evaluation of our TPS model, we have gathered evaluative data in several forms. We adapted our standard course evaluation to include special items for TPS courses (e.g., assessing how well the course fostered a transdisciplinary approach, included perspectives from diverse disciplines, or used a cells-to-society approach). We also have substantial anecdotal evidence of impact from the perspective of faculty members, students, and prospective students.

Faculty members’ interest in developing and teaching TPS courses quickly grew to the point that we had to implement a multistage committee process to review, approve, and track TPS-designated courses. Student demand for the courses also grew, and many wanted to take TPS courses beyond the program requirements and use them to fulfill elective credits. As a result, we have had to control registration to ensure that students needing to meet their TPS requirements have enrollment priorities over those who have already completed the requirement.

In addition, every year a significant portion of applicants to our MPH program indicates (without solicitation) that their interest in our program is closely tied to our transdisciplinary focus. Part of this almost certainly reflects prospective students customizing their applications to match our program’s identity, but it is equally clear that there is a sizable niche of students with not only genuine interest in transdisciplinary thinking but also undergraduate or professional experiences learning and applying this approach. Finally, it is remarkable how quickly, thoroughly, and palpably the TPS perspective has permeated our school’s culture and identity. “Transdisciplinary” has become as commonplace as “population” or “prevention” in any discussion of public health and the Brown School. And it is not spoken just by those in the school but also by top university leaders, board members, and community partners.

Although influential and important public health leaders have emphasized the importance of training transdisciplinary public health professionals, we are still in a transitional period in which careers, faculty incentives, and professional reinforcements have not caught up. It is important for the public health field writ large to embrace this approach to create a cadre of transdisciplinary scientists and practitioners who will promote systemic change. Students and faculty are motivated to apply this approach to public health education and research.