design based research approach

• Introduction
• Acknowledgements
• 1. Groundwork
• 1.1. Research
• 1.2. Knowing
• 1.3. Theories
• 1.4. Ethics
• 2. Paradigms
• 2.1. Inferential Statistics
• 2.2. Sampling
• 2.3. Qualitative Rigor
• 2.4. Design-Based Research
• 2.5. Mixed Methods
• 3. Learning Theories
• 3.1. Behaviorism
• 3.2. Cognitivism
• 3.3. Constructivism
• 3.4. Socioculturalism
• 3.5. Connectivism
• Appendix A. Supplements
• Appendix B. Example Studies
• Example Study #1. Public comment sentiment on educational videos
• Example Study #2. Effects of open textbook adoption on teachers' open practices
• Appendix C. Historical Readings
• Manifesto of the Communist Party (1848)
• On the Origin of Species (1859)
• Science and the Savages (1905)
• Theories of Knowledge (1916)
• Theories of Morals (1916)
• Translations

Design-Based Research

Choose a sign-in option.

Tools and Settings

Questions and Tasks

Citation and Embed Code

In an educational setting, design-based research is a research approach that engages in iterative designs to develop knowledge that improves educational practices. This chapter will provide a brief overview of the origin, paradigms, outcomes, and processes of design-based research (DBR). In these sections we explain that (a) DBR originated because some researchers believed that traditional research methods failed to improve classroom practices, (b) DBR places researchers as agents of change and research subjects as collaborators, (c) DBR produces both new designs and theories, and (d) DBR consists of an iterative process of design and evaluation to develop knowledge.

Origin of DBR

DBR originated as researchers like Allan Collins (1990) and Ann Brown (1992) recognized that educational research often failed to improve classroom practices. They perceived that much of educational research was conducted in controlled, laboratory-like settings. They believed that this laboratory research was not as helpful as possible for practitioners.

Proponents of DBR claim that educational research is often detached from practice (The Design-Based Research Collective, 2002). There are at least two problems that arise from this detachment: (a) practitioners do not benefit from researchers’ work and (b) research results may be inaccurate because they fail to account for context (The Design-Based Research Collective, 2002).

Practitioners do not benefit from researchers’ work if the research is detached from practice. Practitioners are able to benefit from research when they see how the research can inform and improve their designs and practices. Some practitioners believe that educational research is often too abstract or sterilized to be useful in real contexts (The Design-Based Research Collective, 2002).

Not only is lack of relevance a problem, but research results can also be inaccurate by failing to account for context. Findings and theories based on lab results may not accurately reflect what happens in real-world educational settings.

Conversely, a problem that arises from an overemphasis on practice is that while individual practices may improve, the general body of theory and knowledge does not increase. Scholars like Collins (1990) and Brown (1992) believed that the best way to conduct research would be to achieve the right balance between theory-building and practical impact.

Paradigms of DBR

Proponents of DBR believe that conducting research in context, rather than in a controlled laboratory setting, and iteratively designing interventions yields authentic and useful knowledge. Sasha Barab (2004) says that the goal of DBR is to “directly impact practice while advancing theory that will be of use to others” (p. 8). This implies “a pragmatic philosophical underpinning, one in which the value of a theory lies in its ability to produce changes in the world” (p. 6). The aims of DBR and the role of researchers and subjects are informed by this philosophical underpinning.

Aims of DBR

Traditional, experimental research is conducted by theorists focused on isolating variables to test and refine theory. DBR is conducted by designers focused on (a) understanding contexts, (b) designing effective systems, and (c) making meaningful changes for the subjects of their studies (Barab & Squire, 2004; Collins, 1990). Traditional methods of research generate refined understandings of how the world works, which may indirectly affect practice. In DBR there is an intentionality in the research process to both refine theory and practice (Collins et al., 2004).

Role of DBR Researcher

In DBR, researchers assume the roles of “curriculum designers, and implicitly, curriculum theorists” (Barab & Squire, 2004, p.2). As curriculum designers, DBR researchers come into their contexts as informed experts with the purpose of creating, “test[ing] and refin[ing] educational designs based on principles derived from prior research” (Collins et al., 2004, p. 15). These educational designs may include curricula, practices, software, or tangible objects beneficial to the learning process (Barab & Squire, 2004). As curriculum theorists, DBR researchers also come into their research contexts with the purpose to refine extant theories about learning (Brown, 1992).

This duality of roles for DBR researchers contributes to a greater sense of responsibility and accountability within the field. Traditional, experimental researchers isolate themselves from the subjects of their study (Barab & Squire, 2004). This separation is seen as a virtue, allowing researchers to make dispassionate observations as they test and refine their understandings of the world around them. In comparison, design-based researchers “bring agendas to their work,” see themselves as necessary agents of change and see themselves as accountable for the work they do (Barab & Squire, 2004, p. 2).

Role of DBR Subjects

Within DBR, research subjects are seen as key contributors and collaborators in the research process. Classic experimentalism views the subjects of research as things to be observed or experimented on, suggesting a unidirectional relationship between researcher and research subject. The role of the research subject is to be available and genuine so that the researcher can make meaningful observations and collect accurate data. In contrast, design-based researchers view the subjects of their research (e.g., students, teachers, schools) as “co-participants” (Barab & Squire, 2004, p. 3) and “co-investigators” (Collins, 1990, p. 4). Research subjects are seen as necessary in “helping to formulate the questions,” “making refinements in the designs,” “evaluating the effects of...the experiment,” and “reporting the results of the experiment to other teachers and researchers” (Collins, 1990, pp. 4-5). Research subjects are co-workers with the researcher in iteratively pushing the study forward.

Outcomes of DBR

DBR educational research develops knowledge through this collaborative, iterative research process. The knowledge developed by DBR can be separated into two categories: (a) tangible, practical outcomes and (b) intangible, theoretical outcomes.

Tangibles Outcomes

A major goal of design-based research is producing meaningful interventions and practices. Within educational research these interventions may “involve the development of technological tools [and] curricula” (Barab & Squire, 2004, p. 1). But more than just producing meaningful educational products for a specific context, DBR aims to produce meaningful, effective educational products that can be transferred and adapted (Barab & Squire, 2004). As expressed by Brown (1992), “an effective intervention should be able to migrate from our experimental classroom to average classrooms operated by and for average students and teachers” (p.143).

Intangible Outcomes

It is important to recognize that DBR is not only concerned with improving practice but also aims to advance theory and understanding (Collins et al., 2004). DBR’s emphasis on the importance of context enhances the knowledge claims of the research. “Researchers investigate cognition in context...with the broad goal of developing evidence-based claims derived from both laboratory-based and naturalistic investigations that result in knowledge about how people learn” (Barab & Squire, 2004, p.1). This new knowledge about learning then drives future research and practice.

Process of DBR

A hallmark of DBR is the iterative nature of its interventions. As each iteration progresses, researchers refine and rework the intervention drawing on a variety of research methods that best fit the context. This flexibility allows the end result to take precedence over the process. While each researcher may use different methods, McKenny and Reeves (2012) outlined three core processes of DBR: (a) analysis and exploration, (b) design and construction, and (c) evaluation and reflection. To put these ideas in context, we will refer to a recent DBR study completed by Siko and Barbour regarding the use of PowerPoint games in the classroom.

Analysis and Exploration

Analysis is a critical aspect of DBR and must be used throughout the entire process. At the start of a DBR project, it is critical to understand and define which problem will be addressed. In collaboration with practitioners, researchers seek to understand all aspects of a problem. Additionally, they “seek out and learn from how others have viewed and solved similar problems ” (McKenny & Reeves, 2012, p. 85). This analysis helps to provide an understanding of the context within which to execute an intervention.

Since theories cannot account for the variety of variables in a learning situation, exploration is needed to fill the gaps. DBR researchers can draw from a number of disciplines and methodologies as they execute an intervention. The decision of which methodologies to use should be driven by the research context and goals.

Siko and Barbour (2016) used the DBR process to address a gap they found in research regarding the effectiveness of having students create their own PowerPoint games to review for a test. In analyzing existing research, they found studies that stated teaching students to create their own PowerPoint games did not improve content retention. Siko and Barbour wanted to “determine if changes to the implementation protocol would lead to improved performance” (Siko & Barbour, 2016, p. 420). They chose to test their theory in three different phases and adapt the curriculum following each phase.

Design and Construction

Informed by the analysis and exploration, researchers design and construct interventions, which may be a specific technology or “less concrete aspects such as activity structures, institutions, scaffolds, and curricula” (Design-Based Research Collective, 2003, pp. 5–6). This process involves laying out a variety of options for a solution and then creating the idea with the most promise.

Within Siko and Barbour’s design, they planned to observe three phases of a control group and a test group. Each phase would use t-tests to compare two unit tests for each group. They worked with teachers to implement time for playing PowerPoint games as well as a discussion on what makes games successful. The first implementation was a control phase that replicated past research and established a baseline. Once they finished that phase, they began to evaluate.

Evaluation and Reflection

Researchers can evaluate their designs both before and after use. The cyclical process involves careful, constant evaluation for each iteration so that improvements can be made. While tests and quizzes are a standard way of evaluating educational progress, interviews and observations also play a key role, as they allow for better understanding of how teachers and students might see the learning situation.

Reflection allows the researcher to make connections between actions and results. Researchers must take the time to analyze what changes allowed them to have success or failure so that theory and practice at large can be benefited. Collins (1990) states:

It is important to analyze the reasons for failure and to take steps to fix them. It is critical to document the nature of the failures and the attempted revisions, as well as the overall results of the experiment, because this information informs the path to success. (pg. 5)

As researchers reflect on each change they made, they find what is most useful to the field at large, whether it be a failure or a success.

After evaluating results of the first phase, Siko and Barbour revisited the literature of instructional games. Based on that research, they first tried extending the length of time students spent creating the games. They also discovered that the students struggled to design effective test questions, so the researchers tried working with teachers to spend more time explaining how to ask good questions. As they explored these options, researchers were able to see unit test scores improve.

Reflection on how the study was conducted allowed the researchers to properly place their experiences within the context of existing research. They recognized that while they found positive impacts as a result of their intervention, there were a number of limitations with the study. This is an important realization for the research and allows readers to not misinterpret the scope of the findings.

This chapter has provided a brief overview of the origin, paradigms, outcomes, and processes of Design-Based Research (DBR). We explained that (a) DBR originated because some researchers believed that traditional research methods failed to improve classroom practices, (b) DBR places researchers as agents of change and research subjects as collaborators, (c) DBR produces both new designs and theories, and (d) DBR consists of an iterative process of design and evaluation to develop knowledge.

Barab, S., & Squire, K. (2004). Design-based research: putting a stake in the ground. Journal of the Learning Sciences, 13(1), 1–14.

Brown, A. L. (1992). Design experiments: theoretical and methodological challenges in creating complex interventions in classroom settings. Journal of the Learning Sciences, 2(2), 141–178.

Collins, A. (1990). Toward a design science of education (Report No. 1). Washington, DC: Center for Technology in Education.

Collins, A., Joseph, D., & Bielaczyc, K. (2004). Design research: Theoretical and methodological issues. Journal of the Learning Sciences, 13(1), 15–42.

Mckenney, S., & Reeves, T.C. (2012) Conducting Educational Design Research. New York, NY: Routledge.

Siko, J. P., & Barbour, M. K. (2016). Building a better mousetrap: how design-based research was used to improve homemade PowerPoint games. TechTrends, 60(5), 419–424.

The Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5–8.

This content is provided to you freely by BYU Open Learning Network.

Access it online or download it at https://open.byu.edu/education_research/design_based_research .

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• CBE Life Sci Educ
• v.19(3); Fall 2020

Design-Based Research: A Methodology to Extend and Enrich Biology Education Research

Emily e. scott.

† Department of Biology, University of Washington, Seattle, WA 98195

Mary Pat Wenderoth

Jennifer h. doherty.

Recent calls in biology education research (BER) have recommended that researchers leverage learning theories and methodologies from other disciplines to investigate the mechanisms by which students to develop sophisticated ideas. We suggest design-based research from the learning sciences is a compelling methodology for achieving this aim. Design-based research investigates the “learning ecologies” that move student thinking toward mastery. These “learning ecologies” are grounded in theories of learning, produce measurable changes in student learning, generate design principles that guide the development of instructional tools, and are enacted using extended, iterative teaching experiments. In this essay, we introduce readers to the key elements of design-based research, using our own research into student learning in undergraduate physiology as an example of design-based research in BER. Then, we discuss how design-based research can extend work already done in BER and foster interdisciplinary collaborations among cognitive and learning scientists, biology education researchers, and instructors. We also explore some of the challenges associated with this methodological approach.

INTRODUCTION

There have been recent calls for biology education researchers to look toward other fields of educational inquiry for theories and methodologies to advance, and expand, our understanding of what helps students learn to think like biologists ( Coley and Tanner, 2012 ; Dolan, 2015 ; Peffer and Renken, 2016 ; Lo et al. , 2019 ). These calls include the recommendations that biology education researchers ground their work in learning theories from the cognitive and learning sciences ( Coley and Tanner, 2012 ) and begin investigating the underlying mechanisms by which students to develop sophisticated biology ideas ( Dolan, 2015 ; Lo et al. , 2019 ). Design-based research from the learning sciences is one methodology that seeks to do both by using theories of learning to investigate how “learning ecologies”—that is, complex systems of interactions among instructors, students, and environmental components—support the process of student learning ( Brown, 1992 ; Cobb et al. , 2003 ; Collins et al. , 2004 ; Peffer and Renken, 2016 ).

The purpose of this essay is twofold. First, we want to introduce readers to the key elements of design-based research, using our research into student learning in undergraduate physiology as an example of design-based research in biology education research (BER). Second, we will discuss how design-based research can extend work already done in BER and explore some of the challenges of its implementation. For a more in-depth review of design-based research, we direct readers to the following references: Brown (1992) , Barab and Squire (2004) , and Collins et al. (2004) , as well as commentaries by Anderson and Shattuck (2012) and McKenney and Reeves (2013) .

WHAT IS DESIGN-BASED RESEARCH?

Design-based research is a methodological approach that aligns with research methods from the fields of engineering or applied physics, where products are designed for specific purposes ( Brown, 1992 ; Joseph, 2004 ; Middleton et al. , 2008 ; Kelly, 2014 ). Consequently, investigators using design-based research approach educational inquiry much as an engineer develops a new product: First, the researchers identify a problem that needs to be addressed (e.g., a particular learning challenge that students face). Next, they design a potential “solution” to the problem in the form of instructional tools (e.g., reasoning strategies, worksheets; e.g., Reiser et al. , 2001 ) that theory and previous research suggest will address the problem. Then, the researchers test the instructional tools in a real-world setting (i.e., the classroom) to see if the tools positively impact student learning. As testing proceeds, researchers evaluate the instructional tools with emerging evidence of their effectiveness (or lack thereof) and progressively revise the tools— in real time —as necessary ( Collins et al. , 2004 ). Finally, the researchers reflect on the outcomes of the experiment, identifying the features of the instructional tools that were successful at addressing the initial learning problem, revising those aspects that were not helpful to learning, and determining how the research informed the theory underlying the experiment. This leads to another research cycle of designing, testing, evaluating, and reflecting to refine the instructional tools in support of student learning. We have characterized this iterative process in Figure 1 after Sandoval (2014) . Though we have portrayed four discrete phases to design-based research, there is often overlap of the phases as the research progresses (e.g., testing and evaluating can occur simultaneously).

The four phases of design-based research experienced in an iterative cycle (A). We also highlight the main features of each phase of our design-based research project investigating students’ use of flux in physiology (B).

Design-based research has no specific requirements for the form that instructional tools must take or the manner in which the tools are evaluated ( Bell, 2004 ; Anderson and Shattuck, 2012 ). Instead, design-based research has what Sandoval (2014) calls “epistemic commitments” 1 that inform the major goals of a design-based research project as well as how it is implemented. These epistemic commitments are: 1) Design based research should be grounded in theories of learning (e.g., constructivism, knowledge-in-pieces, conceptual change) that both inform the design of the instructional tools and are improved upon by the research ( Cobb et al. , 2003 ; Barab and Squire, 2004 ). This makes design-based research more than a method for testing whether or not an instructional tool works; it also investigates why the design worked and how it can be generalized to other learning environments ( Cobb et al. , 2003 ). 2) Design-based research should aim to produce measurable changes in student learning in classrooms around a particular learning problem ( Anderson and Shattuck, 2012 ; McKenney and Reeves, 2013 ). This requirement ensures that theoretical research into student learning is directly applicable, and impactful, to students and instructors in classroom settings ( Hoadley, 2004 ). 3) Design-based research should generate design principles that guide the development and implementation of future instructional tools ( Edelson, 2002 ). This commitment makes the research findings broadly applicable for use in a variety of classroom environments. 4) Design-based research should be enacted using extended, iterative teaching experiments in classrooms. By observing student learning over an extended period of time (e.g., throughout an entire term or across terms), researchers are more likely to observe the full effects of how the instructional tools impact student learning compared with short-term experiments ( Brown, 1992 ; Barab and Squire, 2004 ; Sandoval and Bell, 2004 ).

HOW IS DESIGN-BASED RESEARCH DIFFERENT FROM AN EXPERIMENTAL APPROACH?

Many BER studies employ experimental approaches that align with traditional scientific methods of experimentation, such as using treatment versus control groups, randomly assigning treatments to different groups, replicating interventions across multiple spatial or temporal periods, and using statistical methods to guide the kinds of inferences that arise from an experiment. While design-based research can similarly employ these strategies for educational inquiry, there are also some notable differences in its approach to experimentation ( Collins et al. , 2004 ; Hoadley, 2004 ). In this section, we contrast the differences between design-based research and what we call “experimental approaches,” although both paradigms represent a form of experimentation.

The first difference between an experimental approach and design-based research regards the role participants play in the experiment. In an experimental approach, the researcher is responsible for making all the decisions about how the experiment will be implemented and analyzed, while the instructor facilitates the experimental treatments. In design-based research, both researchers and instructors are engaged in all stages of the research from conception to reflection ( Collins et al. , 2004 ). In BER, a third condition frequently arises wherein the researcher is also the instructor. In this case, if the research questions being investigated produce generalizable results that have the potential to impact teaching broadly, then this is consistent with a design-based research approach ( Cobb et al. , 2003 ). However, when the research questions are self-reflective about how a researcher/instructor can improve his or her own classroom practices, this aligns more closely with “action research,” which is another methodology used in education research (see Stringer, 2013 ).

A second difference between experimental research and design-based research is the form that hypotheses take and the manner in which they are investigated ( Collins et al. , 2004 ; Sandoval, 2014 ). In experimental approaches, researchers develop a hypothesis about how a specific instructional intervention will impact student learning. The intervention is then tested in the classroom(s) while controlling for other variables that are not part of the study in order to isolate the effects of the intervention. Sometimes, researchers designate a “control” situation that serves as a comparison group that does not experience the intervention. For example, Jackson et al. (2018) were interested in comparing peer- and self-grading of weekly practice exams to if they were equally effective forms of deliberate practice for students in a large-enrollment class. To test this, the authors (including authors of this essay J.H.D., M.P.W.) designed an experiment in which lab sections of students in a large lecture course were randomly assigned to either a peer-grading or self-grading treatment so they could isolate the effects of each intervention. In design-based research, a hypothesis is conceptualized as the “design solution” rather than a specific intervention; that is, design-based researchers hypothesize that the designed instructional tools, when implemented in the classroom, will create a learning ecology that improves student learning around the identified learning problem ( Edelson, 2002 ; Bell, 2004 ). For example, Zagallo et al. (2016) developed a laboratory curriculum (i.e., the hypothesized “design solution”) for molecular and cellular biology majors to address the learning problem that students often struggle to connect scientific models and empirical data. This curriculum entailed: focusing instruction around a set of target biological models; developing small-group activities in which students interacted with the models by analyzing data from scientific papers; using formative assessment tools for student feedback; and providing students with a set of learning objectives they could use as study tools. They tested their curriculum in a novel, large-enrollment course of upper-division students over several years, making iterative changes to the curriculum as the study progressed.

By framing the research approach as an iterative endeavor of progressive refinement rather than a test of a particular intervention when all other variables are controlled, design-based researchers recognize that: 1) classrooms, and classroom experiences, are unique at any given time, making it difficult to truly “control” the environment in which an intervention occurs or establish a “control group” that differs only in the features of an intervention; and 2) many aspects of a classroom experience may influence the effectiveness of an intervention, often in unanticipated ways, which should be included in the research team’s analysis of an intervention’s success. Consequently, the research team is less concerned with controlling the research conditions—as in an experimental approach—and instead focuses on characterizing the learning environment ( Barab and Squire, 2004 ). This involves collecting data from multiple sources as the research progresses, including how the instructional tools were implemented, aspects of the implementation process that failed to go as planned, and how the instructional tools or implementation process was modified. These characterizations can provide important insights into what specific features of the instructional tools, or the learning environment, were most impactful to learning ( DBR Collective, 2003 ).

A third difference between experimental approaches and design-based research is when the instructional interventions can be modified. In experimental research, the intervention is fixed throughout the experimental period, with any revisions occurring only after the experiment has concluded. This is critical for ensuring that the results of the study provide evidence of the efficacy of a specific intervention. By contrast, design-based research takes a more flexible approach that allows instructional tools to be modified in situ as they are being implemented ( Hoadley, 2004 ; Barab, 2014 ). This flexibility allows the research team to modify instructional tools or strategies that prove inadequate for collecting the evidence necessary to evaluate the underlying theory and ensures a tight connection between interventions and a specific learning problem ( Collins et al. , 2004 ; Hoadley, 2004 ).

Finally, and importantly, experimental approaches and design-based research differ in the kinds of conclusions they draw from their data. Experimental research can “identify that something meaningful happened; but [it is] not able to articulate what about the intervention caused that story to unfold” ( Barab, 2014 , p. 162). In other words, experimental methods are robust for identifying where differences in learning occur, such as between groups of students experiencing peer- or self-grading of practice exams ( Jackson et al. , 2018 ) or receiving different curricula (e.g., Chi et al. , 2012 ). However, these methods are not able to characterize the underlying learning process or mechanism involved in the different learning outcomes. By contrast, design-based research has the potential to uncover mechanisms of learning, because it investigates how the nature of student thinking changes as students experience instructional interventions ( Shavelson et al. , 2003 ; Barab, 2014 ). According to Sandoval (2014) , “Design research, as a means of uncovering causal processes, is oriented not to finding effects but to finding functions , to understanding how desired (and undesired) effects arise through interactions in a designed environment” (p. 30). In Zagallo et al. (2016) , the authors found that their curriculum supported students’ data-interpretation skills, because it stimulated students’ spontaneous use of argumentation during which group members coconstructed evidence-based claims from the data provided. Students also worked collaboratively to decode figures and identify data patterns. These strategies were identified from the researchers’ qualitative data analysis of in-class recordings of small-group discussions, which allowed them to observe what students were doing to support their learning. Because design-based research is focused on characterizing how learning occurs in classrooms, it can begin to answer the kinds of mechanistic questions others have identified as central to advancing BER ( National Research Council [NRC], 2012 ; Dolan, 2015 ; Lo et al. , 2019 ).

DESIGN-BASED RESEARCH IN ACTION: AN EXAMPLE FROM UNDERGRADUATE PHYSIOLOGY

To illustrate how design-based research could be employed in BER, we draw on our own research that investigates how students learn physiology. We will characterize one iteration of our design-based research cycle ( Figure 1 ), emphasizing how our project uses Sandoval’s four epistemic commitments (i.e., theory driven, practically applied, generating design principles, implemented in an iterative manner) to guide our implementation.

Identifying the Learning Problem

Understanding physiological phenomena is challenging for students, given the wide variety of contexts (e.g., cardiovascular, neuromuscular, respiratory; animal vs. plant) and scales involved (e.g., using molecular-level interactions to explain organism functioning; Wang, 2004 ; Michael, 2007 ; Badenhorst et al. , 2016 ). To address these learning challenges, Modell (2000) identified seven “general models” that undergird most physiology phenomena (i.e., control systems, conservation of mass, mass and heat flow, elastic properties of tissues, transport across membranes, cell-to-cell communication, molecular interactions). Instructors can use these models as a “conceptual framework” to help students build intellectual coherence across phenomena and develop a deeper understanding of physiology ( Modell, 2000 ; Michael et al. , 2009 ). This approach aligns with theoretical work in the learning sciences that indicates that providing students with conceptual frameworks improves their ability to integrate and retrieve knowledge ( National Academies of Sciences, Engineering, and Medicine, 2018 ).

Before the start of our design-based project, we had been using Modell’s (2000) general models to guide our instruction. In this essay, we will focus on how we used the general models of mass and heat flow and transport across membranes in our instruction. These two models together describe how materials flow down gradients (e.g., pressure gradients, electrochemical gradients) against sources of resistance (e.g., tube diameter, channel frequency). We call this flux reasoning. We emphasized the fundamental nature and broad utility of flux reasoning in lecture and lab and frequently highlighted when it could be applied to explain a phenomenon. We also developed a conceptual scaffold (the Flux Reasoning Tool) that students could use to reason about physiological processes involving flux.

Although these instructional approaches had improved students’ understanding of flux phenomena, we found that students often demonstrated little commitment to using flux broadly across physiological contexts. Instead, they considered flux to be just another fact to memorize and applied it to narrow circumstances (e.g., they would use flux to reason about ions flowing across membranes—the context where flux was first introduced—but not the bulk flow of blood in a vessel). Students also struggled to integrate the various components of flux (e.g., balancing chemical and electrical gradients, accounting for variable resistance). We saw these issues reflected in students’ lower than hoped for exam scores on the cumulative final of the course. From these experiences, and from conversations with other physiology instructors, we identified a learning problem to address through design-based research: How do students learn to use flux reasoning to explain material flows in multiple physiology contexts?

The process of identifying a learning problem usually emerges from a researcher’s own experiences (in or outside a classroom) or from previous research that has been described in the literature ( Cobb et al. , 2003 ). To remain true to Sandoval’s first epistemic commitment, a learning problem must advance a theory of learning ( Edelson, 2002 ; McKenney and Reeves, 2013 ). In our work, we investigated how conceptual frameworks based on fundamental scientific concepts (i.e., Modell’s general models) could help students reason productively about physiology phenomena (National Academies of Sciences, Engineering, and Medicine, 2018; Modell, 2000 ). Our specific theoretical question was: Can we characterize how students’ conceptual frameworks around flux change as they work toward robust ideas? Sandoval’s second epistemic commitment stated that a learning problem must aim to improve student learning outcomes. The practical significance of our learning problem was: Does using the concept of flux as a foundational idea for instructional tools increase students’ learning of physiological phenomena?

We investigated our learning problem in an introductory biology course at a large R1 institution. The introductory course is the third in a biology sequence that focuses on plant and animal physiology. The course typically serves between 250 and 600 students in their sophomore or junior years each term. Classes have the following average demographics: 68% male, 21% from lower-income situations, 12% from an underrepresented minority, and 26% first-generation college students.

Design-Based Research Cycle 1, Phase 1: Designing Instructional Tools

The first phase of design-based research involves developing instructional tools that address both the theoretical and practical concerns of the learning problem ( Edelson, 2002 ; Wang and Hannafin, 2005 ). These instructional tools can take many forms, such as specific instructional strategies, classroom worksheets and practices, or technological software, as long as they embody the underlying learning theory being investigated. They must also produce classroom experiences or materials that can be evaluated to determine whether learning outcomes were met ( Sandoval, 2014 ). Indeed, this alignment between theory, the nature of the instructional tools, and the ways students are assessed is central to ensuring rigorous design-based research ( Hoadley, 2004 ; Sandoval, 2014 ). Taken together, the instructional tools instantiate a hypothesized learning environment that will advance both the theoretical and practical questions driving the research ( Barab, 2014 ).

In our work, the theoretical claim that instruction based on fundamental scientific concepts would support students’ flux reasoning was embodied in our instructional approach by being the central focus of all instructional materials, which included: a revised version of the Flux Reasoning Tool ( Figure 2 ); case study–based units in lecture that explicitly emphasized flux phenomena in real-world contexts ( Windschitl et al. , 2012 ; Scott et al. , 2018 ; Figure 3 ); classroom activities in which students practiced using flux to address physiological scenarios; links to online videos describing key flux-related concepts; constructed-response assessment items that cued students to use flux reasoning in their thinking; and pretest/posttest formative assessment questions that tracked student learning ( Figure 4 ).

The Flux Reasoning Tool given to students at the beginning of the quarter.

An example flux case study that is presented to students at the beginning of the neurophysiology unit. Throughout the unit, students learn how ion flows into and out of cells, as mediated by chemical and electrical gradients and various ion/molecular channels, sends signals throughout the body. They use this information to better understand why Jaime experiences persistent neuropathy. Images from: uz.wikipedia.org/wiki/Fayl:Blausen_0822_SpinalCord.png and commons.wikimedia.org/wiki/File:Figure_38_01_07.jpg.

An example flux assessment question about ion flows given in a pre-unit/post-unit formative assessment in the neurophysiology unit.

Phase 2: Testing the Instructional Tools

In the second phase of design-based research, the instructional tools are tested by implementing them in classrooms. During this phase, the instructional tools are placed “in harm’s way … in order to expose the details of the process to scrutiny” ( Cobb et al. , 2003 , p. 10). In this way, researchers and instructors test how the tools perform in real-world settings, which may differ considerably from the design team’s initial expectations ( Hoadley, 2004 ). During this phase, if necessary, the design team may make adjustments to the tools as they are being used to account for these unanticipated conditions ( Collins et al. , 2004 ).

We implemented the instructional tools during the Autumn and Spring quarters of the 2016–2017 academic year. Students were taught to use the Flux Reasoning Tool at the beginning of the term in the context of the first case study unit focused on neurophysiology. Each physiology unit throughout the term was associated with a new concept-based case study (usually about flux) that framed the context of the teaching. Embedded within the daily lectures were classroom activities in which students could practice using flux. Students were also assigned readings from the textbook and videos related to flux to watch during each unit. Throughout the term, students took five exams that each contained some flux questions as well as some pre- and post-unit formative assessment questions. During Winter quarter, we conducted clinical interviews with students who would take our course in the Spring term (i.e., “pre” data) as well as students who had just completed our course in Autumn (i.e., “post” data).

Phase 3: Evaluating the Instructional Tools

The third phase of a design-based research cycle involves evaluating the effectiveness of instructional tools using evidence of student learning ( Barab and Squire, 2004 ; Anderson and Shattuck, 2012 ). This can be done using products produced by students (e.g., homework, lab reports), attitudinal gains measured with surveys, participation rates in activities, interview testimonials, classroom discourse practices, and formative assessment or exam data (e.g., Reiser et al. , 2001 ; Cobb et al. , 2003 ; Barab and Squire, 2004 ; Mohan et al. , 2009 ). Regardless of the source, evidence must be in a form that supports a systematic analysis that could be scrutinized by other researchers ( Cobb et al. , 2003 ; Barab, 2014 ). Also, because design-based research often involves multiple data streams, researchers may need to use both quantitative and qualitative analytical methods to produce a rich picture of how the instructional tools affected student learning ( Collins et al. , 2004 ; Anderson and Shattuck, 2012 ).

In our work, we used the quality of students’ written responses on exams and formative assessment questions to determine whether students improved their understanding of physiological phenomena involving flux. For each assessment question, we analyzed a subset of student’s pretest answers to identify overarching patterns in students’ reasoning about flux, characterized these overarching patterns, then ordinated the patterns into different levels of sophistication. These became our scoring rubrics, which identified five different levels of student reasoning about flux. We used the rubrics to code the remainder of students’ responses, with a code designating the level of student reasoning associated with a particular reasoning pattern. We used this ordinal rubric format because it would later inform our theoretical understanding of how students build flux conceptual frameworks (see phase 4). This also allowed us to both characterize the ideas students held about flux phenomena and identify the frequency distribution of those ideas in a class.

By analyzing changes in the frequency distributions of students’ ideas across the rubric levels at different time points in the term (e.g., pre-unit vs. post-unit), we could track both the number of students who gained more sophisticated ideas about flux as the term progressed and the quality of those ideas. If the frequency of students reasoning at higher levels increased from pre-unit to post-unit assessments, we could conclude that our instructional tools as a whole were supporting students’ development of sophisticated flux ideas. For example, on one neuromuscular ion flux assessment question in the Spring of 2017, we found that relatively more students were reasoning at the highest levels of our rubric (i.e., levels 4 and 5) on the post-unit test compared with the pre-unit test. This meant that more students were beginning to integrate sophisticated ideas about flux (i.e., they were balancing concentration and electrical gradients) in their reasoning about ion movement.

To help validate this finding, we drew on three additional data streams: 1) from in-class group recordings of students working with flux items, we noted that students increasingly incorporated ideas about gradients and resistance when constructing their explanations as the term progressed; 2) from plant assessment items in the latter part of the term, we began to see students using flux ideas unprompted; and 3) from interviews, we observed that students who had already taken the course used flux ideas in their reasoning.

Through these analyses, we also noticed an interesting pattern in the pre-unit test data for Spring 2017 when compared with the frequency distribution of students’ responses with a previous term (Autumn 2016). In Spring 2017, 42% of students reasoned at level 4 or 5 on the pre-unit test, indicating these students already had sophisticated ideas about ion flux before they took the pre-unit assessment. This was surprising, considering only 2% of students reasoned at these levels for this item on the Autumn 2016 pre-unit test.

Phase 4: Reflecting on the Instructional Tools and Their Implementation

The final phase of a design-based research cycle involves a retrospective analysis that addresses the epistemic commitments of this methodology: How was the theory underpinning the research advanced by the research endeavor (theoretical outcome)? Did the instructional tools support student learning about the learning problem (practical outcome)? What were the critical features of the design solution that supported student learning (design principles)? ( Cobb et al. , 2003 ; Barab and Squire, 2004 ).

Theoretical Outcome (Epistemic Commitment 1).

Reflecting on how a design-based research experiment advances theory is critical to our understanding of how students learn in educational settings ( Barab and Squire, 2004 ; Mohan et al. , 2009 ). In our work, we aimed to characterize how students’ conceptual frameworks around flux change as they work toward robust ideas. To do this, we drew on learning progression research as our theoretical framing ( NRC, 2007 ; Corcoran et al. , 2009 ; Duschl et al. , 2011 ; Scott et al. , 2019 ). Learning progression frameworks describe empirically derived patterns in student thinking that are ordered into levels representing cognitive shifts in the ways students conceive a topic as they work toward mastery ( Gunckel et al. , 2012 ). We used our ion flux scoring rubrics to create a preliminary five-level learning progression framework ( Table 1 ). The framework describes how students’ ideas about flux often start with teleological-driven accounts at the lowest level (i.e., level 1), shift to focusing on driving forces (e.g., concentration gradients, electrical gradients) in the middle levels, and arrive at complex ideas that integrate multiple interacting forces at the higher levels. We further validated these reasoning patterns with our student interviews. However, our flux conceptual framework was largely based on student responses to our ion flux assessment items. Therefore, to further validate our learning progression framework, we needed a greater diversity of flux assessment items that investigated student thinking more broadly (i.e., about bulk flow, water movement) across physiological systems.

The preliminary flux learning progression framework characterizing the patterns of reasoning students may exhibit as they work toward mastery of flux reasoning. The student exemplars are from the ion flux formative assessment question presented in Figure 4 . The “/” divides a student’s answers to the first and second parts of the question. Level 5 represents the most sophisticated ideas about flux phenomena.

Practical Outcome (Epistemic Commitment 2).

In design-based research, learning theories must “do real work” by improving student learning in real-world settings ( DBR Collective, 2003 ). Therefore, design-based researchers must reflect on whether or not the data they collected show evidence that the instructional tools improved student learning ( Cobb et al. , 2003 ; Sharma and McShane, 2008 ). We determined whether our flux-based instructional approach aided student learning by analyzing the kinds of answers students provided to our assessment questions. Specifically, we considered students who reasoned at level 4 or above as demonstrating productive flux reasoning. Because almost half of students were reasoning at level 4 or 5 on the post-unit assessment after experiencing the instructional tools in the neurophysiology unit (in Spring 2017), we concluded that our tools supported student learning in physiology. Additionally, we noticed that students used language in their explanations that directly tied to the Flux Reasoning Tool ( Figure 2 ), which instructed them to use arrows to indicate the magnitude and direction of gradient-driving forces. For example, in a posttest response to our ion flux item ( Figure 4 ), one student wrote:

Ion movement is a function of concentration and electrical gradients . Which arrow is stronger determines the movement of K+. We can make the electrical arrow bigger and pointing in by making the membrane potential more negative than Ek [i.e., potassium’s equilibrium potential]. We can make the concentration arrow bigger and pointing in by making a very strong concentration gradient pointing in.

Given that almost half of students reasoned at level 4 or above, and that students used language from the Flux Reasoning Tool, we concluded that using fundamental concepts was a productive instructional approach for improving student learning in physiology and that our instructional tools aided student learning. However, some students in the 2016–2017 academic year continued to apply flux ideas more narrowly than intended (i.e., for ion and simple diffusion cases, but not water flux or bulk flow). This suggested that students had developed nascent flux conceptual frameworks after experiencing the instructional tools but could use more support to realize the broad applicability of this principle. Also, although our cross-sectional interview approach demonstrated how students’ ideas, overall, could change after experiencing the instructional tools, it did not provide information about how a student developed flux reasoning.

Reflecting on practical outcomes also means interpreting any learning gains in the context of the learning ecology. This reflection allowed us to identify whether there were particular aspects of the instructional tools that were better at supporting learning than others ( DBR Collective, 2003 ). Indeed, this was critical for our understanding why 42% of students scored at level 3 and above on the pre-unit ion assessment in the Spring of 2017, while only 2% of students scored level 3 and above in Autumn of 2016. When we reviewed notes of the Spring 2017 implementation scheme, we saw that the pretest was due at the end of the first day of class after students had been exposed to ion flux ideas in class and in a reading/video assignment about ion flow, which may be one reason for the students’ high performance on the pretest. Consequently, we could not tell whether students’ initial high performance was due to their learning from the activities in the first day of class or for other reasons we did not measure. It also indicated we needed to close pretests before the first day of class for a more accurate measure of students’ incoming ideas and the effectiveness of the instructional tools employed at the beginning of the unit.

Design Principles (Epistemic Commitment 3).

Although design-based research is enacted in local contexts (i.e., a particular classroom), its purpose is to inform learning ecologies that have broad applications to improve learning and teaching ( Edelson, 2002 ; Cobb et al. , 2003 ). Therefore, design-based research should produce design principles that describe characteristics of learning environments that researchers and instructors can use to develop instructional tools specific to their local contexts (e.g., Edelson, 2002 ; Subramaniam et al. , 2015 ). Consequently, the design principles must balance specificity with adaptability so they can be used broadly to inform instruction ( Collins et al. , 2004 ; Barab, 2014 ).

From our first cycle of design-based research, we developed the following design principles: 1) Key scientific concepts should provide an overarching framework for course organization. This way, the individual components that make up a course, like instructional units, activities, practice problems, and assessments, all reinforce the centrality of the key concept. 2) Instructional tools should explicitly articulate the principle of interest, with specific guidance on how that principle is applied in context. This stresses the applied nature of the principle and that it is more than a fact to be memorized. 3) Instructional tools need to show specific instances of how the principle is applied in multiple contexts to combat students’ narrow application of the principle to a limited number of contexts.

Design-Based Research Cycle 2, Phase 1: Redesign and Refine the Experiment

The last “epistemic commitment” Sandoval (2014) articulated was that design-based research be an iterative process with an eye toward continually refining the instructional tools, based on evidence of student learning, to produce more robust learning environments. By viewing educational inquiry as formative research, design-based researchers recognize the difficulty in accounting for all variables that could impact student learning, or the implementation of the instructional tools, a priori ( Collins et al. , 2004 ). Robust instructional designs are the products of trial and error, which are strengthened by a systematic analysis of how they perform in real-world settings.

To continue to advance our work investigating student thinking using the principle of flux, we began a second cycle of design-based research that continued to address the learning problem of helping students reason with fundamental scientific concepts. In this cycle, we largely focused on broadening the number of physiological systems that had accompanying formative assessment questions (i.e., beyond ion flux), collecting student reasoning from a more diverse population of students (e.g., upper division, allied heath, community college), and refining and validating the flux learning progression with both written and interview data in a student through time. We developed a suite of constructed-response flux assessment questions that spanned neuromuscular, cardiovascular, respiratory, renal, and plant physiological contexts and asked students about several kinds of flux: ion movement, diffusion, water movement, and bulk flow (29 total questions; available at beyondmultiplechoice.org). This would provide us with rich qualitative data that we could use to refine the learning progression. We decided to administer written assessments and conduct interviews in a pretest/posttest manner at the beginning and end of each unit both as a way to increase our data about student reasoning and to provide students with additional practice using flux reasoning across contexts.

From this second round of designing instructional tools (i.e., broader range of assessment items), testing them in the classroom (i.e., administering the assessment items to diverse student populations), evaluating the tools (i.e., developing learning progression–aligned rubrics across phenomena from student data, tracking changes in the frequency distribution of students across levels through time), and reflecting on the tools’ success, we would develop a more thorough and robust characterization of how students use flux across systems that could better inform our creation of new instructional tools to support student learning.

HOW CAN DESIGN-BASED RESEARCH EXTEND AND ENRICH BER?

While design-based research has primarily been used in educational inquiry at the K–12 level (see Reiser et al. , 2001 ; Mohan et al. , 2009 ; Jin and Anderson, 2012 ), other science disciplines at undergraduate institutions have begun to employ this methodology to create robust instructional approaches (e.g., Szteinberg et al. , 2014 in chemistry; Hake, 2007 , and Sharma and McShane, 2008 , in physics; Kelly, 2014 , in engineering). Our own work, as well as that by Zagallo et al. (2016) , provides two examples of how design-based research could be implemented in BER. Below, we articulate some of the ways incorporating design-based research into BER could extend and enrich this field of educational inquiry.

Design-Based Research Connects Theory with Practice

One critique of BER is that it does not draw heavily enough on learning theories from other disciplines like cognitive psychology or the learning sciences to inform its research ( Coley and Tanner, 2012 ; Dolan, 2015 ; Peffer and Renken, 2016 ; Davidesco and Milne, 2019 ). For example, there has been considerable work in BER developing concept inventories as formative assessment tools that identify concepts students often struggle to learn (e.g., Marbach-Ad et al. , 2009 ; McFarland et al. , 2017 ; Summers et al. , 2018 ). However, much of this work is detached from a theoretical understanding of why students hold misconceptions in the first place, what the nature of their thinking is, and the learning mechanisms that would move students to a more productive understanding of domain ideas ( Alonzo, 2011 ). Using design-based research to understand the basis of students’ misconceptions would ground these practical learning problems in a theoretical understanding of the nature of student thinking (e.g., see Coley and Tanner, 2012 , 2015 ; Gouvea and Simon, 2018 ) and the kinds of instructional tools that would best support the learning process.

Design-Based Research Fosters Collaborations across Disciplines

Recently, there have been multiple calls across science, technology, engineering, and mathematics education fields to increase collaborations between BER and other disciplines so as to increase the robustness of science education research at the collegiate level ( Coley and Tanner, 2012 ; NRC, 2012 ; Talanquer, 2014 ; Dolan, 2015 ; Peffer and Renken, 2016 ; Mestre et al. , 2018 ; Davidesco and Milne, 2019 ). Engaging in design-based research provides both a mechanism and a motivation for fostering interdisciplinary collaborations, as it requires the design team to have theoretical knowledge of how students learn, domain knowledge of practical learning problems, and instructional knowledge for how to implement instructional tools in the classroom ( Edelson, 2002 ; Hoadley, 2004 ; Wang and Hannafin, 2005 ; Anderson and Shattuck, 2012 ). For example, in our current work, our research team consists of two discipline-based education learning scientists from an R1 institution, two physiology education researchers/instructors (one from an R1 institution the other from a community college), several physiology disciplinary experts/instructors, and a K–12 science education expert.

Design-based research collaborations have several distinct benefits for BER: first, learning or cognitive scientists could provide theoretical and methodological expertise that may be unfamiliar to biology education researchers with traditional science backgrounds ( Lo et al. , 2019 ). This would both improve the rigor of the research project and provide biology education researchers with the opportunity to explore ideas and methods from other disciplines. Second, collaborations between researchers and instructors could help increase the implementation of evidence-based teaching practices by instructors/faculty who are not education researchers and would benefit from support while shifting their instructional approaches ( Eddy et al. , 2015 ). This may be especially true for community college and primarily undergraduate institution faculty who often do not have access to the same kinds of resources that researchers and instructors at research-intensive institutions do ( Schinske et al. , 2017 ). Third, making instructors an integral part of a design-based research project ensures they are well versed in the theory and learning objectives underlying the instructional tools they are implementing in the classroom. This can improve the fidelity of implementation of the instructional tools, because the instructors understand the tools’ theoretical and practical purposes, which has been cited as one reason there have been mixed results on the impact of active learning across biology classes ( Andrews et al. , 2011 ; Borrego et al. , 2013 ; Lee et al. , 2018 ; Offerdahl et al. , 2018 ). It also gives instructors agency to make informed adjustments to the instructional tools during implementation that improve their practical applications while remaining true to the goals of the research ( Hoadley, 2004 ).

Design-Based Research Invites Using Mixed Methods to Analyze Data

The diverse nature of the data that are often collected in design-based research can require both qualitative and quantitative methodologies to produce a rich picture of how the instructional tools and their implementation influenced student learning ( Anderson and Shattuck, 2012 ). Using mixed methods may be less familiar to biology education researchers who were primarily trained in quantitative methods as biologists ( Lo et al. , 2019 ). However, according to Warfa (2016 , p. 2), “Integration of research findings from quantitative and qualitative inquiries in the same study or across studies maximizes the affordances of each approach and can provide better understanding of biology teaching and learning than either approach alone.” Although the number of BER studies using mixed methods has increased over the past decade ( Lo et al. , 2019 ), engaging in design-based research could further this trend through its collaborative nature of bringing social scientists together with biology education researchers to share research methodologies from different fields. By leveraging qualitative and quantitative methods, design-based researchers unpack “mechanism and process” by characterizing the nature of student thinking rather than “simply reporting that differences did or did not occur” ( Barab, 2014 , p. 158), which is important for continuing to advance our understanding of student learning in BER ( Dolan, 2015 ; Lo et al. , 2019 ).

CHALLENGES TO IMPLEMENTING DESIGN-BASED RESEARCH IN BER

As with any methodological approach, there can be challenges to implementing design-based research. Here, we highlight three that may be relevant to BER.

Collaborations Can Be Difficult to Maintain

While collaborations between researchers and instructors offer many affordances (as discussed earlier), the reality of connecting researchers across departments and institutions can be challenging. For example, Peffer and Renken (2016) noted that different traditions of scholarship can present barriers to collaboration where there is not mutual respect for the methods and ideas that are part and parcel to each discipline. Additionally, Schinske et al. (2017) identified several constraints that community college faculty face for engaging in BER, such as limited time or support (e.g., infrastructural, administrative, and peer support), which could also impact their ability to form the kinds of collaborations inherent in design-based research. Moreover, the iterative nature of design-based research requires these collaborations to persist for an extended period of time. Attending to these challenges is an important part of forming the design team and identifying the different roles researchers and instructors will play in the research.

Design-Based Research Experiments Are Resource Intensive

The focus of design-based research on studying learning ecologies to uncover mechanisms of learning requires that researchers collect multiple data streams through time, which often necessitates significant temporal and financial resources ( Collins et al., 2004 ; O’Donnell, 2004 ). Consequently, researchers must weigh both practical as well as methodological considerations when formulating their experimental design. For example, investigating learning mechanisms requires that researchers collect data at a frequency that will capture changes in student thinking ( Siegler, 2006 ). However, researchers may be constrained in the number of data-collection events they can anticipate depending on: the instructor’s ability to facilitate in-class collection events or solicit student participation in extracurricular activities (e.g., interviews); the cost of technological devices to record student conversations; the time and logistical considerations needed to schedule and conduct student interviews; the financial resources available to compensate student participants; the financial and temporal costs associated with analyzing large amounts of data.

Identifying learning mechanisms also requires in-depth analyses of qualitative data as students experience various instructional tools (e.g., microgenetic methods; Flynn et al. , 2006 ; Siegler, 2006 ). The high intensity of these in-depth analyses often limits the number of students who can be evaluated in this way, which must be balanced with the kinds of generalizations researchers wish to make about the effectiveness of the instructional tools ( O’Donnell, 2004 ). Because of the large variety of data streams that could be collected in a design-based research experiment—and the resources required to collect and analyze them—it is critical that the research team identify a priori how specific data streams, and the methods of their analysis, will provide the evidence necessary to address the theoretical and practical objectives of the research (see the following section on experimental rigor; Sandoval, 2014 ). These are critical management decisions because of the need for a transparent, systematic analysis of the data that others can scrutinize to evaluate the validity of the claims being made ( Cobb et al. , 2003 ).

Concerns with Experimental Rigor

The nature of design-based research, with its use of narrative to characterize versus control experimental environments, has drawn concerns about the rigor of this methodological approach. Some have challenged its ability to produce evidence-based warrants to support its claims of learning that can be replicated and critiqued by others ( Shavelson et al. , 2003 ; Hoadley, 2004 ). This is a valid concern that design-based researchers, and indeed all education researchers, must address to ensure their research meets established standards for education research ( NRC, 2002 ).

One way design-based researchers address this concern is by “specifying theoretically salient features of a learning environment design and mapping out how they are predicted to work together to produce desired outcomes” ( Sandoval, 2014 , p. 19). Through this process, researchers explicitly show before they begin the work how their theory of learning is embodied in the instructional tools to be tested, the specific data the tools will produce for analysis, and what outcomes will be taken as evidence for success. Moreover, by allowing instructional tools to be modified during the testing phase as needed, design-based researchers acknowledge that it is impossible to anticipate all aspects of the classroom environment that might impact the implementation of instructional tools, “as dozens (if not millions) of factors interact to produce the measureable outcomes related to learning” ( Hoadley, 2004 , p. 204; DBR Collective, 2003 ). Consequently, modifying instructional tools midstream to account for these unanticipated factors can ensure they retain their methodological alignment with the underlying theory and predicted learning outcomes so that inferences drawn from the design experiment accurately reflect what was being tested ( Edelson, 2002 ; Hoadley, 2004 ). Indeed, Barab (2014) states, “the messiness of real-world practice must be recognized, understood, and integrated as part of the theoretical claims if the claims are to have real-world explanatory value” (p. 153).

CONCLUSIONS

In this essay, we have highlighted some of the ways design-based research can advance—and expand upon—research done in biology education. These ways include:

• providing a methodology that integrates theories of learning with practical experiences in classrooms,
• using a range of analytical approaches that allow for researchers to uncover the underlying mechanisms of student thinking and learning,
• fostering interdisciplinary collaborations among researchers and instructors, and
• characterizing learning ecologies that account for the complexity involved in student learning

By employing this methodology from the learning sciences, biology education researchers can enrich our current understanding of what is required to help biology students achieve their personal and professional aims during their college experience. It can also stimulate new ideas for biology education that can be discussed and debated in our research community as we continue to explore and refine how best to serve the students who pass through our classroom doors.

Acknowledgments

We thank the UW Biology Education Research Group’s (BERG) feedback on drafts of this essay as well as Dr. L. Jescovich for last-minute analyses. This work was supported by a National Science Foundation award (NSF DUE 1661263/1660643). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF. All procedures were conducted in accordance with approval from the Institutional Review Board at the University of Washington (52146) and the New England Independent Review Board (120160152).

1 “Epistemic commitment” is defined as engaging in certain practices that generate knowledge in an agreed-upon way.

• Alonzo, A. C. (2011). Learning progressions that support formative assessment practices . Measurement , 9 ( 2/3 ), 124–129. [ Google Scholar ]
• Anderson, T., Shattuck, J. (2012). Design-based research: A decade of progress in education research? Educational Researcher , 41 ( 1 ), 16–25. [ Google Scholar ]
• Andrews, T. M., Leonard, M. J., Colgrove, C. A., Kalinowski, S. T. (2011). Active learning not associated with student learning in a random sample of college biology courses . CBE—Life Sciences Education , 10 ( 4 ), 394–405. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Badenhorst, E., Hartman, N., Mamede, S. (2016). How biomedical misconceptions may arise and affect medical students’ learning: A review of theoretical perspectives and empirical evidence . Health Professions Education , 2 ( 1 ), 10–17. [ Google Scholar ]
• Barab, S. (2014). Design-based research: A methodological toolkit for engineering change . In The Cambridge handbook of the learning sciences (2nd ed., pp. 151–170). Cambridge University Press. 10.1017/CBO9781139519526.011 [ CrossRef ] [ Google Scholar ]
• Barab, S., Squire, K. (2004). Design-based research: Putting a stake in the ground . Journal of the Learning Sciences , 13 ( 1 ), 1–14. [ Google Scholar ]
• Bell, P. (2004). On the theoretical breadth of design-based research in education . Educational Psychologist , 39 ( 4 ), 243–253. [ Google Scholar ]
• Borrego, M., Cutler, S., Prince, M., Henderson, C., Froyd, J. E. (2013). Fidelity of implementation of research-based instructional strategies (RBIS) in engineering science courses . Journal of Engineering Education , 102 ( 3 ), 394–425. [ Google Scholar ]
• Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings . Journal of the Learning Sciences , 2 ( 2 ), 141–178. [ Google Scholar ]
• Chi, M. T. H., Roscoe, R. D., Slotta, J. D., Roy, M., Chase, C. C. (2012). Misconceived causal explanations for emergent processes . Cognitive Science , 36 ( 1 ), 1–61. [ PubMed ] [ Google Scholar ]
• Cobb, P., Confrey, J., diSessa, A., Lehrer, R., Schauble, L. (2003). Design experiments in educational research . Educational Researcher , 32 ( 1 ), 9–13. [ Google Scholar ]
• Coley, J. D., Tanner, K. D. (2012). Common origins of diverse misconceptions: Cognitive principles and the development of biology thinking . CBE—Life Sciences Education , 11 ( 3 ), 209–215. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Coley, J. D., Tanner, K. (2015). Relations between intuitive biological thinking and biological misconceptions in biology majors and nonmajors . CBE—Life Sciences Education , 14 ( 1 ). 10.1187/cbe.14-06-0094 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Collins, A., Joseph, D., Bielaczyc, K. (2004). Design research: Theoretical and methodological issues . Journal of the Learning Sciences , 13 ( 1 ), 15–42. [ Google Scholar ]
• Corcoran, T., Mosher, F. A., Rogat, A. D. (2009). Learning progressions in science: An evidence-based approach to reform (CPRE Research Report No. RR-63) . Philadelphia, PA: Consortium for Policy Research in Education. [ Google Scholar ]
• Davidesco, I., Milne, C. (2019). Implementing cognitive science and discipline-based education research in the undergraduate science classroom . CBE—Life Sciences Education , 18 ( 3 ), es4. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry . Educational Researcher , 32 ( 1 ), 5–8. [ Google Scholar ]
• Dolan, E. L. (2015). Biology education research 2.0 . CBE—Life Sciences Education , 14 ( 4 ), ed1. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Duschl, R., Maeng, S., Sezen, A. (2011). Learning progressions and teaching sequences: A review and analysis . Studies in Science Education , 47 ( 2 ), 123–182. [ Google Scholar ]
• Eddy, S. L., Converse, M., Wenderoth, M. P. (2015). PORTAAL: A classroom observation tool assessing evidence-based teaching practices for active learning in large science, technology, engineering, and mathematics classes . CBE—Life Sciences Education , 14 ( 2 ), ar23. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Edelson, D. C. (2002). Design research: What we learn when we engage in design . Journal of the Learning Sciences , 11 ( 1 ), 105–121. [ Google Scholar ]
• Flynn, E., Pine, K., Lewis, C. (2006). The microgenetic method—Time for change? The Psychologist , 19 ( 3 ), 152–155. [ Google Scholar ]
• Gouvea, J. S., Simon, M. R. (2018). Challenging cognitive construals: A dynamic alternative to stable misconceptions . CBE—Life Sciences Education , 17 ( 2 ), ar34. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Gunckel, K. L., Mohan, L., Covitt, B. A., Anderson, C. W. (2012). Addressing challenges in developing learning progressions for environmental science literacy . In Alonzo, A. C., Gotwals, A. W. (Eds.), Learning progressions in science: Current challenges and future directions (pp. 39–75). Rotterdam: SensePublishers. 10.1007/978-94-6091-824-7_4 [ CrossRef ] [ Google Scholar ]
• Hake, R. R. (2007). Design-based research in physics education research: A review . In Kelly, A. E., Lesh, R. A., Baek, J. Y. (Eds.), Handbook of design research methods in mathematics, science, and technology education (p. 24). New York: Routledge. [ Google Scholar ]
• Hoadley, C. M. (2004). Methodological alignment in design-based research . Educational Psychologist , 39 ( 4 ), 203–212. [ Google Scholar ]
• Jackson, M., Tran, A., Wenderoth, M. P., Doherty, J. H. (2018). Peer vs. self-grading of practice exams: Which is better? CBE—Life Sciences Education , 17 ( 3 ), es44. 10.1187/cbe.18-04-0052 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Jin, H., Anderson, C. W. (2012). A learning progression for energy in socio-ecological systems . Journal of Research in Science Teaching , 49 ( 9 ), 1149–1180. [ Google Scholar ]
• Joseph, D. (2004). The practice of design-based research: Uncovering the interplay between design, research, and the real-world context . Educational Psychologist , 39 ( 4 ), 235–242. [ Google Scholar ]
• Kelly, A. E. (2014). Design-based research in engineering education . In Cambridge handbook of engineering education research (pp. 497–518). New York, NY: Cambridge University Press. 10.1017/CBO9781139013451.032 [ CrossRef ] [ Google Scholar ]
• Lee, C. J., Toven-Lindsey, B., Shapiro, C., Soh, M., Mazrouee, S., Levis-Fitzgerald, M., Sanders, E. R. (2018). Error-discovery learning boosts student engagement and performance, while reducing student attrition in a bioinformatics course . CBE—Life Sciences Education , 17 ( 3 ), ar40. 10.1187/cbe.17-04-0061 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Lo, S. M., Gardner, G. E., Reid, J., Napoleon-Fanis, V., Carroll, P., Smith, E., Sato, B. K. (2019). Prevailing questions and methodologies in biology education research: A longitudinal analysis of research in CBE — life sciences education and at the society for the advancement of biology education research . CBE—Life Sciences Education , 18 ( 1 ), ar9. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Marbach-Ad, G., Briken, V., El-Sayed, N. M., Frauwirth, K., Fredericksen, B., Hutcheson, S., Smith, A. C. (2009). Assessing student understanding of host pathogen interactions using a concept inventory . Journal of Microbiology & Biology Education , 10 ( 1 ), 43–50. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• McFarland, J. L., Price, R. M., Wenderoth, M. P., Martinková, P., Cliff, W., Michael, J., … & Wright, A. (2017). Development and validation of the homeostasis concept inventory . CBE—Life Sciences Education , 16 ( 2 ), ar35. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• McKenney, S., Reeves, T. C. (2013). Systematic review of design-based research progress: Is a little knowledge a dangerous thing? Educational Researcher , 42 ( 2 ), 97–100. [ Google Scholar ]
• Mestre, J. P., Cheville, A., Herman, G. L. (2018). Promoting DBER-cognitive psychology collaborations in STEM education . Journal of Engineering Education , 107 ( 1 ), 5–10. [ Google Scholar ]
• Michael, J. A. (2007). What makes physiology hard for students to learn? Results of a faculty survey . AJP: Advances in Physiology Education , 31 ( 1 ), 34–40. [ PubMed ] [ Google Scholar ]
• Michael, J. A., Modell, H., McFarland, J., Cliff, W. (2009). The “core principles” of physiology: What should students understand? Advances in Physiology Education , 33 ( 1 ), 10–16. [ PubMed ] [ Google Scholar ]
• Middleton, J., Gorard, S., Taylor, C., Bannan-Ritland, B. (2008). The “compleat” design experiment: From soup to nuts . In Kelly, A. E., Lesh, R. A., Baek, J. Y. (Eds.), Handbook of design research methods in education: Innovations in science, technology, engineering, and mathematics learning and teaching (pp. 21–46). New York, NY: Routledge. [ Google Scholar ]
• Modell, H. I. (2000). How to help students understand physiology? Emphasize general models . Advances in Physiology Education , 23 ( 1 ), S101–S107. [ PubMed ] [ Google Scholar ]
• Mohan, L., Chen, J., Anderson, C. W. (2009). Developing a multi-year learning progression for carbon cycling in socio-ecological systems . Journal of Research in Science Teaching , 46 ( 6 ), 675–698. [ Google Scholar ]
• National Academies of Sciences, Engineering, and Medicine. (2018). How People Learn II: Learners, Contexts, and Cultures . Washington, DC: National Academies Press. Retrieved June 24, 2019, from 10.17226/24783 [ CrossRef ] [ Google Scholar ]
• National Research Council (NRC). (2002). Scientific research in education . Washington, DC: National Academies Press. Retrieved January 31, 2019, from 10.17226/10236 [ CrossRef ] [ Google Scholar ]
• NRC. (2007). Taking science to school: Learning and teaching science in grades K–8 . Washington, DC: National Academies Press. Retrieved March 22, 2019, from www.nap.edu/catalog/11625/taking-science-to-school-learning-and-teaching-science-in-grades . 10.17226/11625 [ CrossRef ] [ Google Scholar ]
• NRC. (2012). Discipline-based education research: Understanding and improving learning in undergraduate science and engineering . Washington, DC: National Academies Press. Retrieved from www.nap.edu/catalog/13362/discipline-based-education-research-understanding-and-improving-learning-in-undergraduate . 10.17226/13362 [ CrossRef ] [ Google Scholar ]
• NRC. (2018). How people learn II: Learners, contexts, and cultures . Washington, DC: National Academies Press. Retrieved from www.nap.edu/read/24783/chapter/7 . 10.17226/24783 [ CrossRef ] [ Google Scholar ]
• O’Donnell, A. M. (2004). A commentary on design research . Educational Psychologist , 39 ( 4 ), 255–260. [ Google Scholar ]
• Offerdahl, E. G., McConnell, M., Boyer, J. (2018). Can I have your recipe? Using a fidelity of implementation (FOI) framework to identify the key ingredients of formative assessment for learning . CBE—Life Sciences Education , 17 ( 4 ), es16. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Peffer, M., Renken, M. (2016). Practical strategies for collaboration across discipline-based education research and the learning sciences . CBE—Life Sciences Education , 15 ( 4 ), es11. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Reiser, B. J., Smith, B. K., Tabak, I., Steinmuller, F., Sandoval, W. A., Leone, A. J. (2001). BGuILE: Strategic and conceptual scaffolds for scientific inquiry in biology classrooms . In Carver, S. M., Klahr, D. (Eds.), Cognition and instruction: Twenty-five years of progress (pp. 263–305). Mahwah, NJ: Lawrence Erlbaum Associates, Inc. [ Google Scholar ]
• Sandoval, W. (2014). Conjecture mapping: An approach to systematic educational design research . Journal of the Learning Sciences , 23 ( 1 ), 18–36. [ Google Scholar ]
• Sandoval, W. A., Bell, P. (2004). Design-based research methods for studying learning in context: Introduction . Educational Psychologist , 39 ( 4 ), 199–201. [ Google Scholar ]
• Schinske, J. N., Balke, V. L., Bangera, M. G., Bonney, K. M., Brownell, S. E., Carter, R. S., … & Corwin, L. A. (2017). Broadening participation in biology education research: Engaging community college students and faculty . CBE—Life Sciences Education , 16 ( 2 ), mr1. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Scott, E., Anderson, C. W., Mashood, K. K., Matz, R. L., Underwood, S. M., Sawtelle, V. (2018). Developing an analytical framework to characterize student reasoning about complex processes . CBE—Life Sciences Education , 17 ( 3 ), ar49. 10.1187/cbe.17-10-0225 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Scott, E., Wenderoth, M. P., Doherty, J. H. (2019). Learning progressions: An empirically grounded, learner-centered framework to guide biology instruction . CBE—Life Sciences Education , 18 ( 4 ), es5. 10.1187/cbe.19-03-0059 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sharma, M. D., McShane, K. (2008). A methodological framework for understanding and describing discipline-based scholarship of teaching in higher education through design-based research . Higher Education Research & Development , 27 ( 3 ), 257–270. [ Google Scholar ]
• Shavelson, R. J., Phillips, D. C., Towne, L., Feuer, M. J. (2003). On the science of education design studies . Educational Researcher , 32 ( 1 ), 25–28. [ Google Scholar ]
• Siegler, R. S. (2006). Microgenetic analyses of learning . In Damon, W., Lerner, R. M. (Eds.), Handbook of child psychology (pp. 464–510). Hoboken, NJ: John Wiley & Sons, Inc. 10.1002/9780470147658.chpsy0211 [ CrossRef ] [ Google Scholar ]
• Stringer, E. T. (2013). Action research . Thousand Oaks, CA: Sage Publications, Inc. [ Google Scholar ]
• Subramaniam, M., Jean, B. S., Taylor, N. G., Kodama, C., Follman, R., Casciotti, D. (2015). Bit by bit: Using design-based research to improve the health literacy of adolescents . JMIR Research Protocols , 4 ( 2 ), e62. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Summers, M. M., Couch, B. A., Knight, J. K., Brownell, S. E., Crowe, A. J., Semsar, K., … & Batzli, J. (2018). EcoEvo-MAPS: An ecology and evolution assessment for introductory through advanced undergraduates . CBE—Life Sciences Education , 17 ( 2 ), ar18. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Szteinberg, G., Balicki, S., Banks, G., Clinchot, M., Cullipher, S., Huie, R., … & Sevian, H. (2014). Collaborative professional development in chemistry education research: Bridging the gap between research and practice . Journal of Chemical Education , 91 ( 9 ), 1401–1408. [ Google Scholar ]
• Talanquer, V. (2014). DBER and STEM education reform: Are we up to the challenge? Journal of Research in Science Teaching , 51 ( 6 ), 809–819. [ Google Scholar ]
• Wang, F., Hannafin, M. J. (2005). Design-based research and technology-enhanced learning environments . Educational Technology Research and Development , 53 ( 4 ), 5–23. [ Google Scholar ]
• Wang, J.-R. (2004). Development and validation of a Two-tier instrument to examine understanding of internal transport in plants and the human circulatory system . International Journal of Science and Mathematics Education , 2 ( 2 ), 131–157. [ Google Scholar ]
• Warfa, A.-R. M. (2016). Mixed-methods design in biology education research: Approach and uses . CBE—Life Sciences Education , 15 ( 4 ), rm5. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Windschitl, M., Thompson, J., Braaten, M., Stroupe, D. (2012). Proposing a core set of instructional practices and tools for teachers of science . Science Education , 96 ( 5 ), 878–903. [ Google Scholar ]
• Zagallo, P., Meddleton, S., Bolger, M. S. (2016). Teaching real data interpretation with models (TRIM): Analysis of student dialogue in a large-enrollment cell and developmental biology course . CBE—Life Sciences Education , 15 ( 2 ), ar17. [ PMC free article ] [ PubMed ] [ Google Scholar ]

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Kimberly Christensen and Richard E. West

Design-Based Research (DBR) is one of the most exciting evolutions in research methodology of our time, as it allows for the potential knowledge gained through the intimate connections designers have with their work to be combined with the knowledge derived from research. These two sources of knowledge can inform each other, leading to improved design interventions as well as improved local and generalizable theory. However, these positive outcomes are not easily attained, as DBR is also a difficult method to implement well. The good news is that we can learn much from other disciplines who are also seeking to find effective strategies for intertwining design and research. In this chapter, we will review the history of DBR as well as Interdisciplinary Design Research (IDR) and then discuss potential implications for our field.

Shared Origins with IDR

These two types of design research, both DBR and IDR, share a common genesis among the design revolution of the 1960s, where designers, researchers, and scholars sought to elevate design from mere practice to an independent scholarly discipline, with its own research and distinct theoretical and methodological underpinnings. A scholarly focus on design methods, they argued, would foster the development of design theories, which would in turn improve the quality of design and design practice (Margolin, 2010). Research on design methods, termed design research, would be the foundation of this new discipline.

Design research had existed in primitive form—as market research and process analysis—since before the turn of the 20th century, and, although it had served to improve processes and marketing, it had not been applied as scientific research. John Chris Jones, Bruce Archer, and Herbert Simon were among the first to shift the focus from research for design (e.g., research with the intent of gathering data to support product development) to research on design (e.g., research exploring the design process). Their efforts framed the initial development of design research and science.

John Chris Jones

An engineer, Jones (1970) felt that the design process was ambiguous and often too abstruse to discuss effectively. One solution, he offered, was to define and discuss design in terms of methods. By identifying and discussing design methods, researchers would be able to create transparency in the design process, combating perceptions of design being more or less mysteriously inspired. This discussion of design methods, Jones proposed, would in turn raise the level of discourse and practice in design.

Bruce Archer

Archer, also an engineer, worked with Jones and likewise supported the adoption of research methods from other disciplines. Archer (1965) proposed that applying systematic methods would improve the assessment of design problems and foster the development of effective solutions. Archer recognized, however, that improved practice alone would not enable design to achieve disciplinary status. In order to become a discipline, design required a theoretical foundation to support its practice. Archer (1981) advocated that design research was the primary means by which theoretical knowledge could be developed. He suggested that the application of systematic inquiry, such as existed in engineering, would yield knowledge about not only product and practice, but also the theory that guided each.

Herbert Simon

It was multidisciplinary social scientist Simon, however, that issued the clarion call for transforming design into design science (Buchanan, 2007; Collins, 1992; Collins, Joseph, & Bielaczyc, 2004; Cross, 1999; Cross, 2007; Friedman, 2003; Jonas, 2007; Willemien, 2009). In The Sciences of the Artificial, Simon (1969) reasoned that the rigorous inquiry and discussion surrounding naturally occurring processes and phenomena was just as necessary for man-made products and processes. He particularly called for “[bodies] of intellectually tough, analytic, partly formalizable, partly empirical, teachable doctrine about the design process” (p. 132). This call for more scholarly discussion and practice resonated with designers across disciplines in design and engineering (Buchanan, 2007; Cross, 1999; Cross, 2007; Friedman, 2003; Jonas, 2007; Willemien, 2009). IDR sprang directly from this early movement and has continued to gain momentum, producing an interdisciplinary body of research encompassing research efforts in engineering, design, and technology.

Years later, in the 1980s, Simon’s work inspired the first DBR efforts in education (Collins et al., 2004). Much of the DBR literature attributes its beginnings to the work of Ann Brown and Allan Collins (Cobb, Confrey, diSessa, Lehrer, & Schauble, 2003; Collins et al., 2004; Kelly, 2003; McCandliss, Kalchman, & Bryant, 2003; Oh & Reeves, 2010; Reeves, 2006; Shavelson, Phillips, Towne, & Feuer, 2003; Tabak, 2004; van den Akker, 1999). Their work, focusing on research and development in authentic contexts, drew heavily on research approaches and development practices in the design sciences, including the work of early design researchers such as Simon (Brown, 1992; Collins, 1992; Collins et al., 2004). However, over generations of research, this connection has been all but forgotten, and DBR, although similarly inspired by the early efforts of Simon, Archer, and Jones, has developed into an isolated and discipline-specific body of design research, independent from its interdisciplinary cousin.

Current Issues in DBR

The initial obstacle to understanding and engaging in DBR is understanding what DBR is. What do we call it? What does it entail? How do we do it? Many of the current challenges facing DBR concern these questions. Specifically, there are three issues that influence how DBR is identified, implemented, and discussed. First, proliferation of terminology among scholars and inconsistent use of these terms have created a sprawling body of literature, with various splinter DBR groups hosting scholarly conversations regarding their particular brand of DBR. Second, DBR, as a field, is characterized by a lack of definition, in terms of its purpose, its characteristics, and the steps or processes of which it is comprised. Third, the one consistent element of DBR across the field is an unwieldy set of considerations incumbent upon the researcher.

Because it is so difficult to define and conceptualize DBR, it is similarly difficult to replicate authentically. Lack of scholarly agreement on the characteristics and outcomes that define DBR withholds a structure by which DBR studies can be identified and evaluated and, ultimately, limits the degree to which the field can progress. The following sections will identify and explore the three greatest challenges facing DBR today: proliferation of terms, lack of definition, and competing demands.

Proliferation of terminology

One of the most challenging characteristics of DBR is the quantity and use of terms that identify DBR in the research literature. There are seven common terms typically associated with DBR: design experiments, design research, design-based research, formative research, development research, developmental research, and design-based implementation research.

Synonymous terms

Collins and Brown first termed their efforts design experiments (Brown, 1992; Collins, 1992). Subsequent literature stemming from or relating to Collins’ and Brown’s work used design research and design experiments synonymously (Anderson & Shattuck, 2012; Collins et al., 2004). Design-based research was introduced to distinguish DBR from other research approaches. Sandoval and Bell (2004) best summarized this as follows:

We have settled on the term design-based research over the other commonly used phrases “design experimentation,” which connotes a specific form of controlled experimentation that does not capture the breadth of the approach, or “design research,” which is too easily confused with research design and other efforts in design fields that lack in situ research components. (p. 199)

Variations by discipline

Terminology across disciplines refers to DBR approaches as formative research, development research, design experiments, and developmental research. According to van den Akker (1999), the use of DBR terminology also varies by educational sub-discipline, with areas such as (a) curriculum, (b) learning and instruction, (c) media and technology, and (d) teacher education and didactics favoring specific terms that reflect the focus of their research (Figure 1).

Figure 1. Variations in DBR terminology across educational sub-disciplines.

Lack of definition

This variation across disciplines, with design researchers tailoring design research to address discipline-specific interests and needs, has created a lack of definition in the field overall. In addition, in the literature, DBR has been conceptualized at various levels of granularity. Here, we will discuss three existing approaches to defining DBR: (a) statements of the overarching purpose, (b) lists of defining characteristics, and (c) models of the steps or processes involved.

General purpose

In literature, scholars and researchers have made multiple attempts to isolate the general purpose of design research in education, with each offering a different insight and definition. According to van den Akker (1999), design research is distinguished from other research efforts by its simultaneous commitment to (a) developing a body of design principles and methods that are based in theory and validated by research and (b) offering direct contributions to practice. This position was supported by Sandoval and Bell (2004), who suggested that the general purpose of DBR was to address the “tension between the desire for locally usable knowledge, on the one hand, and scientifically sound, generalizable knowledge on the other” (p. 199). Cobb et al. (2003) particularly promoted the theory-building focus, asserting “design experiments are conducted to develop theories, not merely to empirically tune ‘what works’” (p. 10). Shavelson et al. (2003) recognized the importance of developing theory but emphasized that the testing and building of instructional products was an equal focus of design research rather than the means to a theoretical end.

The aggregate of these definitions suggests that the purpose of DBR involves theoretical and practical design principles and active engagement in the design process. However, DBR continues to vary in its prioritization of these components, with some focusing largely on theory, others emphasizing practice or product, and many examining neither but all using the same terms.

Specific characteristics

Another way to define DBR is by identifying the key characteristics that both unite and define the approach. Unlike other research approaches, DBR can take the form of multiple research methodologies, both qualitative and quantitative, and thus cannot be recognized strictly by its methods. Identifying characteristics, therefore, concern the research process, context, and focus. This section will discuss the original characteristics of DBR, as introduced by Brown and Collins, and then identify the seven most common characteristics suggested by DBR literature overall.

Brown’s concept of DBR. Brown (1992) defined design research as having five primary characteristics that distinguished it from typical design or research processes. First, a design is engineered in an authentic, working environment. Second, the development of research and the design are influenced by a specific set of inputs: classroom environment, teachers and students as researchers, curriculum, and technology. Third, the design and development process includes multiple cycles of testing, revision, and further testing. Fourth, the design research process produces an assessment of the design’s quality as well as the effectiveness of both the design and its theoretical underpinnings. Finally, the overall process should make contributions to existing learning theory.

Collins’s concept of DBR. Collins (1990, 1992) posed a similar list of design research characteristics. Collins echoed Brown’s specifications of authentic context, cycles of testing and revision, and design and process evaluation. Additionally, Collins provided greater detail regarding the characteristics of the design research processes—specifically, that design research should include the comparison of multiple sample groups, be systematic in both its variation within the experiment and in the order of revisions (i.e., by testing the innovations most likely to succeed first), and involve an interdisciplinary team of experts including not just the teacher and designer, but technologists, psychologists, and developers as well. Unlike Brown, however, Collins did not refer to theory building as an essential characteristic.

Current DBR characteristics. The DBR literature that followed expanded, clarified, and revised the design research characteristics identified by Brown and Collins. The range of DBR characteristics discussed in the field currently is broad but can be distilled to seven most frequently referenced identifying characteristics of DBR: design driven, situated, iterative, collaborative, theory building, practical, and productive.

Design driven.  All literature identifies DBR as focusing on the evolution of a design (Anderson & Shattuck, 2012; Brown, 1992; Cobb et al., 2003; Collins, 1992; Design-Based Research Collective, 2003). While the design can range from an instructional artifact to an intervention, engagement in the design process is what yields the experience, data, and insight necessary for inquiry.

Situated.  Recalling Brown’s (1992) call for more authentic research contexts, nearly all definitions of DBR situate the aforementioned design process in a real-world context, such as a classroom (Anderson & Shattuck, 2012; Barab & Squire, 2004; Cobb et al., 2003).

Iterative. Literature also appears to agree that a DBR process does not consist of a linear design process, but rather multiple cycles of design, testing, and revision (Anderson & Shattuck, 2012; Barab & Squire, 2004; Brown, 1992; Design-Based Research Collective, 2003; Shavelson et al., 2003). These iterations must also represent systematic adjustment of the design, with each adjustment and subsequent testing serving as a miniature experiment (Barab & Squire, 2004; Collins, 1992).

Collaborative.  While the literature may not always agree on the roles and responsibilities of those engaged in DBR, collaboration between researchers, designers, and educators appears to be key (Anderson & Shattuck, 2012; Barab & Squire, 2004; McCandliss et al., 2003). Each collaborator enters the project with a unique perspective and, as each engages in research, forms a role-specific view of phenomena. These perspectives can then be combined to create a more holistic view of the design process, its context, and the developing product.

Theory building.  Design research focuses on more than creating an effective design; DBR should produce an intimate understanding of both design and theory (Anderson & Shattuck, 2012; Barab & Squire, 2004; Brown, 1992; Cobb et al., 2003; Design-Based Research Collective, 2003; Joseph, 2004; Shavelson et al., 2003). According to Barab & Squire (2004), “Design-based research requires more than simply showing a particular design works but demands that the researcher . . . generate evidence-based claims about learning that address contemporary theoretical issues and further the theoretical knowledge of the field” (p. 6). DBR needs to build and test theory, yielding findings that can be generalized to both local and broad theory (Hoadley, 2004).

Practical.  While theoretical contributions are essential to DBR, the results of DBR studies “must do real work” (Cobb et al., 2003, p. 10) and inform instructional, research, and design practice (Anderson & Shattuck, 2012; Barab & Squire, 2004; Design-Based Research Collective, 2003; McCandliss et al., 2003).

Productive.  Not only should design research produce theoretical and practical insights, but also the design itself must produce results, measuring its success in terms of how well the design meets its intended outcomes (Barab & Squire, 2004; Design-Based Research Collective, 2003; Joseph, 2004; McCandliss et al., 2003).

Steps and processes

The third way DBR could possibly be defined is to identify the steps or processes involved in implementing it. The sections below illustrate the steps outlined by Collins (1990) and Brown (1992) as well as models by Bannan-Ritland (2003), Reeves (2006), and an aggregate model presented by Anderson & Shattuck (2012).

Collins’s design experimentation steps.  In his technical report, Collins (1990) presented an extensive list of 10 steps in design experimentation (Figure 2). While Collins’s model provides a guide for experimentally testing and developing new instructional programs, it does not include multiple iterative stages or any evaluation of the final product. Because Collins was interested primarily in development, research was not given much attention in his model.

Brown’s design research example.  The example of design research Brown (1992) included in her article was limited and less clearly delineated than Collins’s model (Figure 2). Brown focused on the development of educational interventions, including additional testing with minority populations. Similar to Collins, Brown also omitted any summative evaluation of intervention quality or effectiveness and did not specify the role of research through the design process.

Bannan-Ritland’s DBR model.  Bannan-Ritland (2003) reviewed design process models in fields such as product development, instructional design, and engineering to create a more sophisticated model of design-based research. In its simplest form, Bannan-Ritland’s model is comprised of multiple processes subsumed under four broad stages: (a) informed exploration, (b) enactment, (c) evaluation of local impact, and (d) evaluation of broad impact. Unlike Collins and Brown, Bannan-Ritland dedicated large portions of the model to evaluation in terms of the quality and efficacy of the final product as well as the implications for theory and practice.

Reeves’s development research model.  Reeves (2006) provided a simplified model consisting of just four steps (Figure 2). By condensing DBR into just a few steps, Reeves highlighted what he viewed as the most essential processes, ending with a general reflection on both the process and product generated in order to develop theoretical and practical insights.

Anderson and Shattuck’s aggregate model.  Anderson and Shattuck (2012) reviewed design-based research abstracts over the past decade and, from their review, presented an eight-step aggregate model of DBR (Figure 2). As an aggregate of DBR approaches, this model was their attempt to unify approaches across DBR literature, and includes similar steps to Reeves’s model. However, unlike Reeves, Anderson and Shattuck did not include summative reflection and insight development.

Comparison of models. Following in Figure 2, we provide a comparison of all these models side-by-side.

Competing demands and roles

The third challenge facing DBR is the variety of roles researchers are expected to fulfill, with researchers often acting simultaneously as project managers, designers, and evaluators. However, with most individuals able to focus on only one task at a time, these competing demands on resources and researcher attention and faculties can be challenging to balance, and excess focus on one role can easily jeopardize others. The literature has recognized four major roles that a DBR professional must perform simultaneously: researcher, project manager, theorist, and designer.

Researcher as researcher

Planning and carrying out research is already comprised of multiple considerations, such as controlling variables and limiting bias. The nature of DBR, with its collaboration and situated experimentation and development, innately intensifies some of these issues (Hoadley, 2004). While simultaneously designing the intervention, a design-based researcher must also ensure that high-quality research is accomplished, per typical standards of quality associated with quantitative or qualitative methods.

However, research is even more difficult in DBR because the nature of the method leads to several challenges. First, it can be difficult to control the many variables at play in authentic contexts (Collins et al., 2004). Many researchers may feel torn between being able to (a) isolate critical variables or (b) study the comprehensive, complex nature of the design experience (van den Akker, 1999). Second, because many DBR studies are qualitative, they produce large amounts of data, resulting in demanding data collection and analysis (Collins et al., 2004). Third, according to Anderson and Shattuck (2012), the combination of demanding data analysis and highly invested roles of the researchers leaves DBR susceptible to multiple biases during analysis. Perhaps best expressed by Barab and Squire (2004), “if a researcher is intimately involved in the conceptualization, design, development, implementation, and researching of a pedagogical approach, then ensuring that researchers can make credible and trustworthy assertions is a challenge” (p. 10). Additionally, the assumption of multiple roles invests much of the design and research in a single person, diminishing the likelihood of replicability (Hoadley, 2004). Finally, it is impossible to document or account for all discrete decisions made by the collaborators that influenced the development and success of the design (Design-Based Research Collective, 2003).

Quality research, though, was never meant to be easy! Despite these challenges, DBR has still been shown to be effective in simultaneously developing theory through research as well as interventions that can benefit practice—the two simultaneous goals of any instructional designer.

Researcher as project manager

The collaborative nature of DBR lends the approach one of its greatest strengths: multiple perspectives. While this can be a benefit, collaboration between researchers, developers, and practitioners needs to be highly coordinated (Collins et al., 2004), because it is difficult to manage interdisciplinary teams and maintain a productive, collaborative partnership (Design-Based Research Collective, 2003).

Researcher as theorist

For many researchers in DBR, the development or testing of theory is a foundational component and primary focus of their work. However, the iterative and multi-tasking nature of a DBR process may not be well-suited to empirically testing or building theory. According to Hoadley (2004), “the treatment’s fidelity to theory [is] initially, and sometimes continually, suspect” (p. 204). This suggests that researchers, despite intentions to test or build theory, may not design or implement their solution in alignment with theory or provide enough control to reliably test the theory in question.

Researcher as designer

Because DBR is simultaneously attempting to satisfy the needs of both design and research, there is a tension between the responsibilities of the researcher and the responsibilities of the designer (van den Akker, 1999). Any design decision inherently alters the research. Similarly, research decisions place constraints on the design. Skilled design-based researchers seek to balance these competing demands effectively.

What we can learn from IDR

IDR has been encumbered by similar issues that currently exist in DBR. While IDR is by no means a perfect field and is still working to hone and clarify its methods, it has been developing for two decades longer than DBR. The history of IDR and efforts in the field to address similar issues can yield possibilities and insights for the future of DBR. The following sections address efforts in IDR to define the field that hold potential for application in DBR, including how professionals in IDR have focused their efforts to increase unity and worked to define sub-approaches more clearly.

Defining Approaches

Similar to DBR, IDR has been subject to competing definitions as varied as the fields in which design research has been applied (i.e., product design, engineering, manufacturing, information technology, etc.) (Findeli, 1998; Jonas, 2007; Schneider, 2007). Typically, IDR scholars have focused on the relationship between design and research, as well as the underlying purpose, to define the approach. This section identifies three defining conceptualizations of IDR—the prepositional approach trinity, Cross’s -ologies, and Buchanan’s strategies of productive science—and discusses possible implications for DBR.

The approach trinity

One way of defining different purposes of design research is by identifying the preposition in the relationship between research and design: research into design, research for design, and research through design (Buchanan, 2007; Cross, 1999; Findeli, 1998; Jonas, 2007; Schneider, 2007).

Jonas (2007) identified research into design as the most prevalent—and straightforward—form of IDR. This approach separates research from design practice; the researcher observes and studies design practice from without, commonly addressing the history, aesthetics, theory, or nature of design (Schneider, 2007). Research into design generally yields little or no contribution to broader theory (Findeli, 1998).

Research for design applies to complex, sophisticated projects, where the purpose of research is to foster product research and development, such as in market and user research (Findeli, 1998; Jonas, 2007). Here, the role of research is to build and improve the design, not contribute to theory or practice.

According to Jonas’s (2007) description, research through design bears the strongest resemblance to DBR and is where researchers work to shape their design (i.e., the research object) and establish connections to broader theory and practice. This approach begins with the identification of a research question and carries through the design process experimentally, improving design methods and finding novel ways of controlling the design process (Schneider, 2007). According to Findeli (1998), because this approach adopts the design process as the research method, it helps to develop authentic theories of design.

Cross’s -ologies

Cross (1999) conceived of IDR approaches based on the early drive toward a science of design and identified three bodies of scientific inquiry: epistemology, praxiology, and phenomenology. Design epistemology primarily concerns what Cross termed “designerly ways of knowing” or how designers think and communicate about design (Cross, 1999; Cross, 2007). Design praxiology deals with practices and processes in design or how to develop and improve artifacts and the processes used to create them. Design phenomenology examines the form, function, configuration, and value of artifacts, such as exploring what makes a cell phone attractive to a user or how changes in a software interface affect user’s activities within the application.

Buchanan’s strategies of productive science

Like Cross, Buchanan (2007) viewed IDR through the lens of design science and identified four research strategies that frame design inquiry: design science, dialectic inquiry, rhetorical inquiry, and productive science (Figure 2). Design science focuses on designing and decision-making, addressing human and consumer behavior. According to Buchanan (2007), dialectic inquiry examines the “social and cultural context of design; typically [drawing] attention to the limitations of the individual designer in seeking sustainable solutions to problems” (p.57). Rhetorical inquiry focuses on the design experience as well as the designer’s process to create products that are usable, useful, and desirable. Productive science studies how the potential of a design is realized through the refinement of its parts, including materials, form, and function. Buchanan (2007) conceptualized a design research—what he termed design inquiry—that includes elements of all four strategies, looking at the designer, the design, the design context, and the refinement process as a holistic experience.

Implications for DBR

While the literature has yet to accept any single approach to defining types of IDR, it may still be helpful for DBR to consider similar ways of limiting and defining sub-approaches in the field. The challenges brought on by collaboration, multiple researcher roles, and lack of sufficient focus on the design product could be addressed and relieved by identifying distinct approaches to DBR. This idea is not new. Bell and Sandoval (2004) opposed the unification of DBR, specifically design-based research, across educational disciplines (such as developmental psychology, cognitive science, and instructional design). However, they did not suggest any potential alternatives. Adopting an IDR approach, such as the approach trinity, could serve to both unite studies across DBR and clearly distinguish the purpose of the approach and its primary functions. Research into design could focus on the design process and yield valuable insights on design thinking and practice. Research for design could focus on the development of an effective product, which development is missing from many DBR approaches. Research through design would use the design process as a vehicle to test and develop theory, reducing the set of expected considerations. Any approach to dividing or defining DBR efforts could help to limit the focus of the study, helping to prevent the diffusion of researcher efforts and findings.

In this chapter we have reviewed the historical development of both design-based research and interdisciplinary design research in an effort to identify strategies in IDR that could benefit DBR development. Following are a few conclusions, leading to recommendations for the DBR field.

Improve interdisciplinary collaboration

Overall, one key advantage that IDR has had—and that DBR presently lacks—is communication and collaboration with other fields. Because DBR has remained so isolated, only rarely referencing or exploring approaches from other design disciplines, it can only evolve within the constraints of educational inquiry. IDR’s ability to conceive solutions to issues in the field is derived, in part, from a wide variety of disciplines that contribute to the body of research. Engineers, developers, artists, and a range of designers interpose their own ideas and applications, which are in turn adopted and modified by others. Fostering collaboration between DBR and IDR, while perhaps not the remedy to cure all scholarly ills, could yield valuable insights for both fields, particularly in terms of refining methodologies and promoting the development of theory.

Simplify terminology and improve consistency in use

As we identified in this paper, a major issue facing DBR is the proliferation of terminology among scholars and the inconsistency in usage. From IDR comes the useful acknowledgement that there can be research into design, for design, and through design (Buchanan, 2007; Cross, 1999; Findeli, 1998; Jonas, 2007; Schneider, 2007). This framework was useful for scholars in our conversations at the conference. A resulting recommendation, then, is that, in published works, scholars begin articulating which of these approaches they are using in that particular study. This can simplify the requirements on DBR researchers, because instead of feeling the necessity of doing all three in every paper, they can emphasize one. This will also allow us to communicate our research better with IDR scholars.

Describe DBR process in publications

Oftentimes authors publish DBR studies using the same format as regular research studies, making it difficult to recognize DBR research and learn how other DBR scholars mitigate the challenges we have discussed in this chapter. Our recommendation is that DBR scholars publish the messy findings resulting from their work and pull back the curtain to show how they balanced competing concerns to arrive at their results. We believe it would help if DBR scholars adopted more common frameworks for publishing studies. In our review of the literature, we identified the following characteristics, which are the most frequently used to identify DBR:

• DBR is design driven and intervention focused
• DBR is situated within an actual teaching/learning context
• DBR is iterative
• DBR is collaborative between researchers, designers, and practitioners
• DBR builds theory but also needs to be practical and result in useful interventions

One recommendation is that DBR scholars adopt these as the characteristics of their work that they will make explicit in every published paper so that DBR articles can be recognized by readers and better aggregated together to show the value of DBR over time. One suggestion is that DBR scholars in their methodology sections could adopt these characteristics as subheadings. So in addition to discussing data collection and data analysis, they would also discuss Design Research Type (research into, through, or of design), Description of the Design Process and Product, Design and Learning Context, Design Collaborations, and a discussion explicitly of the Design Iterations, perhaps by listing each iteration and then the data collection and analysis for each. Also in the concluding sections, in addition to discussing research results, scholars would discuss Applications to Theory (perhaps dividing into Local Theory and Outcomes and Transferable Theory and Findings) and Applications for Practice. Papers that are too big could be broken up with different papers reporting on different iterations but using this same language and formatting to make it easier to connect the ideas throughout the papers. Not all papers would have both local and transferable theory (the latter being more evident in later iterations), so it would be sufficient to indicate in a paper that local theory and outcomes were developed and met with some ideas for transferable theory that would be developed in future iterations. The important thing would be to refer to each of these main characteristics in each paper so that scholars can recognize the work as DBR, situate it appropriately, and know what to look for in terms of quality during the review process.

Application Exercises

• According to the authors, what are the major issues facing DBR and what are some things that can be done to address this problem?
• Imagine you have designed a new learning app for use in public schools. How would you go about testing it using design-based research?

Anderson, T., & Shattuck, J. (2012). Design-based research: A decade of progress in education research? Educational Researcher, 41 (1), 16–25.

Archer, L.B. (1965). Systematic method for designers. In N. Cross (ed.), Developments in design methodology. London, England: John Wiley, 1984, pp. 57–82.

Archer, L. B. (1981). A view of the nature of design research. In R. Jacques & J.A. Powell (Eds.), Design: Science: Method (pp. 36-39). Guilford, England: Westbury House.

Bannan-Ritland, B. (2003). The role of design in research: The integrative learning design framework. Educational Researcher, 32 (1), 21 –24. doi:10.3102/0013189X032001021

Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. The Journal of the Learning Sciences, 13 (1), 1–14.

Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Learning Sciences, 2 (2), 141–178.

Buchanan, R. (2007). Strategies of design research: Productive science and rhetorical inquiry. In R. Michel (Ed.), Design research now (pp. 55–66). Basel, Switzerland: Birkhäuser Verlag AG.

Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32 (1), 9–13. doi:10.3102/0013189X032001009

Collins, A. (1990). Toward a Design Science of Education. Technical Report No. 1.

Collins, A. (1992). Toward a design science of education. In E. Scanlon & T. O’Shea (Eds.), New directions in educational technology. Berlin, Germany: Springer-Verlag.

Collins, A., Joseph, D., & Bielaczyc, K. (2004). Design research: Theoretical and methodological issues. The Journal of the Learning Sciences, 13 (1), 15–42.

Cross, N. (1999). Design research: A disciplined conversation. Design Issues, 15 (2), 5–10. doi:10.2307/1511837

Cross, N. (2007). Forty years of design research. Design Studies, 28 (1), 1–4. doi:10.1016/j.destud.2006.11.004

Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32 (1), 5–8. doi:10.3102/0013189X032001005

Findeli, A. (1998). A quest for credibility: Doctoral education and research in design at the University of Montreal. Doctoral Education in Design, Ohio, 8–11 October 1998.

Friedman, K. (2003). Theory construction in design research: Criteria: approaches, and methods. Design Studies, 24 (6), 507–522.

Hoadley, C. M. (2004). Methodological alignment in design-based research. Educational Psychologist, 39 (4), 203–212.

Jonas, W. (2007). Design research and its meaning to the methodological development of the discipline. In R. Michel (Ed.), Design research now (pp. 187–206). Basel, Switzerland: Birkhäuser Verlag AG.

Jones, J. C. (1970). Design methods: Seeds of human futures. New York, NY: John Wiley & Sons Ltd.

Joseph, D. (2004). The practice of design-based research: uncovering the interplay between design, research, and the real-world context. Educational Psychologist, 39 (4), 235–242.

Kelly, A. E. (2003). Theme issue: The role of design in educational research. Educational Researcher, 32 (1), 3–4. doi:10.3102/0013189X032001003

Margolin, V. (2010). Design research: Towards a history. Presented at the Design Research Society Annual Conference on Design & Complexity, Montreal, Canada. Retrieved from http://www.drs2010.umontreal.ca/data/PDF/080.pdf

McCandliss, B. D., Kalchman, M., & Bryant, P. (2003). Design experiments and laboratory approaches to learning: Steps toward collaborative exchange. Educational Researcher, 32 (1), 14–16. doi:10.3102/0013189X032001014

Michel, R. (Ed.). (2007). Design research now. Basel, Switzerland: Birkhäuser Verlag AG

Oh, E., & Reeves, T. C. (2010). The implications of the differences between design research and instructional systems design for educational technology researchers and practitioners. Educational Media International, 47 (4), 263–275.

Reeves, T. C. (2006). Design research from a technology perspective. In J. van den Akker, K. Gravemeijer, S. McKenney, & N. Nieveen (Eds.), Educational design research (Vol. 1, pp. 52–66). London, England: Routledge.

Reigeluth, C. M., & Frick, T. W. (1999). Formative research: A methodology for creating and improving design theories. In C. Reigeluth (Ed.), Instructional-design theories and models. A new paradigm of instructional theory (Vol. 2) (pp. 633–651), Mahwah, NJ: Lawrence Erlbaum Associates.

Richey, R. C., & Nelson, W. A. (1996). Developmental research. In D. Jonassen (Ed.), Handbook of research for educational communications and technology (pp. 1213–1245), London, England: Macmillan.

Sandoval, W. A., & Bell, P. (2004). Design-based research methods for studying learning in context: Introduction. Educational Psychologist, 39 (4), 199–201.

Schneider, B. (2007). Design as practice, science and research. In R. Michel (Ed.), Design research now (pp. 207–218). Basel, Switzerland: Birkhäuser Verlag AG.

Shavelson, R. J., Phillips, D. C., Towne, L., & Feuer, M. J. (2003). On the science of education design studies. Educational Researcher, 32 (1), 25–28. doi:10.3102/0013189X032001025

Simon, H. A. (1969). The sciences of the artificial. Cambridge, MA: The MIT Press.

Tabak, I. (2004). Reconstructing context: Negotiating the tension between exogenous and endogenous educational design. Educational Psychologist, 39 (4), 225–233.

van den Akker, J. (1999). Principles and methods of development research. In J. van den Akker, R. M. Branch, K. Gustafson, N. Nieveen, & T. Plomp (Eds.), Design approaches and tools in education and training (pp. 1–14). Norwell, MA: Kluwer Academic Publishers.

van den Akker, J., & Plomp, T. (1993). Development research in curriculum: Propositions and experiences. Paper presented at the annual meeting of the American Educational Research Association, April 12–14, Atlanta, GA.

Walker, D.F., (1992). Methodological issues in curriculum research, In Jackson, P. (Ed.), Handbook of research on curriculum (pp. 98–118). New York, NY: Macmillan.

Walker, D. & Bresler, L. (1993). Development research: Definitions, methods, and criteria.  Paper presented at the annual meeting of the American Educational Research Association, April 12–16, Atlanta, GA.

Willemien, V. (2009). Design: One, but in different forms. Design Studies, 30 (3), 187–223. doi:10.1016/j.destud.2008.11.004

Further Video Resource

Video available at  http://bit.ly/WestDBRX

Foundations of Learning and Instructional Design Technology Copyright © 2018 by Kimberly Christensen and Richard E. West is licensed under a Creative Commons Attribution 4.0 International License , except where otherwise noted.

Share This Book

• About this Book
• Learning Theories
• Behaviorism
• Cognitivism
• Constructivism
• Socioculturalism
• Research Methods
• Case Studies

Design-Based Research

• Inferential Statistics
• Learning Analytics
• Psychometrics
• Design and Development
• Continuous Improvement Dashboards
• Design Layers
• Gamification
• Informal Learning
• Instructional Design Methods
• Personalized and Blended Learning
• Project-Based Learning
• Secure Web Application Development
• Socratic Seminar
• Visual Aesthetics
• List of Authors
• Translations

Choose a Sign-in Option

Tools and Settings

Questions and Tasks

Citation and Embed Code

In an educational setting, design-based research is a research approach that engages in iterative designs to develop knowledge that improves educational practices. This chapter will provide a brief overview of the origin, paradigms, outcomes, and processes of design-based research (DBR). In these sections we explain that (a) DBR originated because some researchers believed that traditional research methods failed to improve classroom practices, (b) DBR places researchers as agents of change and research subjects as collaborators, (c) DBR produces both new designs and theories, and (d) DBR consists of an iterative process of design and evaluation to develop knowledge.

Origin of DBR

DBR originated as researchers like Allan Collins (1990) and Ann Brown (1992) recognized that educational research often failed to improve classroom practices. They perceived that much of educational research was conducted in controlled, laboratory-like settings. They believed that this laboratory research was not as helpful as possible for practitioners.

Proponents of DBR claim that educational research is often detached from practice (The Design-Based Research Collective, 2002). There are at least two problems that arise from this detachment: (a) practitioners do not benefit from researchers’ work and (b) research results may be inaccurate, because they fail to account for context (The Design-Based Research Collective, 2002).

Practitioners do not benefit from researchers’ work if the research is detached from practice. Practitioners are able to benefit from research when they see how the research can inform and improve their designs and practices. Some practitioners believe that educational research is often too abstract or sterilized to be useful in real contexts (The Design-Based Research Collective, 2002).

Not only is lack of relevance a problem, but research results can also be inaccurate by failing to account for context. Findings and theories based on lab results may not accurately reflect what happens in real-world educational settings.

Conversely, a problem that arises from an overemphasis on practice is that while individual practices may improve, the general body of theory and knowledge does not increase. Scholars like Collins (1990) and Brown (1992) believed that the best way to conduct research would be to achieve the right balance between theory-building and practical impact.

Paradigms of DBR

Proponents of DBR believe that conducting research in context, rather than in a controlled laboratory setting, and iteratively designing interventions yields authentic and useful knowledge. Sasha Barab (2004) says that the goal of DBR is to “directly impact practice while advancing theory that will be of use to others” (p. 8). This implies “a pragmatic philosophical underpinning, one in which the value of a theory lies in its ability to produce changes in the world” (p. 6). The aims of DBR and the role of researchers and subjects are informed by this philosophical underpinning.

Aims of DBR

Traditional, experimental research is conducted by theorists focused on isolating variables to test and refine theory. DBR is conducted by designers focused on (a) understanding contexts, (b) designing effective systems, and (c) making meaningful changes for the subjects of their studies (Barab & Squire, 2004; Collins, 1990). Traditional methods of research generate refined understandings of how the world works, which may indirectly affect practice. In DBR there is an intentionality in the research process to both refine theory and practice (Collins et al., 2004).

Role of DBR Researcher

In DBR, researchers assume the roles of “curriculum designers, and implicitly, curriculum theorists” (Barab & Squire, 2004, p.2). As curriculum designers, DBR researchers come into their contexts as informed experts with the purpose of creating, “test[ing] and refin[ing] educational designs based on principles derived from prior research” (Collins et al., 2004, p. 15). These educational designs may include curricula, practices, software, or tangible objects beneficial to the learning process (Barab & Squire, 2004). As curriculum theorists, DBR researchers also come into their research contexts with the purpose to refine extant theories about learning (Brown, 1992).

This duality of roles for DBR researchers contributes to a greater sense of responsibility and accountability within the field. Traditional, experimental researchers isolate themselves from the subjects of their study (Barab & Squire, 2004). This separation is seen as a virtue, allowing researchers to make dispassionate observations as they test and refine their understandings of the world around them. In comparison, design-based researchers “bring agendas to their work,” see themselves as necessary agents of change and see themselves as accountable for the work they do (Barab & Squire, 2004, p. 2).

Role of DBR Subjects

Within DBR, research subjects are seen as key contributors and collaborators in the research process. Classic experimentalism views the subjects of research as things to be observed or experimented on, suggesting a unidirectional relationship between researcher and research subject. The role of the research subject is to be available and genuine so that the researcher can make meaningful observations and collect accurate data. In contrast, design-based researchers view the subjects of their research (e.g., students, teachers, schools) as “co-participants” (Barab & Squire, 2004, p. 3) and “co-investigators” (Collins, 1990, p. 4). Research subjects are seen as necessary in “helping to formulate the questions,” “making refinements in the designs,” “evaluating the effects of...the experiment,” and “reporting the results of the experiment to other teachers and researchers” (Collins, 1990, pp. 4-5). Research subjects are co-workers with the researcher in iteratively pushing the study forward.

Outcomes of DBR

DBR educational research develops knowledge through this collaborative, iterative research process. The knowledge developed by DBR can be separated into two categories: (a) tangible, practical outcomes and (b) intangible, theoretical outcomes.

Tangibles Outcomes

A major goal of design-based research is producing meaningful interventions and practices. Within educational research these interventions may “involve the development of technological tools [and] curricula” (Barab & Squire, 2004, p. 1). But more than just producing meaningful educational products for a specific context, DBR aims to produce meaningful, effective educational products that can be transferred and adapted (Barab & Squire, 2004). As expressed by Brown (1992), “an effective intervention should be able to migrate from our experimental classroom to average classrooms operated by and for average students and teachers” (p.143).

Intangible Outcomes

It is important to recognize that DBR is not only concerned with improving practice but also aims to advance theory and understanding (Collins et al., 2004). DBR’s emphasis on the importance of context enhances the knowledge claims of the research. “Researchers investigate cognition in context...with the broad goal of developing evidence-based claims derived from both laboratory-based and naturalistic investigations that result in knowledge about how people learn” (Barab & Squire, 2004, p.1). This new knowledge about learning then drives future research and practice.

Process of DBR

A hallmark of DBR is the iterative nature of its interventions. As each iteration progresses, researchers refine and rework the intervention drawing on a variety of research methods that best fit the context. This flexibility allows the end result to take precedence over the process. While each researcher may use different methods, McKenny and Reeves (2012) outlined three core processes of DBR: (a) analysis and exploration, (b) design and construction, and (c) evaluation and reflection. To put these ideas in context, we will refer to a recent DBR study completed by Siko and Barbour regarding the use of PowerPoint games in the classroom.

The Iterative Process of Design-Based Research

Analysis and Exploration

Analysis is a critical aspect of DBR and must be used throughout the entire process. At the start of a DBR project, it is critical to understand and define which problem will be addressed. In collaboration with practitioners, researchers seek to understand all aspects of a problem. Additionally, they “seek out and learn from how others have viewed and solved similar problems ” (McKenny & Reeves, 2012, p. 85). This analysis helps to provide an understanding of the context within which to execute an intervention.

Since theories cannot account for the variety of variables in a learning situation, exploration is needed to fill the gaps. DBR researchers can draw from a number of disciplines and methodologies as they execute an intervention. The decision of which methodologies to use should be driven by the research context and goals.

Siko and Barbour (2016) used the DBR process to address a gap they found in research regarding the effectiveness of having students create their own PowerPoint games to review for a test. In analyzing existing research, they found studies that stated teaching students to create their own PowerPoint games did not improve content retention. Siko and Barbour wanted to “determine if changes to the implementation protocol would lead to improved performance” (Siko & Barbour, 2016, p. 420). They chose to test their theory in three different phases and adapt the curriculum following each phase.

Design and Construction

Informed by the analysis and exploration, researchers design and construct interventions, which may be a specific technology or “less concrete aspects such as activity structures, institutions, scaffolds, and curricula” (Design-Based Research Collective, 2003, pp. 5–6). This process involves laying out a variety of options for a solution and then creating the idea with the most promise.

Within Siko and Barbour’s design, they planned to observe three phases of a control group and a test group. Each phase would use t-tests to compare two unit tests for each group. They worked with teachers to implement time for playing PowerPoint games as well as a discussion on what makes games successful. The first implementation was a control phase that replicated past research and established a baseline. Once they finished that phase, they began to evaluate.

Evaluation and Reflection

Researchers can evaluate their designs both before and after use. The cyclical process involves careful, constant evaluation for each iteration so that improvements can be made. While tests and quizzes are a standard way of evaluating educational progress, interviews and observations also play a key role, as they allow for better understanding of how teachers and students might see the learning situation.

Reflection allows the researcher to make connections between actions and results. Researchers must take the time to analyze what changes allowed them to have success or failure so that theory and practice at large can be benefited. Collins (1990) states:

It is important to analyze the reasons for failure and to take steps to fix them. It is critical to document the nature of the failures and the attempted revisions, as well as the overall results of the experiment, because this information informs the path to success. (pg. 5)

As researchers reflect on each change they made, they find what is most useful to the field at large, whether it be a failure or a success.

After evaluating results of the first phase, Siko and Barbour revisited the literature of instructional games. Based on that research, they first tried extending the length of time students spent creating the games. They also discovered that the students struggled to design effective test questions, so the researchers tried working with teachers to spend more time explaining how to ask good questions. As they explored these options, researchers were able to see unit test scores improve.

Reflection on how the study was conducted allowed the researchers to properly place their experiences within the context of existing research. They recognized that while they found positive impacts as a result of their intervention, there were a number of limitations with the study. This is an important realization for the research and allows readers to not misinterpret the scope of the findings.

This chapter has provided a brief overview of the origin, paradigms, outcomes, and processes of Design-Based Research (DBR). We explained that (a) DBR originated because some researchers believed that traditional research methods failed to improve classroom practices, (b) DBR places researchers as agents of change and research subjects as collaborators, (c) DBR produces both new designs and theories, and (d) DBR consists of an iterative process of design and evaluation to develop knowledge.

Barab, S., & Squire, K. (2004). Design-based research: putting a stake in the ground. Journal of the Learning Sciences, 13(1), 1–14.

Brown, A. L. (1992). Design experiments: theoretical and methodological challenges in creating complex interventions in classroom settings. Journal of the Learning Sciences, 2(2), 141–178.

Collins, A. (1990). Toward a design science of education (Report No. 1). Washington, DC: Center for Technology in Education.

Collins, A., Joseph, D., & Bielaczyc, K. (2004). Design research: Theoretical and methodological issues. Journal of the Learning Sciences, 13(1), 15–42.

Mckenney, S., & Reeves, T.C. (2012) Conducting Educational Design Research. New York, NY: Routledge.

Siko, J. P., & Barbour, M. K. (2016). Building a better mousetrap: how design-based research was used to improve homemade PowerPoint games. TechTrends, 60(5), 419–424.

The Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5–8.

This content is provided to you freely by EdTech Books.

Access it online or download it at https://edtechbooks.org/studentguide/design-based_research .

Design-Based Research Methods (DBR)

Summary: Design-Based Research is a lens or set of analytical techniques that balances the positivist and interpretivist paradigms and attempts to bridge theory and practice in education. A blend of empirical educational research with the theory-driven design of learning environments, DBR is an important methodology for understanding how, when, and why educational innovations work in practice; DBR methods aim to uncover the relationships between educational theory, designed artefact, and practice.

Originators: A. Brown [1] , A. Collins [2] , DBR Collective [3] , and others

Keywords: design experiments, iterative, interventionist, theory-building, theory-driven

In recent years, educators have been trying to narrow the chasm between research and practice. Part of the challenge is that research that is detached from practice “may not account for the influence of contexts, the emergent and complex nature of outcomes, and the incompleteness of knowledge about which factors are relevant for prediction” [3] .

According to Collins et al., Design-based Research (also known as design experiments) intends to address several needs and issues central to the study of learning [4] . These include the following:

• The need to address theoretical questions about the nature of learning in context
• The need for approaches to the study of learning phenomena in the real world situations rather than the laboratory
• The need to go beyond narrow measures of learning.
• The need to derive research findings from formative evaluation.

Characteristics of design-based research experiments include:

• addressing complex problems in real, authentic contexts in collaboration with practitioners
• applying integrating known and hypothetical design principles to render plausible solutions
• conducting rigorous and reflective inquiry to test and refine innovative learning environments
• intertwined goals of (1) designing learning environments and (2) developing theories of learning
• research and development through continuous cycles of design, enactment, analysis, and redesign
• research on designs that must lead to sharable theories that help communicate relevant implications to practitioners and other educational designers
• research must account for how designs function in authentic settings
• development of such accounts relies on methods that can document and connect processes of enactment to outcomes of interest [3] .

Design-based research vs. traditional evaluation

The following excerpt highlights the difference between the goals and contributions of design-based research methods can offer and traditional evaluation:

“In traditional evaluation, an intervention (e.g. a textbook, an instructional program, a policy) is measured against a set of standards. During formative evaluation, iterative cycles of development, implementation, and study allow the designer to gather information about how an intervention is or is not succeeding in ways that might lead to better design. Then the intervention is ‘frozen’, and the rigorous summative evaluation begins….Like formative evaluation, design-based research uses mixed methods to analyze an intervention’s outcomes and refine the intervention. Unlike evaluation research, design-based research views a successful innovation as a joint product of the designed intervention and the context. Hence, design-based research goes beyond perfecting a particular product. The intention of design-based research…is to inquire more broadly into the nature of learning in a complex system and to refine generative or predictive theories of learning. Models of successful innovation can be generated through such work — models, rather than particular artifacts or programs, are the goal” [3] .

For more information, see:

• Cobb, P., diSessa, A., Lehrer, R., Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1): 9-13.
• Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Learning Sciences, 2(2): 141-178.
• Collins, A. (1992). Towards a design science of education. In E. Scanlon & T. O’Shea (Eds.), New directions in educational technology (pp. 15-22). Berlin: Springer.
• Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1): 5-8.
• Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. The Journal of the Learning Sciences, 13(1).

Leave a Reply

You must be logged in to post a comment.

Related Posts

21st Century Skills (P21 and others)

Constructivism

Cognitivism

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Study Protocol

Using a design-based research approach to develop a technology-supported physical education course to increase the physical activity levels of university students: Study protocol paper

Roles Conceptualization, Funding acquisition, Project administration, Writing – original draft

* E-mail: [email protected]

Affiliations Sydney School of Education and Social Works, Faculty of Arts and Social Science, The University of Sydney, Sydney, New South Wales, Australia, Faculty of Sport and Health Education, Universitas Pendidikan Indonesia, Bandung, Jawa Barat, Indonesia

Roles Conceptualization, Supervision, Writing – review & editing

Affiliation Sydney School of Education and Social Works, Faculty of Arts and Social Science, The University of Sydney, Sydney, New South Wales, Australia

• Kuston Sultoni, 
• Louisa R. Peralta, 
• Wayne Cotton

• Published: December 1, 2022
• https://doi.org/10.1371/journal.pone.0269759
• Peer Review
• Reader Comments

Promoting physical activity (PA) for university students is essential as PA levels decrease during the transition from secondary to higher education. Providing technology-supported university courses targeting students’ PA levels may be a viable option to combat the problem. However, it is still unclear how and what technologies should be implemented in university courses to promote PA. This study aims to create a series of design principles for technology-supported physical education courses that aim to increase university students’ PA knowledge, motivation and levels.

The proposed methodology underpinning the research program is a seven-phase design-based research (DBR) approach, with the seven phases encompassed in four sequential studies. These four studies are a systematic review, a qualitative focus group study, a pilot study, and a randomised controlled trial (RCT) study. The protocol paper aims to detail the plan for conducting the four studies in a comprehensive and transparent manner, thus contributing to the methodological evidence base in this field.

Design principles generated from this project will contribute to the growing evidence focusing on effective design and implementation features. Future practitioners can also use these to develop physical education courses that aim to promote university students’ physical activity levels, knowledge, and motivation.

Trial registration

The RCT registry number: ACTRN12622000712707 , 18/05/2022.

Citation: Sultoni K, Peralta LR, Cotton W (2022) Using a design-based research approach to develop a technology-supported physical education course to increase the physical activity levels of university students: Study protocol paper. PLoS ONE 17(12): e0269759. https://doi.org/10.1371/journal.pone.0269759

Editor: Walid Kamal Abdelbasset, Prince Sattam Bin Abdulaziz University, College of Applied Medical Sciences, SAUDI ARABIA

Received: June 1, 2022; Accepted: October 11, 2022; Published: December 1, 2022

Copyright: © 2022 Sultoni et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: No datasets were generated or analysed during the current study. All relevant data from this study will be made available upon study completion.

Funding: The primary author (KS) is supported by the Indonesia Endowment Fund for Education Scholarship/ Lembaga Pengelola Dana Pendidikan Republik Indonesia (LPDP RI) under a doctoral degree scholarship (202001222015860). LPDP RI have no authority in study design; collection, management, analysis, and interpretation of data; writing of the report; and the decision to submit the report for publication.

Competing interests: The authors have declared that no competing interests exist.

List of abbreviations: PA, Physical Activity; DBR, Design-Based Research; PETE, Physical Education Teacher Education; ECTS, European Credit Transfer and Accumulation System; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses; PICO, Population, Intervention, Comparator, and Outcome; RCT, Randomized Controlled Trial; LMS, Learning Management System; UX, User Experience; UI, User Interface; IPAQ, International Physical Activity Questionnaire; BREQ-2, Behavioural Regulation in Exercise Questionnaire-2; MET, Metabolic Equivalent of Task

Interventions focusing on increasing physical activity levels among various age groups, from early childhood to the elderly, have been growing over the last five years [ 1 – 4 ]. These studies have suggested that targeting physical activity at different time points across the lifespan is essential, especially when there is a transition related to educational events [ 5 ] (e.g., the transition from preschool to primary school [ 6 – 8 ], from primary to high school [ 9 – 12 ], and from high school to college or university [ 13 , 14 ]). Yet, the last opportunity to intervene before adulthood is the transition into post-secondary studies. Therefore, this period is ultimately the most important for establishing lifelong actions such as personal, psychosocial, and movement behaviours for those who have not yet established them [ 15 – 17 ]. Physical activity tends to significantly decrease when graduating from high school and enrolling in universities [ 18 ] and among first-year university students [ 15 , 19 ], with 80% of university students not meeting physical activity recommendations during this transition [ 14 ]. These studies show that promoting physical activity among university students is essential.

Providing university courses that improve student’s physical activity levels may be a viable option, as universities can provide students with access to a range of sports facilities, highly educated facilitators, and appropriate technologies [ 20 ]. Furthermore, establishing and maintaining quality physical activity university courses has been a concern. As such, a guideline developed in the US and China [ 21 – 23 ] suggests that administration/support, assessment, instructional strategies, professionalism, learning environment, program staffing, and curriculum are essential facets that promote quality. Previous studies also suggest that providing strategies for administration and directors [ 24 , 25 ], modelling the development and support for course instructors [ 26 ], and utilising technology [ 27 – 29 ] can be viable strategies for increasing the quality of university courses that aim to improve the physical activity levels of university students.

There are various factors that influence physical activity behaviour in young adults [ 30 ]. Gaining knowledge of physical activity is considered one of the principle determining factors of physical activity in university-age students [ 31 ]. A cross-sectional study involving 258 adults in Hong Kong found that physical activity knowledge had a positive correlation with levels of physical activity, with this correlation strongest among the university student participants [ 32 ]. This finding is also supported by a Chinese cross-sectional study recruiting 9826 university students [ 33 ], with this study finding that knowledge of the physical activity guidelines was correlated with higher physical activity levels. Thus, physical activity knowledge should be considered as one of the learning outcomes of university courses that promote physical activity in a university setting.

Another important factor that is widely known to be associated with physical activity is motivation [ 34 , 35 ]. Exercise motivation plays an important role in long-term physical activity behaviour [ 36 ]. Systematic reviews examine relationships between motivation and physical activity [ 37 ] and examine the effects of physical activity interventions underpinned by motivational principles [ 38 ] show that motivation significantly increases physical activity levels [ 37 , 38 ]. Furthermore, a cross-sectional study involving 1079 participants aged 24±9 years showed that motivation for physical activity and exercise is associated with frequency, intensity, and duration of exercise [ 39 ]. This finding is supported by an observational study using a web-based survey involving 320 wearable activity monitor users, where motivational regulation was correlated with moderate to vigorous physical activity [ 40 ]. Hence, having motivational content and outcomes as part of a university course that promotes physical activity in university settings should also be considered.

Research focusing on technologies that promote university students’ physical activity levels is gaining more attention. The most common strategy utilised in previous studies has been internet websites [ 41 – 49 ]. Seven out of nine (78%) studies utilising internet websites have been successful in increasing physical activity levels [ 43 – 49 ]. Another common form of technology utilised to enhance university students’ physical activity levels are wearable devices. These have ranged from pedometers [ 50 – 53 ] to activity trackers (Misfit, Jawbone UP, Polar M400, Fitbit, and MyWellness Key) [ 54 – 58 ]. Two of four (50%) studies that utilised a pedometer successfully increased participants mean steps per day [ 53 , 59 ]. Two of five (40%) studies using activity trackers also increased student’s physical activity levels [ 55 , 58 ]. Social media, smartphone applications or mobile apps have also become prominent as a technology to enhance the physical activity levels of university students [ 60 – 66 ]. Three of seven (43%) studies using mobile phone apps have successfully increased students’ physical activity levels [ 62 , 63 , 65 ]. However, it is important to highlight that most of the studies (19 of 25 (76%)) utilising technology (internet website, wearable device, mobile phone apps) that aim to increase student’s physical activity levels in university settings were non-course-based interventions. That means only six of 25 (24%) studies were course-based interventions, with four of these six studies significantly increasing students’ physical activity levels. Nevertheless, it is still unclear what design principles and features of technology are the most effective to support the implementation of these course-based interventions to increase students’ physical activity knowledge, motivation and levels. There is also far too little attention paid to creating a set of design principles that can direct future improvement endeavours in this space [ 67 ]. Hence, research that focuses on development goals is needed to incorporate technology into courses at universities that aim to increase students’ physical activity knowledge, motivation and levels.

The aim of this design-based research project is to create a series of design principles, develop an intervention based on the design principles and measure the impact of the intervention [ 67 – 69 ]. The impact of the intervention will be measured through an increase in physical activity levels, knowledge, and motivation. The purpose of this protocol paper is to fully detail the plan for conducting a design-based research approach to develop a technology-supported physical activity course for increasing university students’ physical activity knowledge, motivation and levels in a comprehensive, transparent manner, contributing to the methodological evidence base in this field.

Aims and objectives

The primary purpose of this research project is to create a series of design principles which can guide the development of a technology-supported physical activity course to increase university student’s physical activity knowledge, motivation and levels.

Study design

The research methodology for the study was guided by the Design-Based Research model [ 67 ], which simplifies collaboration with practitioners by integrating known and hypothetical design principles with technological affordances to render plausible solutions to these problems. Design-based research was heralded as a practical research methodology that could effectively bridge the chasm between research and practice in formal education [ 70 ], including physical education [ 71 ]. Design-based research can also be used as an alternate model for enquiry in the field of educational technology to make future progress in improving teaching and learning through technology [ 72 ]. The original Reeves design-based research model contains four steps (see Fig 1 ). However, in this study, the four steps will be represented in seven phases to accommodate the cyclic nature of design-based research (see Fig 2 ). This adaptation has been made because the nature of design-based research is the flexibility of the process that still have some principles [ 73 ]. McKenney and Reeves [ 73 ] have highlighted that design-based research use: 1) scientific knowledge to ground design work; 2) produce scientific knowledge; 3) three main phases (analysis, design, and evaluation phase); and 4) development of both interventions in practise and reusable knowledge. Therefore, the seven phases in this study are: 1) needs analysis and creation of initial design principles from a systematic literature review and focus group discussions; 2) development of a prototype technology-supported physical education course based on initial design principles; 3) evaluation and testing of prototype technology-supported physical education course (pilot study); 4) revision of initial design principles; 5) modification of the prototype technology-supported physical education course based on revised design principles; 6) evaluation and testing of modified prototype technology-based physical education course (randomised controlled trial); and 7) the final design principles. These seven phases will be housed in four sequential studies. These are a: 1) systematic review; 2) qualitative focus group study; 3) pilot study, and 4) randomised controlled trial study. This project has obtained ethics approval from the University of Sydney Human Research Ethics Committee (Project No. 2021/071; Project No. 2021/935). Written informed consent was obtained from all participants before the study commenced.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0269759.g001

The four studies are identified in italics.

https://doi.org/10.1371/journal.pone.0269759.g002

Location and setting

This study will be conducted at the Universitas Pendidikan Indonesia (Indonesia University of Education) located in Bandung, West Java, Indonesia ( www.upi.edu ). The physical activity education course of interest is offered to first year and second-year undergraduate students as required (mandatory) learning or as an elective. To clarify, there is also university courses called Physical Education Teacher Education (PETE) programs that aim to educate preservice physical education teachers to prepare them to become beginning teachers in school settings. However, the physical activity education course (herein now called physical education course) in this study is unit of study with the main outcome is to promote active lifestyles or lifelong physical activity for all undergraduate students. Based on the higher education system in Indonesia, the workload used in the physical education course is a semester credit unit. In this system, one credit is equivalent to 48 hours/semester, 16 meetings, and 3 hours for each meeting consisting of scheduled lecture activities, structured, and independent assignments. The credit system is different from European Credit Transfer and Accumulation System (ECTS) which has a workload of 25 hours of study or 2.5 hours for 10 meetings per semester. When the credit system is changed to the ECTS, the students’ workloads for this course is 48:25 x 2 credits = 3.84 ECTS.

Phase 1. Needs analysis and creation of initial design principles

The purpose of Phase 1 is the creation of the initial design principles from the systematic literature review and needs analysis with key stakeholders. In addition, this phase will also identify appropriate technologies that will facilitate the implementation of a physical education course for increasing university students’ physical activity knowledge, motivation and levels.

Systematic review (study I)

Purpose . This systematic review aims to form initial design principles focusing on implementing a physical education course for university settings to improve university students’ physical activity knowledge, motivation and levels.

The first study in this project is systematic literature review aims to identify, critically appraise, and summarise the best available evidence regarding the effectiveness of technology-supported university courses for increasing the physical activity knowledge, motivation and levels of university students. This systematic review will follow the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [ 74 ]. The protocol of the study will be registered with PROSPERO (ID: CRD42020210327 ). The process will involve planning a review, searching, and selecting studies, data collection, risk of bias assessment, analysis, and interpreting results. The PICO (Population, Intervention, Comparator, and Outcome) formula is used in this study to limit the question. The population is a university student or college students, or students who enrol in higher institution. Intervention is technology-based physical education or course-based intervention using technology. Technology is limited to online delivery, learning management system, website, wearable device, mobile application, activity tracker, and blended learning using technology. Comparation is studies with a control group including RCT or Non-RCT or quasi-experimental study describing interventions using technology-supported university courses targeting physical activity among university students. The primary outcome will be physical activity levels. Eight electronic bibliographic databases will be sought including CINAHL, ERIC, MEDLINE, ProQuest, PsycINFO, Scopus, SPORTDiscus, and Web of Science from 1st January 2010 to 31st December 2020. 10 item quality assessment scale derived from Van Sluijs and colleagues [ 75 ] will be used to measure the quality of selected studies.

Qualitative focus group study (study 2)

Purpose . The purpose of this study is to confirm and add to the initial series of design principles to inform the design of Phase 2.

Setting access and recruitment . There will be three focus group discussions with key stakeholders. These stakeholders will include administrators and directors, course instructors/lecturers and students. The first discussion is with key stakeholders such as the curriculum development team, course coordinator and the Dean of the Faculty of Sport and Health Education. The discussion will be guided by a form to confirm that the content of the initial design principles is appropriate considering the learning outcomes and current technologies provided and supported by the university. The second focus group discussion is with lecturers who teach the physical education course. This discussion will be guided with lecturers being asked to answer questions focusing on the system and content of the course. The third focus group discussion will be with students who have previously enrolled in and completed the physical education course with questions focusing on the best technologies for their learning. The semi-structured focus group discussion guidelines are attached as supporting information file [see S1 Appendix ].

Participation in these focus groups is voluntary. All participants will be invited using email and will be given the Participant Information Statements. A signed Participant Consent Form will be needed to participate in the focus group discussion.

Sampling . There will be three focus group discussions. In the first focus group discussion, four key stakeholders or policy makers will be invited to participate: 1) a physical education course lecturer; 2) the course coordinator; 3) a member of university curriculum development team; and 4) the Dean of the Faculty of Sport and Health Education. The second focus group discussion will involve six randomly selected lecturers from 30 lecturers who currently implement the physical education course. The third focus group discussion will involve 12 undergraduate students who have enrolled in the physical education course previously. The size of sample in the focus group discussion 1, 2 and 3 will enable data saturation.

Data analysis . Qualitative data from the focus group discussions will be analysed using the thematic analysis [ 76 ]. The analysis will involve transcribing the data before coding individual comments into categories determined by the research question. Each category will then be sub-coded and investigated in more detail. This method will enable issues and themes in the data to emerge and from these issues and themes, conclusions will be able to be made to reinforce the initial design principles derived from the systematic review.

Phase 2. Development of a prototype technology-supported physical education course based on initial design principles

The purpose of Phase 2 is to develop a prototype based on the initial design principles from Phase 1. The design includes content and technical development. Content design will be reviewed by three physical education experts. Content development will produce a paper-based prototype of the technology that will show how the technology aligned with the initial design principles. While technical development will produce the design of a Learning Management System (LMS) and mobile application (App). Technical design includes User Experience (UX) represented by the flow chart of how LMS and App work and User Interface (UI) represented by a series of visual design on how LMS and App looks like. The next step is producing the LMS, and App based on technical development by IT developers to create a prototype for the technology-based physical education course.

Phase 3. Evaluation and testing of prototype technology-supported physical education course

The purpose of Phase 3 is to test and evaluate the prototype from Phase 2. In this phase, a pilot study will be conducted to implement and test the prototype from the previous phases to examine the feasibility and acceptability of a randomised controlled trial that will be conducted in Phase 6.

Pilot study (study 3).

Purpose . The pilot study aims to test the prototype with a small sample size to examine the feasibility and acceptability of the intervention.

Setting access and recruitment . The pilot study will include two classes of a semester physical education course. One of the classes will be the intervention group, and the other will be the control group. The intervention group will receive a course incorporated with the prototype for the technology-supported physical education course. The control group will receive the usual physical education course. Three outcomes will be measured pre- and post-intervention. This phase also includes developing the research instruments to determine validity and reliability.

There are two recruitment processes in this pilot study including the lecturer and students. In the lecturer recruitment step, the researcher will invite a lecturer who participated in the focus group discussion in Phase 1. The lecturer will be included in the pilot study if he/she meets the following inclusion criteria: 1) run two classes of the physical education course; 2) willing to take part in lecturer training before the physical education course class begins; and 3) willing to invite their students to take part in the pilot study.

The lecturer will invite their students to participate in the pilot study in the first week of the physical education course. Participation in this pilot study is voluntary, and all participants will be given Participant Information Statements and invited to sign a Participant Consent Form to participate. Participation or non-participation in this study will not affect students’ scores in physical education courses. The number of students in each class varies from 20–50 students. Students will be included in the pilot study if they meet the following inclusion criteria: 1) students enrolled in a physical education course with a lecturer who will participate in the pilot study; 2) voluntarily participate in the pilot study; and 3) owns an android smartphone.

Intervention . The intervention group will receive 16 weeks of the physical education course incorporated with the prototype of technology-supported physical education course implemented by their lecturer. The intervention and control group will have the same content (See Table 1 ); however, the intervention group will have access to the prototype of technology. The prototype will be developed in Phase 2 using the initial design principles generated from the focus group discussions and systematic review in Phase 1.

https://doi.org/10.1371/journal.pone.0269759.t001

Instruments . Data will be collected on student’s physical activity knowledge, motivation and levels, since the outcome of the course is changing student physical activity behaviour, knowledge and motivation to encourage long-life physical activity. Students’ physical activity level will be measured by using International Physical Activity Questionnaire (IPAQ) short form. Students’ physical activity knowledge will be measured by a questionnaire that will be developed in the pilot study based on literature [ 32 , 33 , 77 ]. A physical education expert will confirm content validity of the questionnaire and a reliability test of the instrument will be conducted in the pilot study. Students’ motivation for physical activity and exercise will be measured using The Behavioural Regulation in Exercise Questionnaire-2 (BREQ-2) [ 78 ]. The BREQ-2 consists of a 19-item questionnaire of 5 subscales (amotivation, external regulation, introjected regulation, identified regulation, and intrinsic regulation) to assess motivation to exercise. Construct validity studies showed that the BREQ-2 is valid for measuring exercise motivation among university students [ 79 ]. Validity and reliability of an Indonesian version of the BREQ-2 will be conducted in this pilot study.

Statistics and data analysis . Data will also be gathered from class observations and interviews with the lecturers and students. The class observation will use a semi-structured observation protocol based on ISO 9126 evaluation model for e-learning [ 80 ], focusing on functionality, reliability, efficiency, usability, maintainability, and portability of the prototypes that will be analysed for refining design principles.

Descriptive statistics will be presented (mean and standard deviation) for each group separately. Changes in physical activity level (MET: Metabolic Equivalent of Task), Physical activity knowledge and motivation from pre-test and post-test will be assessed using an ANCOVA. As this is a pilot study, the study will be underpowered; therefore, quantitative outcomes will be interpreted only as feasibility and acceptability measures.

Phase 4. Revise of initial design principles

The purpose of Phase 4 is to modify the initial design principles from Phase 1 based on data gathered from Phase 3 (pilot study, observation, measurement and feedback from lecturer and students). In this phase, the initial design principles will be revised to guide the prototype for technology-based physical education courses to determine the revised set of design principles.

Phase 5. Modification of prototype technology-supported physical education based on revised design principles

The purpose of Phase 5 is to modify the prototype based on Phase 4’s revised design principles. The revised design principles in the previous phase will be a foundation to redesign and build the technology-based physical education course. The next step is an expert review to find problems and recommendations for the technology. The final step in this phase is the refinement and modification of the technology prototype.

Phase 6. Evaluation and testing of modified prototype technology-supported physical education (randomised controlled trial)

In this phase, the refined and modified prototype will be tested and evaluated. The randomised controlled trial will be conducted to examine the effectiveness of a technology-supported physical education course on increasing university students’ physical activity levels, knowledge, and motivation.

Randomised controlled trial study (study 4).

This is a two-arm parallel, randomised controlled trial of a technology-supported physical education course intervention for students enrolled in an elective unit of study. The randomised controlled trial protocol is registered at the Australian New Zealand Clinical Trials Registry (ANZCTR). The RCT request number is ACTRN12622000712707, 18/05/2022. The RCT adheres to SPIRIT guidelines for reporting clinical trial study protocols [see S1 Table ]. The SPIRIT schedule can be seen in the Fig 3 .

https://doi.org/10.1371/journal.pone.0269759.g003

Purpose . The randomised controlled trial aims to test and evaluate the modified prototype with a larger sample size.

Hypothesis . A technology-supported physical education course intervention will increase university student’s physical activity levels, knowledge and motivation more effectively than a non-technology-supported physical education courses.

Setting access and participants . The randomised controlled trial will include six lecturers and six classes (20 to 50 students each class). The recruitment process and intervention in the randomised controlled trial will be the modified version of the intervention from the pilot study. The six lecturers who agree to participate will be randomly assigned to intervention or control. The lecturer will invite their students to participate in the RCT in the first week of the physical education course. The decision for students who the comparator groups are (the control group) is based on their enrolment teacher. Participation in this randomised controlled trial is voluntary, and all lecturers and students will be given Participant Information Statements and be invited to sign a Participant Consent Form to participate.

Sample size including power calculation . The sample size calculation is based on the difference in change in the primary outcome (physical activity) from pre- to post-intervention in both groups (intervention and control group). Based on previous studies of technology-based physical activity interventions among university students, mean effect sizes of around d = 0.5 are expected in analyses [ 47 , 81 ]. To detect such intervention effects in two-sided significance testing (α = .05) with a power of 80%, a sample size of 128 participants is required. Considering an expected study drop-out of about 20% and response rates of about 50%, 300 participants will be included in the study.

Number of participants

N = 6 physical education course lecturers

(a) n = 3 will be assigned to the intervention group,

(b) n = 3 will be assigned to the control group.

n = 300 students will be randomised 1:1 to either an intervention arm

(a) n = 150 a technology-supported physical education course (intervention group)

(b) n = 150 non-technology-supported physical education course (control group).

Teacher training . The lecturers in the intervention group will have teacher training for two days on how to deliver the technology-based physical education course prior to the intervention.

Intervention . The intervention group will receive 16 weeks of the physical education course incorporated with modified prototype from the pilot study in Phase 3.

Instrument . Data will be gathered using valid and reliable instruments developed from the pilot study and fidelity check and interviews to accommodate the final design principles.

Phase 7. Final design principles

This is the final phase of the study involving the entire data collected from Phases 1 to 6 to create a final series of design principles to aid future practitioners who will design, implement, and evaluate technology-supported physical education courses for university students.

The purpose of this study is to evaluate a technology-supported physical education course designed to increase the students’ physical activity levels, knowledge and motivation. The intervention uses a design-based research approach to determine the technologies and features that are most effective in supporting university students in attaining physical activity and motivational outcomes at one university in Indonesia. Determining the technologies and features will help support and facilitate university students physical activity and motivation, will foster scaling-up and sustainability of this course in this setting and perhaps in other university settings, which is an important facet of publishing protocol papers [ 82 , 83 ]. This manuscript will report on the systematic creation of a series of design principles using literature, focus group discussions with key stakeholders and modified through cycles of robust research testing. The systematic review study of this project has been completed and published [ 84 ]. Four design principles generated from the review will be confirmed and enhanced in the series of focus group discussions with key stakeholders (Phase 1) before the prototype is built from the initial design principles (Phase 2). The prototype will be tested and evaluated in the small sample pilot study to ensure prototype feasibility and validity (Phase 3). Then modifications will be performed based on the pilot study findings (Phase 4) with the modified prototype (Phase 5) tested and evaluated in a large randomised controlled trial study (Phase 6). The final design principles will then be created (Phase 7) to aid future research and practitioners who will design, implement, and evaluate university physical education courses. The design principles generated from this project will contribute to future practitioners designing, implementing, and evaluating technology-supported university physical education courses that aim to enhance university students’ physical activity knowledge, motivation and levels. However, this project has limitations. The most important limitation lies in the fact that this project will be conducted during the COVID-19 pandemic, where face-to-face activities in focus groups, as well as the pilot study and randomised controlled trial, will be restricted. In addition, the pandemic will also limit the use of objective measures of physical activity (i.e., accelerometer) as this instrument requires a face-to-face setting and application. Hence, only subjective measures of physical activity levels will be used in this study.

Supporting information

S1 appendix. semi-structured focus group discussion guidelines..

https://doi.org/10.1371/journal.pone.0269759.s001

S1 Table. SPIRIT 2013 checklist: Recommended items to address in a clinical trial protocol and related documents*.

https://doi.org/10.1371/journal.pone.0269759.s002

https://doi.org/10.1371/journal.pone.0269759.s003

• View Article
• Google Scholar
• PubMed/NCBI
• 21. National Association for Sport Physical Education N. Appropriate instructional practice guidelines for higher education physical activity programs. Author. Reston, VA; 2009.
• 31. Sallis JF, Owen N. Physical activity & behavioral medicine. Thousand Oaks, Calif.;: Sage; 1999.
• 50. Perkins J. Baseline comparison and the effects of education and pedometer use on physical activity in undergraduate students in psychology and personal health courses. Ann Arbor: Central Michigan University; 2006.
• 73. McKenney S, Reeves TC. Educational design research. Handbook of research on educational communications and technology: Springer; 2014. p. 131–40.

• Open access
• Published: 09 April 2024

Creating culturally-informed protocols for a stunting intervention using a situated values-based approach ( WeValue InSitu ): a double case study in Indonesia and Senegal

• Annabel J. Chapman 1 ,
• Chike C. Ebido 2 , 3 ,
• Rahel Neh Tening 2 ,
• Yanyan Huang 2 ,
• Ndèye Marème Sougou 4 ,
• Risatianti Kolopaking 5 , 6 ,
• Amadou H. Diallo 7 ,
• Rita Anggorowati 6 , 8 ,
• Fatou B. Dial 9 ,
• Jessica Massonnié 10 , 11 ,
• Mahsa Firoozmand 1 ,
• Cheikh El Hadji Abdoulaye Niang 9 &
• Marie K. Harder 1 , 2  

BMC Public Health volume  24 , Article number:  987 ( 2024 ) Cite this article

155 Accesses

Metrics details

International development work involves external partners bringing expertise, resources, and management for local interventions in LMICs, but there is often a gap in understandings of relevant local shared values. There is a widespread need to better design interventions which accommodate relevant elements of local culture, as emphasised by recent discussions in global health research regarding neo-colonialism. One recent innovation is the concept of producing ‘cultural protocols’ to precede and guide community engagement or intervention design, but without suggestions for generating them. This study explores and demonstrates the potential of an approach taken from another field, named WeValue InSitu , to generate local culturally-informed protocols. WeValue InSitu engages stakeholder groups in meaning-making processes which ‘crystallize’ their envelope of local shared values, making them communicable to outsiders.

Our research context is understanding and reducing child stunting, including developing interventions, carried out at the Senegal and Indonesia sites of the UKRI GCRF Action Against Stunting Hub. Each national research team involves eight health disciplines from micro-nutrition to epigenetics, and extensive collection of samples and questionnaires. Local culturally-informed protocols would be generally valuable to pre-inform engagement and intervention designs. Here we explore generating them by immediately following the group WeValue InSitu crystallization process with specialised focus group discussions exploring: what local life practices potentially have significant influence on the environments affecting child stunting, and which cultural elements do they highlight as relevant. The discussions will be framed by the shared values, and reveal linkages to them. In this study, stakeholder groups like fathers, mothers, teachers, market traders, administrators, farmers and health workers were recruited, totalling 83 participants across 20 groups. Themes found relevant for a culturally-informed protocol for locally-acceptable food interventions included: specific gender roles; social hierarchies; health service access challenges; traditional beliefs around malnutrition; and attitudes to accepting outside help. The concept of a grounded culturally-informed protocol, and the use of WeValue InSitu to generate it, has thus been demonstrated here. Future work to scope out the advantages and limitations compared to deductive culture studies, and to using other formative research methods would now be useful.

Peer Review reports

Although progress has been made towards the SDG of ‘Zero Hunger by 2025’, the global rates of malnutrition and stunting are still high [ 1 ]. Over the past 20 years, researchers have implemented interventions to reduce undernutrition, specifically focussing on the first 1000 days of life, from conception to 24 months [ 2 ]. However, due to both differing determinants between countries [ 3 , 4 ] as well as varying contextual factors, it is clear that no single fixed approach or combination of approaches can be relied on when implementing stunting interventions [ 5 , 6 , 7 ]. Furthermore, when external researchers design interventions for local areas in Low- and Middle-Income Countries (LMICs) they can often overlook relevant local cultural factors that consequently act as barriers to intervention uptake and reduce their effectiveness, such as geographical factors and the levels of migration in certain populations [ 8 , 9 ], or social norms or perceptions relating to accepting outside help, and power dynamics related to gender [ 10 , 11 , 12 ]. The inclusion of cultural level factors in behaviour change interventions has been proposed as a requirement for effective interventions [ 13 ]. However, despite the breadth of literature highlighting the negative impacts from failing to do this, the lack of integration or even regard of local culture remains a persistent problem in Global Health Research [ 14 ], possibly hindering progress towards the SDGs. Thus, there is a need for approaches to integrate local cultural elements into intervention design.

This lack of understanding of relevant local culture, social norms and shared values also has ethical implications. The field of Global Health Ethics was predominantly developed in the Global North, in High Income Countries (HICs), embedding values common in those countries such as the prominence of individual autonomy [ 15 , 16 ]. Researchers from HICs carrying out research in LMICs may wrongly assume that values held in the Global North are universal [ 14 ] and disregard some local values, such as those related to family and collective decision making, which are core to many communities in LMICs. It is therefore important for outside researchers to have an understanding of relevant local values, culture and social norms before conducting research in LMICs so as not to impose values that do not align with local culture and inadvertently cause harm or offence [ 16 , 17 ]. The importance of this is compounded by the colonial history that is often present in relationships between research communities in HICs and LMICs, and the fact that the majority of the funding and leading institutions are still located in the Global North [ 18 , 19 ]. Thus, conscious steps must be taken to avoid neo-colonialism in Global Health Research [ 20 ]. From a health-equity perspective, it is essential to ensure that those in vulnerable communities are not hindered from involvement in interventions to improve nutrition. Encouraging uptake by such communities could be provided if salient local shared values, norms and culture were taken into account [ 21 ].

In a recent paper, Memon et al., (2021) highlight the usefulness of first creating a cultural protocol that can precede and guide subsequent stages of community engagement or intervention design to ensure that salient local values are known to external researchers coming into the community [ 16 ]. We adopt the use of the concept of a cultural protocol, referring to locally-generated guidance about key values, norms, behaviours and customs relevant to working with the local community. However, we prefer the term, ‘culturally-informed protocol’ since this relates to only cultural elements deemed salient by the researchers, and locally, rather than any comprehensive notion of culture, nor extending beyond the research context.

Memon et al. (2021), point out links between the creation of such a protocol and existing codes of practice that have already been created for some cultures such as the Te Ara Tika, a Guideline for Māori Research Ethics [ 22 ]. Currently, research and interventions in Global Health can be informed by a stage of formative research involving one-to-one interviews, focus groups or direct observations, which can sometimes be ethnographic in nature such as within Focussed Ethnographic Studies or Rapid Assessment Procedures [ 23 , 24 , 25 ]. Although these methods can be effective to inform intervention designs, they have disadvantages like: can take long periods to complete [ 26 ], can be resource intensive [ 26 ] and can lack cultural acceptability [ 27 ]. These limitations may account for the frequent neglect of their use generally, highlighted by Aubel and Chibanda (2022) [ 14 ]. Additionally, none of these methods work towards making explicit local values, or towards the creation of a culturally-informed protocol. In brief, the literature suggests a need to develop alternative methods of Formative Research for understanding locally relevant cultural elements, that are less time-consuming and can generate data that is more easily translatable to intervention design. In addition, these approaches must be applicable in different cultures. Additionally, the protocols produced must be actionable and practical not only for guiding interactions between research teams but also for guiding the initial stages of intervention design.

The work presented here aims to address several of these needs. It includes an exploration of the usefulness of the WeValue InSitu ( WVIS ) approach because that has previously been shown, in environmental management domains, to offer a way to gather in-depth values-based perspectives from a target population [ 28 , 29 ] It was first created through action research, and co-designed to enable civil society organisations to better understand and measure the values-based aspects of their work [ 30 ]. The core WeValue InSitu process (detailed in Table 1 ) involves the crystallization of shared values, with a facilitator guiding a group of participants with shared experiences, through cycles of tacit meaning-making (using a stage of photo-elicitation and triggering) [ 31 ], until they can articulate more explicitly their shared values, in concise and precise statements. These statements are then linked together in a framework by the participants. In an example case in Nigeria, the results of the WVIS approach hinted at the creation of a culturally-informed protocol through an analysis of the shared values frameworks to find cultural themes for the creation of an indicator tool that was used to evaluate several development scenarios based on their social acceptability [ 29 ].

Furthermore, it has been found that if a group of WVIS participants take part in a specialised focus group discussion (FGD), named Perspectives EXploration (PEX:FGD) immediately afterward the main workshop, then they easily and articulately express their perspectives on the topics raised for discussion - and with allusions to the shared values they had crystallised just prior. In an example from Shanghai, the PEX:FGDs focussed on eliciting perspectives on climate change, which were shown to be closely linked with the cultural themes existing within the shared values frameworks produced immediately prior [ 32 ]. In that case, the PEX:FGDs allowed the cultural themes generated during the main WVIS workshop to be linked more closely to the research question. Those results suggested that the WVIS plus PEX:FGD approach could be used to create a specialised culturally-informed protocol for improved intervention design.

In the study presented here, the WVIS approach was explored for the purpose of creating culturally-informed protocols to inform the planning of interventions within two localities of the UKRI GCRF Action Against Stunting Hub [ 33 ]. The work was carried out in two parts. Firstly, the WVIS main workshop was used to elicit cultural themes within the target communities, indicating key elements to consider to ensure ethical engagement. Secondly, the PEX focus group discussions focussed on life practices related to stunting which we explored for the purpose of tailoring the culturally-informed protocols to the specific purpose of improving the design of an example intervention. The Action Against Stunting Hub works across three sites where stunting is highly prevalent but via different determinants: East Lombok in Indonesia (estimated 36% of under-fives stunted), Kaffrine in Senegal (estimated 16% of under-fives stunted) and Hyderabad in India (estimated 48% of under-fives stunted) [ 34 ]. We propose that, the information about local shared values in a given site could be used to inform the design of several interventions, but for our specific exploration the focus here is a proposed ‘egg intervention’, in which pregnant women would be provided with an egg three times per week as supplement to their diet. This study proposes that identifying shared values within a community, alongside information about local life practices, provides critical cultural information on the potential acceptability and uptake of this intervention which can be used to generate culturally-informed protocols consisting of recommendations for improved intervention design.

In this paper we aim to explore the use of the WVIS approach to create culturally-informed protocols to guide engagement and inform the design of localised egg interventions to alleviate stunting in East Lombok, Indonesia and Kaffrine, Senegal. We do this by analysing data about local shared values that are crystallized using the WeValue InSitu ( WVIS ) process to provide clear articulation of local values, followed by an analysis of life practices discussed during PEX:FGD to tailor the culturally-informed protocols for the specific intervention design.

Study setting

This research was exploratory rather than explanatory in nature. The emphasis was on demonstrating the usefulness of the WeValue InSitu ( WVIS ) approach to develop culturally-informed protocols of practical use in intervention design, in different cultural sites. This study was set within a broader shared-values workstream within the UKRI GCRF Action Against Stunting Hub project [ 33 ]. The Hub project, which was co-designed and co-researched by researchers from UK, Indonesia, Senegal and India, involves cohorts of 500 women and their babies in each site through pregnancy to 24 months old, using cross-disciplinary studies across gut health, nutrition, food systems, micro-nutrition, home environment, WASH, epigenetics and child development to develop a typology of stunting. Alongside these health studies are studies of the shared values of the communities, obtained via the WVIS approach described here, to understand the cultural contexts of that diverse health data. In this study the data from East Lombok, Indonesia and Kaffrine, Senegal were used: India’s data were not yet ready, and these two countries were deemed sufficient for this exploratory investigation.

The WVIS approach

The WVIS approach is a grounded scaffolding process which facilitates groups of people to make explicit their shared values in their own vocabulary and within their own frames (details in Fig. 1 and activities in Table 1 ). The first stage of the WVIS is Contextualisation, whereby the group identifies themselves and set the context of their shared experiences, for example, as ‘mothers in East Lombok, Indonesia’. Subsequently, there is a stage of Photo Elicitation, in which the group are first asked to consider what is important, meaningful or worthwhile to them about their context (e.g., ‘being mothers in East Lombok, Indonesia’) and then asked to choose photos from a localised set that they can use as props to help describe their answer to the group [ 29 ]. After this, a localised Trigger List is used. This Trigger List consists of 109 values statements that act as prompts for the group. Examples of these values statements are included below but all the statements begin with “it is important to me/us that…”. The group are asked to choose which statements within the trigger list resonate with them, and those are taken forward for group intersubjective discussion. After a topic of their shared values has been explored, the group begin to articulate and write down their own unique statements of them. These also all begin with “It is important to me/us that…”. After discussing all pressing topics, the group links the written statements on the table into a unique Framework, and one member provides a narrative to communicate it to ‘outsiders’. The WVIS provides a lens of each group’s local shared values, and it is through this lens that they view the topics in the focus group discussions which immediately follow, termed Perspectives EXplorations (PEX:FGDs).

Schematic of the macro-level activities carried out during the WeValue InSitu ( WVIS ) main workshop session

This results in very grounded perspectives being offered, of a different nature to those obtained in questionnaires or using external frameworks [ 31 ]. The specific PEX:FGD topics are chosen as pertinent to stunting contextual issues, including eating habits, food systems and environments, early educational environments, and perceptions of stunting. The local researchers ensured that all topics were handled sensitively, with none that could cause distress to the participants. The data for this study were collected over 2 weeks within December 2019–January 2020 in workshops in East Lombok, Indonesia, and 2 weeks within December 2020 in Kaffrine, Senegal.

The PEX:FGDs were kept open-ended so that participants could dictate the direction of the discussion, which allowed for topics that may not have been pre-considered by the facilitators to arise. Sessions were facilitated by local indigenous researchers, guided in process by researchers more experienced in the approach, and were carried out in the local languages, Bahasa in East Lombok, Indonesia and French or Wolof in Kaffrine, Senegal.

Development of localised WVIS materials

Important to the WVIS approach is the development of localised materials (Table 1 ). The main trigger list has been found applicable in globalised places where English is the first language, but otherwise the trigger lists are locally generated in the local language, incorporating local vocabulary and ways of thinking. To generate these, 5–8 specific interviews are taken with local community members, by indigenous university researchers, eliciting local phrases and ways of thinking. This is a necessary step because shared tacit values cannot be easily accessed without using local language. Examples of localised Trigger Statements produced this way are given below: (they all start with: “It is important to me/us that…”):

…there is solidarity and mutual aid between the people

…I can still be in communication with my children, even if far away

…husbands are responsible for the care of their wives and family

…the town council fulfils its responsibility to meet our needs

…people are not afraid of hard, and even manual work

Study participants

The group participants targeted for recruitment, were selected by local country Hub co-researchers to meet two sets of requirements. For suitability for the WVIS approach they should be between 3 and 12 in number; belong to naturally existing groups that have some history of shared experiences; are over 18 years old; do not include members holding significantly more power than others; and speak the same native language. For suitability in the PEX:FGD to offer life practices with relevance to the research topic of stunting, the groups were chosen to represent stakeholders with connections to the food or learning environment of children (which the Action Against Stunting Hub refer to as the Whole Child approach) [ 33 ]. The university researchers specialising in shared values from the UK, and Senegal and Indonesia respectively, discussed together which stakeholder groups might be appropriate to recruit. The local researchers made the final decisions. Each group was taken through both a WVIS workshop and the immediately-subsequent PEX:FGD.

Data collection and analysis

Standard data output from the WeValue session includes i) the jointly-negotiated bespoke Statements of shared values, linked together in their unique Framework, and ii) an oral recording of a descriptive Narrative of it, given by the group. These were digitized to produce a single presentation for each group as in Fig. 2 . It represents the synthesised culmination of the crystallisation process: a portrait of what was ‘important’ to each stakeholder group. Separately, statements from the group about the authenticity/ownership of the statements are collected.

An illustrative example of one digitized Shared Values Framework and accompanying Narrative from a teacher’s group in East Lombok, Indonesia. The “…” refers to each statement being preceded by “It is important to us that…”

When these Frameworks of ‘Statements of Shared Values’ are viewed across all the groups from one locality (Locality Shared Values Statements), they provide portraits of ‘what is important’ to people living there, often in intimate detail and language. They can be used to communicate to ‘outsiders’ what the general cultural shared values are. In this work the researchers thematically coded them using Charmaz constructionist grounded theory coding [ 35 ] to find broad Major Cultural Themes within each separate locality.

The second area of data collection was in the post- WVIS event: the PEX:FGD for each group. A translator/interpreter provided a running commentary during these discussions, which was audio recorded and then transcribed. The specific topics raised for each group to discuss varied depending on their local expertise. This required completely separate workstreams of coding of the dataset with respect to each topic. This was carried out independently by two researchers: one from UK (using NVivo software (Release 1.3.1)) and one from the local country, who resolved any small differences. All the transcripts were then collated and inductively, interpretively analysed to draw out insights that should be relayed back to the Action Against Stunting Hub teams as contextual material.

The extracts of discussion which were identified as relevant within a particular Hub theme (e.g. hygiene) were then meta-ethnographically synthesised [ 36 ] into ‘Hub Theme Statements’ on each topic, which became the core data for later communication and interrogation by other researchers within the Action Against Stunting Hub. These statements are interpretations of participants’ intended meanings, and links from each of them to data quotes were maintained, enabling future interpretations to refer to them for consistency checks between received and intended meaning.

In this investigation, those Hub Theme Statements (derived from PEX:FGD transcripts) were then deductively coded with respect to any topics with potential implications of the egg intervention. Literature regarding barriers and facilitators to nutrition interventions indicated the following topics could be relevant: attitudes to accepting help; community interactions; cooking and eating habits; traditional beliefs about malnutrition; sharing; social hierarchies [ 12 , 37 , 38 ] to which we added anything related to pregnancy or eggs. This analysis produced our Egg Intervention Themes from the data.

The Major Cultural Themes and Egg Intervention Themes were then used to create a set of culture-based recommendations and intervention specific recommendations respectively for each locality. These recommendations were then combined to form specialized culturally-informed protocols for the egg intervention in each locality: East Lombok, Indonesia and Kaffrine, Senegal. The process is displayed schematically in Fig.  3 .

Schematic representation of the method of production of the culturally-informed protocol for each locality

The preparation of the localised WVIS materials at each site took 6 hours of interview field work, and 40 person hours for analysis. The 10 workshops and data summaries were concluded within 10 workdays by two people (80 person hours). The analysis of the PEX:FGD data took a further 80 person hours. Thus, the total research time was approximately 200 person hours.

The stakeholder group types are summarised in Table 2 . The data is presented in three parts. Firstly, the Major Cultural Themes found in East Lombok, Indonesia and in Kaffrine, Senegal are described – the ones most heavily emphasised by participants. Then, the Egg Intervention Themes and finally, the combined set of Recommendations to comprise a culturally-informed protocol for intervention design for each location. Quotations are labelled INDO or SEN for East Lombok, Indonesia and Kaffrine, Senegal, respectively.

Major cultural themes from frameworks and narratives

These were derived from the Locality Shared Values Statements produced in the WVIS .

East Lombok, Indonesia

Religious values.

Islamic values were crucially important for participants from East Lombok, Indonesia and to their way of life. Through living by the Quran, participating in Islamic community practices, and teaching Islamic values to their children, participants felt they develop their spirituality and guarantee a better afterlife for themselves and their children. Participants stated the Quran tells them to breastfeed their children for 2 years, so they do. Despite no explicit religious official curriculum in Kindergarten, the teachers stated that it was important to incorporate religious teaching.

“East Lombok people always uphold the religious values of all aspects of social life.”

“It is important for me to still teach religious values even though they are not clearly stated in the curriculum.” – Workshop 1 INDO (teachers).

“In Quran for instance, we are told to breastfeed our kids for 2 years. We can even learn about that ” – Workshop 3 INDO (mothers).

Related to this was the importance of teaching manners to children and preventing them from saying harsh words. Teachers stated that it was important to create a happy environment for the children and to ensure that they are polite and well-behaved. Similarly, mothers emphasised the need to teach their children good religious values to ensure they will be polite and helpful to their elders.

“Children don’t speak harsh words.”

“My children can help me like what I did to my parents”.

– Workshop 8 INDO (mothers).

Togetherness within families and the community

The Locality Shared Values Frameworks stressed the importance of togetherness, both within family and community. Comments mentioned it being important that people rely heavily on their family and come together in times of need to support each other and provide motivation. This was also important more broadly, in that people in society should support each other, and that children grow up to contribute to society. This was also reflected in comments around roles within the family. Despite women being primary care givers, and men working to finance the family, participants stated that they follow a process of consultation to make decisions, and when facing hardships.

“that we have the sense of kinship throughout our society”.

“We have togetherness as mothers”.

“For the family side, whatever happens we need to be able to be united as a whole family. We need to have the [sense of] forgiveness for the sake of the children” – Workshop 2 INDO (mothers).

Attitudes about extra-marital pregnancy

In East Lombok, Indonesia, it was essential to both mothers and fathers that pregnancy happened within a marriage, this was to ensure that the honour of the family was upheld and that the lineage of the child was clear. The potential danger to health that early pregnancies can cause was also acknowledged.

“If they don’t listen to parents’ advice, there will be the possibility of pre-marital pregnancy happening, which will affect the family [so much].

The affect is going to be ruining the good name, honour and family dignity. When the children [are] born outside [of] marriage, she or he will have many difficulties like getting a birth certificate [and] having a hard time when registering to school or family” - Workshop 4 INDO (mothers).

“ To make sure that our children avoid getting married at a very young age and moreover [avoid] having free sex so that they will not get pregnant before the marriage” - Workshop 9 INDO (fathers).

Kaffrine, Senegal

The Major Cultural Themes which emerged from the Kaffrine data are described below. As these are grounded themes, they are different than those seen in East Lombok, Indonesia.

Access to healthcare

A recurring theme amongst the groups in Kaffrine were aspirations of affordable and easy-to-access healthcare. Community health workers stated the importance of encouraging women to give birth in hospitals and spoke of the importance of preventing early pregnancy which result from early marriages. Giving birth in hospitals was also a concern for Public Office Administrators who highlighted that this leads to subsequent issues with registering children for school. Mothers and fathers stated the importance of being able to afford health insurance and access healthcare so that they could take care of themselves.

“That the women give birth in the hospital” – Workshop 11 SEN (CHWS).

“To have affordable health insurance ” – Workshop 10 SEN (mothers).

“To have access to health care ” – Workshop 3 SEN (fathers).

“It is important that women give birth in the hospital in order to be able to have a certificate that allows us to establish the civil status” – Workshop 9 SEN (administrators).

Additionally, Community health workers spoke of their aspiration to have enough supplements to provide to their community so as to avoid frustration at the lack of supply, and mothers spoke of their desire to be provided with supplements.

“To have dietary supplements in large quantities to give them to all those who need them, so as not to create frustration” – Workshop 11 SEN (CHWS).

Another aspect of access to healthcare, was mistrust between fathers and community health workers. Community health workers explained that sometimes men can blame them when things go wrong in a pregnancy or consider their ideas to be too progressive. Thus, to these community health workers the quality of endurance was very important.

“Endurance (Sometimes men can accuse us of influencing their wives when they have difficulties in conceiving)” – Workshop 5 SEN (CHWs).

Another recurring theme was the importance of having secure employment and a means to support themselves; that there were also jobs available for young people, and that women had opportunities to make money to help support the family. This included preventing early marriages so girls could stay in school. Having jobs was stated as essential for survival and important to enable being useful to the community and society.

“To have more means of survival (subsistence) to be able to feed our families”.

“To have a regular and permanent job”.

“We assure a good training and education for our children so that they will become useful to us and the community”.

“ Our women should have access to activities that will support us and lessen our burden” – Workshop 3 SEN (fathers).

It was considered very important to have a religious education and respect for religious elders. Moreover, living by, and teaching, religious values such as being hard working, humble and offering mutual aid to others, was significant for people in Kaffrine.

“Have an education in the Islamic Culture (Education that aligns with the culture of Islam)”.

“Respect toward religious leaders” – Workshop 3 SEN (fathers).

“ To organize religious discussions to develop our knowledge about Islam ” - Workshop 10 SEN (mothers).

“ Have belief and be prayerful and give good counselling to people ” - Workshop 4 SEN (grandmothers).

Egg intervention themes from each country from perspectives EXplorations focus group discussion data

Below are results of analyses of comments made during the PEX:FGDs in East Lombok, Indonesia and Kaffrine, Senegal. The following codes were used deductively: attitudes to accepting outside help, traditional gender roles, food sharing, traditional beliefs, social hierarchies and understanding of stunting and Other. These topics were spoken about during open discussion and were not the subject of direct questions. For example, topics relating to traditional gender roles came up in East Lombok, during conversations around the daily routine. Thus, in order to more accurately reflect the intended meaning of the participants, these were labelled food practices, under the “Other” theme. If any of the themes were not present in the discussion, they are not shown below.

Attitudes to accepting outside help

Few mentions were made that focussed on participants attitudes to accepting outside help, but participants were sure that they would not make changes to their menus based on the advice of outside experts. Additionally, teachers mentioned that they are used to accepting help from local organisations that could to help them to identify under-developed children.

“ We don’t believe that [the outsiders are] going to change our eating habits or our various menus ” – Workshop 3 INDO (Mothers).

Traditional gender roles

In East Lombok, mothers spoke about how their husbands go to work and then provide them with daily money to buy the food for the day. However, this was discussed in relation to why food is bought daily and is thus discussed below in the topics Other – Food practices.

Food sharing

In East Lombok, Indonesia, in times when they have extra food, they share it with neighbours, in the hope that when they face times of hardship, their neighbours will share with them. Within the household, they mentioned sharing food from their plate with infants and encouraging children to share. Some mothers mentioned the importance of weekly meetings with other mothers to share food and sharing food during celebrations.

“ Sometimes we share our food with our family. So, when we cook extra food, we will probably send over the food to our neighbour, to our families. So, sometimes, with the hope that when we don’t have anything to eat, our neighbour will pay for it and will [share with] us.” – Workshop 3 INDO (Mothers).

“Even they serve food for the kids who come along to the house. So, they teach the kids to share with their friends. They provide some food. So, whenever they play [at their] house, they will [eat] the same.” – Workshop 2 INDO (Mothers).

Understanding of stunting

The teachers in East Lombok were aware of child stunting through Children’s Development Cards provided by local healthcare organizations. They stated that they recognise children with nutrition problems as having no patience period, no expression, no energy for activities and less desire to socialise and play with other children. The teachers said that stunted children do not develop the same as other children and are not as independent as children who are the proper height and weight for their development. They also stated that they recognise stunted children by their posture, pale faces and bloated stomachs. They explained how they usually use the same teaching methods for stunting children, but will sometimes allow them to do some activities, like singing, later, once the other children are leaving.

“ They have no patience period, don’t have any energy to do any of the activities. No expression, only sitting down and not mingling around with the kids. They are different way to learn. They are much slower than the other kids .” – Workshop 1 INDO (teachers).

“ When they are passive in singing, they will do it later when everyone else is leaving, they just do it [by] themselves ” – Workshop 1 INDO (teachers).

Specific views on eggs

In East Lombok, Indonesia, there were no superstitions or traditional beliefs around the consumption of eggs. When asked specifically on their views of eggs, and if they would like to be provided with eggs, women in East Lombok said that they would be happy to accept eggs. They also mentioned that eggs were a food they commonly eat, feed to children and use for convenience. Eggs were considered healthy and were common in their house.

“ We choose eggs instead. If we don’t have time, we just probably do some omelettes or sunny side up. So, it happens, actually when we get up late, we don’t have much time to be able to escort our kids to the school, then we fry the eggs or cook the instant noodles. And it happens to all mothers. So, if my kids are being cranky, that’s what happens, I’m not going to cook proper meals so, probably just eggs and instant noodles.” – Workshop 3 INDO (Mothers).

Other important topics – food practices

Some detailed themes about food practices were heard in East Lombok, Indonesia. The women were responsible for buying and preparing the food, which they purchased daily mainly due to the cost (their husbands were paid daily and so provided them with a daily allowance) and lack of storage facilities. They also bought from mobile vendors who came to the street, because they could buy very small amounts and get occasional credit. The mother decided the menu for the family and cooked once per day in the morning: the family then took from this dish throughout the day. Mothers always washed their fruits and vegetables and tried to include protein in their meals when funds allowed: either meat, eggs, tofu or tempeh.

“ One meal a day. They [the mothers] cook one time and they [the children] can eat it all day long. Yes, they can take it all day long. They find that they like [to take the food], because they tend to feel hungry.” – Workshop 6 INDO (Mothers).

“ They shop every day because they don’t have any storage in their house and the other factor is because the husband has a daily wage. They don’t have monthly wage. In the morning, the husband gives the ladies the money and the ladies go to the shop for the food. ” – Workshop 4 INDO (Mothers).

In Kaffrine, the following themes emerged relating to an egg intervention: they were different in content and emphasis to Lombok and contained uniquely local cultural emphases.

Mothers were welcoming of eggs as a supplement to improve their health during pregnancy and acknowledged the importance of good nutrition during pregnancy. However, they also mentioned that their husbands can sometimes be resistant to accepting outside help and provided an example of a vaccination programme in which fathers were hesitant to participate. However, participants stated that the Government should be the source of assistance to them (but currently was not perceived to be so).

“But if these eggs are brought by external bodies, we will hesitate to take it. For example, concerning vaccination some fathers hesitate to vaccinate their children even if they are locals who are doing it. So, educating the fathers to accept this is really a challenge” – Workshop 11 SEN (CHWs).

Some traditional gender roles were found to be strong. The participants emphasised that men are considered the head of the household, as expected in Islam, with the mother as primary caregiver for children. This is reflected in the comments from participants regarding the importance of Islam and living their religious values. The men thus made the family decisions and would need to be informed and agree to any family participation in any intervention – regardless of the education level of the mother. The paternal grandmother also played a very important role in the family and may also make decisions for the family in the place of the father. Community Health Workers emphasised that educating paternal grandmothers was essential to improve access to healthcare for women.

“There are people who are not flexible with their wives and need to be informed. Sometimes the mother-in-law can decide the place of the husband. But still, the husband’s [permission] is still necessary.” – Workshop 1 SEN (CHWs).

“[We recommend] communication with mothers-in-law and the community. Raise awareness through information, emphasizing the well-being of women and children.” – Workshop 1 SEN (CHWs).

“The [grand]mothers take care of the children so that the daughters in-law will take care of them in return So it’s very bad for a daughter in law not to take care of her mother in-law. Society does not like people who distance themselves from children.” – Workshop 4 SEN (grandmothers).

Social hierarchies

In addition to hierarchies relating to gender/position in the family such as grandmothers have decision making power, there was some mention of social hierarchies in Kaffrine, Senegal. For example, during times of food stress it was said that political groups distribute food and elected officials who choose the neighbourhoods in which the food will be distributed. Neighbourhood leaders then decide to whom the food is distributed, meaning there is a feeling that some people are being left out.

“ It’s political groups that come to distribute food or for political purposes…organizations that often come to distribute food aid, but in general it is always subject to a selection on the part of elected officials, in particular the neighbourhood leaders, who select the people they like and who leave the others ” – Workshop 11 SEN (CHWs).

Participants explained that during mealtimes, the family will share food from one large plate from which the father will eat first as a sign of respect and courtesy. Sometimes, children would also eat in their neighbour’s house to encourage them to eat.

“ Yes, it happens that we use that strategy so that children can eat. Note that children like to imitate so that’s why we [send them to the neighbour’s house]” – Workshop 11 SEN (CHWs)”.

Traditional beliefs about malnutrition

In Kaffrine, Senegal, some participants spoke of traditional beliefs relating to malnutrition, which are believed by fewer people these days. For example, uncovered food might attract bad spirits, and any person who eats it will become ill. There were a number of food taboos spoken of which were thought to have negative consequences for the baby, for example watermelon and grilled meat which were though to lead to birth complications and bleeding. Furthermore, cold water was thought to negatively impact the baby. Groups spoke of a tradition known as “bathie” in which traditional healers wash stunted children with smoke.

“ There are traditional practices called (Bathie) which are practiced by traditional healers. Parents are flexible about the practice of Bathie ” – Workshop 1 SEN (CHWs).

Causes of malnutrition and stunting were thought to be a lack of a balanced diet, lack of vitamin A, disease, intestinal worms, poor hygiene, socio-cultural issues such as non-compliance with food taboos, non-compliance with exclusive breastfeeding and close pregnancies. Malnutrition was also thought by some to be hereditary. Numerous signs of malnutrition were well known amongst the groups in Kaffrine. For example, signs of malnutrition were thought to be a big bloated belly, diarrhoea, oedema of the feet, anaemia, small limbs and hair loss as well as other symptoms such as red hair and a pale complexion. Despite this, malnutrition was thought to be hard to identify in Kaffrine as not all children will visit health centres, but mothers do try to take their babies heights and weights monthly. The groups were aware of the effect of poverty on the likelihood of stunting as impoverished parents cannot afford food. Furthermore, the groups mentioned that there is some stigma towards stunted children, and they can face mockery from other children although most local people feel pity and compassion towards them. Malnourished children are referred to as Khiibon or Lonpogne in the local language of Wolof.

“ It is poverty that is at the root of malnutrition, because parents do not have enough money [and] will have difficulty feeding their families well, so it is the situation of poverty that is the first explanatory factor of malnutrition here in Kaffrine” – Workshop 9 SEN (administrators).

“It can happen that some children are the victim of jokes for example of mockery from children of their same age, but not from adults and older ” – Workshop 9 SEN (administrators).

Pregnancy beliefs

In Kaffrine, Senegal, there were concerns around close pregnancies, and pregnancies in women who were too young, and for home births. Within the communities there was a stigma around close pregnancies, which prevented them from attending antenatal appointments. Similarly, there were superstitions around revealing early pregnancies, which again delayed attendance at health centres.

Groups acknowledged the role of good nutrition, and mentioned some forbidden foods such as salty foods, watermelon and grilled meat (which sometimes related back to a traditional belief that negative impacts would be felt in the pregnancy such as birth complications and bleeding). Similarly, drinking cold water was thought to negatively affect the baby. Beneficial foods mentioned included vegetables and meat, during pregnancy.

“ Often when a woman has close pregnancies, she can be ashamed, and this particularly delays the time of consultation” – Workshop 5 SEN (CHWs).

“Yes, there are things that are prohibited for pregnant women like salty foods” – Workshop 11 SEN (CHWs).

In Kaffrine, Senegal, some participants spoke of a traditional belief that if a pregnant woman consumes eggs then her baby might be overweight, or have problems learning how to talk. Despite this, mothers in Kaffrine said that they would be happy to accept eggs as a supplement, although if supplements are provided that require preparation (such as powdered supplements), they would be less likely to accept them.

“These restrictions are traditional, and more women no longer believe that eggs will cause a problem to the child. But if these eggs are brought by external bodies, we will hesitate to take it.” – Workshop 11 SEN (CHWs).

“They don’t eat eggs before the child starts speaking (the child only eats eggs when he starts talking). This is because it’s very heavy and can cause bloating and may also lead to intestinal problems.” – Workshop 4 SEN (grandmothers).

Other important topics – access to health services

For the participants in Kaffrine, Senegal, accessing health services was problematic, particularly for pre- and post-natal appointments, which faced frequent delays. Some women had access due to poor roads and chose to give birth at home. Access issues were further compounded by poverty and social factors, as procedures in hospitals can be costly, and women with close pregnancies (soon after an earlier one) can feel shame from society and hide their pregnancy.

“Women really have problems of lack of finances. There are social services in the hospital; but those services rarely attend to women without finances. Even when a child dies at birth they will require money to do the necessary procedure ” – Workshop 11 SEN (CHWs).

Creation of the culturally-informed protocols

Recommendations that comprise a culturally-informed protocol for intervention design in each locality are given in Table 3 .

The Major Cultural Themes, and specific Egg Intervention Themes drawn out from only 9–11 carefully planned group sessions in each country provided a rich set of recommendations towards a culturally-informed protocol for the localised design of a proposed Egg Intervention for both East Lombok, Indonesia and Kaffrine, Senegal. A culturally-informed protocol designed in this way comprises cultural insights which are worthy of consideration in local intervention design and should guide future stages of engagement and provide a platform from which good rapport and trust can be built between researchers and the community [ 16 ]. For example, in Kaffrine, Senegal, the early involvement of husbands and grandmothers is crucial, which reflects values around shared decision making within families that are noted to be more prevalent in LMICs, in contrast to individualistic values in HICs [ 16 , 39 ]. Similarly, due to strong religious values in both East Lombok, Indonesia and Kaffrine, Senegal, partnerships with Islamic leaders is likely to improve engagement. Past studies show the crucial role that religious leaders can play in determining social acceptability of interventions, particularly around taboo topics such as birth spacing [ 40 ].

The WVIS plus PEX:FGD method demonstrated here produced both broad cultural themes from shared values, which were in a concise and easy-to-understand format which could be readily communicated with the wider Action Against Stunting Hub, as well as life practices relevant to stunting in Kaffrine, Senegal and in East Lombok, Indonesia. Discussions of shared values during the WVIS main workshop provided useful cultural background within each community. PEX:FGD discussion uncovered numerous cultural factors within local life practices that could influence on the Egg Intervention engagement and acceptability. Combining themes from the WVIS workshop and PEX:FGDs allowed for specific recommendations to be made towards a culturally-informed protocol for the design of an Egg Intervention that included both broad cultural themes and specific Intervention insights (Table 3 ). For example, in Kaffrine, Senegal, to know that the husband’s authoritative family decision-making for health care (specific) is rooted in Islamic foundations (wider cultural) points to an Intervention Recommendation within the protocol, involving consultations with Islamic Leaders to lead community awareness targeting fathers. Similarly, in East Lombok, Indonesia the (specific) behaviour of breastfeeding for 2 years was underpinned by (wider cultural) shared values of living in Islam. This understanding of local values could prevent the imposition of culturally misaligned values, which Bernal and Adames (2017) caution against [ 17 ].

There are a number of interesting overlaps between values seen in the WVIS Frameworks and Narratives and the categories of Schwartz (1992) and The World Values Survey (2023) [ 41 , 42 ]. For example, in both Kaffrine, Senegal and East Lombok, Indonesia, strong religious values were found, and the groups spoke of the importance of practicing their religion with daily habits. This would align with traditional and conservation values [ 41 , 43 ]. Furthermore, in Kaffrine, Senegal participants often mentioned the importance of mutual aid within the community, and similar values of togetherness and respect in the community were found in East Lombok, Indonesia. These would seem to align with traditional, survival and conservation values [ 41 , 43 ]. However, the values mentioned by the groups in the WVIS workshops are far more specific, and it is possible that through asking what is most worthwhile, valuable and meaningful about their context, the participants are able to prioritise which aspects of their values are most salient to their daily lives. Grounded shared values such as these are generally neglected in Global Health Research, and values predominant in the Global North are often assumed to be universal [ 14 ]. Thus, by excluding the use of a predefined external framework, we minimized the risk of imposing our own ideas of values in the community, and increased the relevance, significance and local validity of the elicited information [ 28 ].

Participatory methods of engagement are an essential step in conducting Global Health Research but there is currently a paucity of specific guidance for implementing participatory methods in vulnerable communities [ 16 , 44 ]. In addition, there is acknowledgement in the literature that it is necessary to come into communities in LMICs without assumptions about their held values, and to use bottom-up participatory approaches to better understand local values [ 14 , 16 ]. The WVIS plus PEX:FGD methodology highlighted here exemplifies a method that is replicable in multiple country contexts [ 28 , 32 ] and can be used to crystallize local In Situ Shared Values which can be easily communicated to external researchers. Coupled with the specialised FGD (PEX:FGD), values-based perceptions of specific topics (in this case stunting) can be elicited leading to the creation of specific Culture-based recommendations. This therefore takes steps to answer the call by Memon and colleagues (2021) for the creation of cultural protocols ahead of conducting research in order to foster ethical research relationships [ 16 ]. We believe that the potential usefulness of the WVIS approach to guide engagement and inform intervention design is effectively demonstrated in this study and WVIS offers a method of making explicit local values in a novel and valuable way.

However, we acknowledge that our approach has several limitations. It has relied heavily on the local university researchers to debate and decide which participant stakeholder groups should be chosen, and although they did this in the context of the Whole Child approach, it would have been advantageous to have involved cultural researchers with a deeper understanding of cultural structures, to ensure sufficient opportunities for key cultural elements to emerge. This would have in particular strengthened the intervention design derived from the PEX:FGD data. For example, we retrospectively realised that our study could have been improved if grandmothers had been engaged in East Lombok. Understanding this limitation leads to suggestion for further work: to specifically investigate the overlap of this approach with disciplinary studies of culture, where social interactions and structures are taken into account via formal frameworks.

There are more minor limitations to note. For example, the WVIS approach can only be led by a trained and experienced facilitator: not all researchers can do this. A training programme is currently under development that could be made more widely available through online videos and a Handbook. Secondly, although the groups recruited do not need to be representative of the local population, the number recruited should be increased until theoretical saturation is achieved of the themes which emerge, which was not carried out in this study as we focussed on demonstrating the feasibility of the tool. Thirdly, there is a limit to the number of topics that can be explored in the PEX:FGDs within the timeframe of one focus group (depending on the stamina of the participants), and so if a wider range of topics need formative research, then more workshops are needed. Lastly, this work took place in a large, highly collaborative project involving expert researchers from local countries as well as international experts in WVIS : other teams may not have these resources. However, local researchers who train in WVIS could lead on their own (and in this Hub project such training was available).

The need for better understanding, acknowledgement and integration of local culture and shared values is increasing as the field of Global Health Research develops. This study demonstrates that the WVIS plus PEX:FGD shared values approach provides an efficient approach to contextualise and localise interventions, through eliciting and making communicable shared values and local life practices which can be used towards the formation of a culturally-informed protocols. Were this method to be used for intervention design in future, it is possible that more focus should be given to existing social structures and support systems and a greater variety of stakeholders should be engaged. This study thus contributes to the literature on methods to culturally adapt interventions. This could have significant implications for improving the uptake of nutrition interventions to reduce malnutrition through improved social acceptability, which could help progression towards the goal of Zero Hunger set within the SDGs. The transferability and generalisability of the WVIS plus PEX:FGD approach should now be investigated further in more diverse cultures and for providing formative research information for a wider range of research themes. Future studies could also focus on establishing its scaling and pragmatic usefulness as a route to conceptualising mechanisms of social acceptability, for example a mechanism may be that in communities with strong traditional religious values, social hierarchies involving religious leaders and fathers exist and their buy-in to the intervention is crucial to its social acceptability. Studies could also focus on the comparison or combination of WVIS plus PEX:FGD with other qualitative methods used for intervention design and implementation.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request [email protected], Orcid number 0000–0002–1811-4597. These include deidentified Frameworks of Shared Values and Accompanying Narrative from each Group; deidentified Hub Insight Statements of relevant themes.

World Health Organization and Food and Agriculture Organization of the United Nations. Driving commitment for nutrition within the UN Decade of Action on Nutrition: policy brief. Geneva: World Health Organization; 2018.

Google Scholar  

Victora CG, Christian P, Vidaletti LP, Gatica-Domínguez G, Menon P, Black RE. Revisiting maternal and child undernutrition in low-income and middle-income countries: variable progress towards an unfinished agenda. Lancet. 2021;397(10282):1388–99.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Aguayo VM, Nair R, Badgaiyan N, Krishna V. Determinants of stunting and poor linear growth in children under 2 years of age in India: an in-depth analysis of Maharashtra's comprehensive nutrition survey. Matern Child Nutr. 2016;12 Suppl 1(Suppl 1):121–40.

Article   PubMed   Google Scholar  

Beal T, Tumilowicz A, Sutrisna A, Izwardy D, Neufeld LM. A review of child stunting determinants in Indonesia. Matern Child Nutr. 2018;14(4):e12617.

Article   PubMed   PubMed Central   Google Scholar  

Hossain M, Choudhury N, Adib Binte Abdullah K, Mondal P, Jackson AA, Walson J, et al. Evidence-based approaches to childhood stunting in low and middle income countries: a systematic review. Arch Dis Child. 2017;102(10):903–9.

Brar S, Akseer N, Sall M, Conway K, Diouf I, Everett K, et al. Drivers of stunting reduction in Senegal: a country case study. Am J Clin Nutr. 2020;112(Suppl 2):860s–74s.

Heidkamp RA, Piwoz E, Gillespie S, Keats EC, D'Alimonte MR, Menon P, et al. Mobilising evidence, data, and resources to achieve global maternal and child undernutrition targets and the sustainable development goals: an agenda for action. Lancet. 2021;397(10282):1400–18.

Article   CAS   PubMed   Google Scholar  

Goudet SM, Bogin BA, Madise NJ, Griffiths PL. Nutritional interventions for preventing stunting in children (birth to 59 months) living in urban slums in low- and middle-income countries (LMIC). Cochrane Database Syst Rev. 2019;6(6):Cd011695.

PubMed   Google Scholar  

Desai S, Misra M, Das A, Singh RJ, Sehgal M, Gram L, et al. Community interventions with women's groups to improve women's and children's health in India: a mixed-methods systematic review of effects, enablers and barriers. BMJ Glob Health. 2020;5(12)

Muraya KW, Jones C, Berkley JA, Molyneux S. Perceptions of childhood undernutrition among rural households on the Kenyan coast – a qualitative study. BMC Public Health. 2016;16(1):693.

Isler J, Sawadogo NH, Harling G, Bärnighausen T, Adam M, Sié A, et al. 'If he sees it with his own eyes, he will understand': how gender informed the content and delivery of a maternal nutrition intervention in Burkina Faso. Health Policy Plan. 2020;35(5):536–45.

Zaidi S, Das JK, Khan GN, Najmi R, Shah MM, Soofi SB. Food supplements to reduce stunting in Pakistan: a process evaluation of community dynamics shaping uptake. BMC Public Health. 2020;20(1):1046.

Wight D, Plummer M, Ross D. The need to promote behaviour change at the cultural level: one factor explaining the limited impact of the MEMA kwa Vijana adolescent sexual health intervention in rural Tanzania. A process evaluation. BMC Public Health. 2012;12(1):788.

Aubel J, Chibanda D. The neglect of culture in global health research and practice. BMJ Glob Health. 2022;7(9):e009914.

Article   PubMed Central   Google Scholar  

Myser C. Defining "global health ethics": offering a research agenda for more bioethics and multidisciplinary contributions-from the global south and beyond the health sciences-to enrich global health and global health ethics initiatives. J Bioeth Inq. 2015;12(1):5–10.

Memon R, Asif M, Khoso AB, Tofique S, Kiran T, Chaudhry N, et al. Recognising values and engaging communities across cultures: towards developing a cultural protocol for researchers. BMC Med Ethics. 2021;22(1):47.

Bernal G, Adames C. Cultural adaptations: conceptual, ethical, contextual, and methodological issues for working with Ethnocultural and majority-world populations. Prev Sci. 2017;18(6):681–8.

Glickman SW, McHutchison JG, Peterson ED, Cairns CB, Harrington RA, Califf RM, et al. Ethical and scientific implications of the globalization of clinical research. N Engl J Med. 2009;360(8):816–23.

King KF, Kolopack P, Merritt MW, Lavery JV. Community engagement and the human infrastructure of global health research. BMC Med Ethics. 2014;15(1):84.

Horton R. Offline: is global health neocolonialist? Lancet. 2013;382(9906):1690.

Article   Google Scholar  

Woodward EN, Matthieu MM, Uchendu US, Rogal S, Kirchner JE. The health equity implementation framework: proposal and preliminary study of hepatitis C virus treatment. Implement Sci. 2019;14(1):26.

The Pūtaiora Writing Group. Te Ara Tika Guidelines for Māori research ethics: A framework for researchers and ethics committee members. Health Research Council of New Zealand. (no date). Online, Accessed 5 th May 2023. Available from: http://www.hrc.govt.nz/sites/default/files/2019-06/Resource%20Library%20PDF%20-%20Te%20Ara%20Tika%20Guidelines%20for%20Maori%20Research%20Ethics.pdf

Pelto GH, Armar-Klemesu M, Siekmann J, Schofield D. The focused ethnographic study 'assessing the behavioral and local market environment for improving the diets of infants and young children 6 to 23 months old' and its use in three countries. Matern Child Nutr. 2013;9 Suppl 1(Suppl 1):35–46.

Bentley ME, Johnson SL, Wasser H, Creed-Kanashiro H, Shroff M, Fernandez Rao S, et al. Formative research methods for designing culturally appropriate, integrated child nutrition and development interventions: an overview. Ann N Y Acad Sci. 2014;1308:54–67.

Kodish S, Aburto N, Dibari F, Brieger W, Agostinho SP, Gittelsohn J. Informing a behavior change communication strategy: formative research findings from the scaling up nutrition movement in Mozambique. Food Nutr Bull. 2015;36(3):354–70.

Mattes RD, Rowe SB, Ohlhorst SD, Brown AW, Hoffman DJ, Liska DJ, et al. Valuing the diversity of research methods to advance nutrition science. Adv Nutr. 2022;13(4):1324–93.

Robert RC, Bartolini RM, Creed-Kanashiro HM, Verney SA. Using formative research to design context-specific animal source food and multiple micronutrient powder interventions to improve the consumption of micronutrients by infants and young children in Tanzania, Kenya, Bangladesh and Pakistan. Matern Child Nutr. 2021;17(2):e13084.

Sethamo OA, Masika RJ, Harder MK. Understanding the role of crystallizing local shared values in fostering effective community engagement in adaptation planning in Botswana. Clim Dev. 2020;12(5):448–56.

Odii EC, Ebido CC, Harder MK. A values-based approach for generating localized social indicators for use in sustainability assessment and decision-making: test case of brownfield soft reuse in Nigeria. Sci Total Environ. 2020;711:135045.

Podger D, Piggot G, Zahradnik M, Janoušková S, Velasco I, Hak T, et al. The earth charter and the ESDinds initiative: developing indicators and assessment tools for civil society organisations to examine the values dimensions of sustainability projects. J Educ Sustain Dev. 2010;4(2):297–305.

Odii BC, Huang Y, Bouvrie N, Harder MK. Cycles of meaning-making crystallization in the WeValue InSitu process as clear contributions towards transformative learning. J Clean Prod. 2021;304:127024.

Huang Y, Wu W, Xue Y, Harder MK. Perceptions of climate change impacts on city life in Shanghai: through the lens of shared values. Cleaner Prod Lett. 2022;3:100018.

Action Against Stunting Hub. UKRI GCRF action against stunting: alleviating child undernutrition, globally. 2020. URL: https://actionagainststunting.org/ [Accessed 15/08/2022].

UNICEF. Nutrition country profiles . 2010. URL: https://data.unicef.org/resources/nutrition-country-profiles/ [Accessed 10/05/2023].

Charmaz K. Constructing grounded theory. 2nd ed. London: SAGE; 2014.

Noblit GW, Hare RD, Hare RD. Meta-ethnography: synthesizing qualitative studies: sage; 1988.

Stelle I, McDonagh LK, Hossain I, Kalea AZ, Pereira DIA. Nutrients. 2021;13(4):1140.

Muraya KW, Jones C, Berkley JA, Molyneux S. "If it's issues to do with nutrition…I can decide…": gendered decision-making in joining community-based child nutrition interventions within rural coastal Kenya. Health Policy Plan. 2017;32(suppl_5):v31–v9.

Aubel J. Grandmothers — a neglected family resource for saving newborn lives. BMJ Glob Health. 2021;6(2):e003808.

Egeh AA, Dugsieh O, Erlandsson K, Osman F. The views of Somali religious leaders on birth spacing - a qualitative study. Sex Reprod Healthc. 2019;20:27–31.

Schwartz SH. Universals in the content and structure of values: Theoretical advances and empirical tests in 20 countries. Advances in experimental social psychology. 25: Elsevier; 1992. p. 1–65.

World Values Survey Association. Findings and insights, 2020 [Available from: https://www.worldvaluessurvey.org/WVSContents.jsp .]

Inglehart R, Welzel C. Value change and the persistence of cultural traditions. Modernization, cultural change, and democracy: the human development sequence. Cambridge: Cambridge University Press; 2005. p. 48–76.

Glandon D, Paina L, Alonge O, Peters DH, Bennett S. 10 best resources for community engagement in implementation research. Health Policy Plan. 2017;32(10):1457–65.

Download references

Acknowledgements

We thank the Hub PI, Claire Heffernan, for feedback on a late draft of the manuscript.

The Action Against Stunting Hub is funded by the Medical Research Council through the UK Research and Innovation (UKRI) Global Challenges Research Fund (GCRF), Grant No.: MR/S01313X/1.

Author information

Authors and affiliations.

Values & Sustainability Research Group, School of Architecture, Technology and Engineering, University of Brighton, Brighton, UK

Annabel J. Chapman, Mahsa Firoozmand & Marie K. Harder

Department of Environmental Science and Engineering, Fudan University, Shanghai, People’s Republic of China

Chike C. Ebido, Rahel Neh Tening, Yanyan Huang & Marie K. Harder

Department of Zoology and Environmental Biology, University of Nigeria, Nsukka, Nigeria

Chike C. Ebido

Preventive Medicine and Public Health, Université Cheikh Anta Diop (UCAD), Dakar, Senegal

Ndèye Marème Sougou

Faculty of Psychology, Universitas Islam Negeri Syarif Hidayatullah, Jakarta, Indonesia

Risatianti Kolopaking

Southeast Asian Ministers of Education Organization Regional Centre for Food and Nutrition (SEAMEO RECFON) Universitas Indonesia, Jakarta, Indonesia

Risatianti Kolopaking & Rita Anggorowati

International Research Laboratory (IRL 3189) Environnement santé et sociétés/CNRS/UCAD, Dakar, Senegal

Amadou H. Diallo

Department of Medical Records and Health Information, Faculty of Health and Technology, Universitas Bandung, Bandung, Indonesia

Rita Anggorowati

Laboratory of Cultural Anthropology, IFAN, Université Cheikh Anta Diop (UCAD), Dakar, Senegal

Fatou B. Dial & Cheikh El Hadji Abdoulaye Niang

School of Education, Languages and Linguistics, Faculty of Humanities and Social Sciences, University of Portsmouth, Portsmouth, UK

Jessica Massonnié

Department of Learning and Leadership, IOE, UCL’s Faculty of Education and Society, University College London, London, UK

You can also search for this author in PubMed   Google Scholar

Contributions

MKH formulated the initial research question and study design. AJC developed the specific research question. Data collection in Senegal involved CCE, NMS, AHD, FBD, RNT, CEHAN and JM. Data collection in Indonesia involved RA, RK, YH and MKH. Cultural interpretation in Senegal Involved AHD, FBD, NMS, RNT and JM. Analysis involved AJC and MF. AJC and MKH wrote the paper.

Corresponding author

Correspondence to Marie K. Harder .

Ethics declarations

Ethics approval and consent to participate.

The research was conducted in accordance with the Declaration of Helsinki and has been approved by the Ethics Review Board of the University of Brighton, and national ethics committees for research in Indonesia and Senegal. Informed consent was obtained in the local vernacular language, Bahasa, French or Wolof. Participants retained a copy of the informed consent document for reference.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Chapman, A.J., Ebido, C.C., Tening, R.N. et al. Creating culturally-informed protocols for a stunting intervention using a situated values-based approach ( WeValue InSitu ): a double case study in Indonesia and Senegal. BMC Public Health 24 , 987 (2024). https://doi.org/10.1186/s12889-024-18485-y

Download citation

Received : 27 September 2023

Accepted : 29 March 2024

Published : 09 April 2024

DOI : https://doi.org/10.1186/s12889-024-18485-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Health intervention
• Shared values
• Culturally-informed protocol
• Cultural factors
• WeValue InSitu

BMC Public Health

ISSN: 1471-2458

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Design principles to develop digital innovation skills: a design-based research approach

• Development Article
• Open access
• Published: 10 November 2023

Cite this article

You have full access to this open access article

• Carolien van den Berg   ORCID: orcid.org/0000-0002-2243-8375 1 &
• Vivienne Bozalek 2  

926 Accesses

Explore all metrics

The digitisation of the world has led to a multitude of far-reaching implications that require students to be prepared for the dynamic era of rapid change, complexity, fluidity, and ubiquity in which they will work at the forefront of technology. To succeed in this environment, students must be able to design and implement digital innovations within the broader spectrum of digital transformation. Despite the importance of this area, the literature shows a lack of research on how digital business innovation skills can be effectively taught to students. To address this gap, a design-based research (DBR) study was conducted using a mixed-methods design through three iterations at a South African university. The study aimed to answer the research question of how digital business innovation skills should be taught to South African Information Systems students. The study commenced with an analysis of practical problems experienced by practitioners, industry, students, and researchers and an initial review of pertinent literature. The literature review focused on the impact of digitisation on future skills requirements to inform the pedagogy, content, and technology applicable to the teaching and learning environment. The findings yielded design principles for the design of the learning environment that were tested and refined via three iterations, resulting in nine design principles. The aim was to ensure a future-oriented, industry-informed curriculum design that is relevant to the digital economy.

Similar content being viewed by others

Design principles for internet skills education: results from a design-based research study in higher education

Ourania Miliou & Charoula Angeli

Broadening design-led education horizons: conceptual insights and future research directions

Natalie Wright & Cara Wrigley

University Innovation Hubs & Technology-Enhanced Learning in K12 Environments

Josh Herron & Kathryn A. Wolfe

Avoid common mistakes on your manuscript.

Introduction

Advances in digital technologies are transforming society and the way we live and work, with digital innovation being a major driver of these changes (Bogers et al., 2022 ). Digital innovation refers to the creation of new market offerings, business processes, or models resulting from the use of digital technology (Nambisan et al., 2017 ). This shift towards a digital future requires a new type of citizen who can function in more unstructured and unpredictable circumstances. Higher education institutions (HEIs) must adapt by offering effective, innovative, and high-quality learning experiences that equip students with the necessary skills for a changing labour market (Alexander et al., 2019 ).

As we move towards a more digital world, the demand for higher-order general cognitive skills such as problem-solving, critical thinking, innovation, creativity, and collaboration is increasing (Djankov et al., 2019 ). Additionally, socio-emotional skills such as collaboration, teamwork, resilience, and adaptability are becoming increasingly important (ibid, 2019). In the field of Information Systems (IS), the general skills needed are moving beyond technical expertise to include higher-level integration and the role of cognitive skills (van den Berg, 2019 ; Goulart & Liboni, 2022 ).

This paper focuses on defining the skills requirements of IS students to become competent digital innovators and to develop design principles to teach digital innovation, ensuring industry-informed curriculum design that is future-proof within a digital economy. Validated design principles that can be adopted by future developers to design learning environments that enhance digital innovation capabilities are proposed. These design principles were developed and refined via a four-phased design-based research (DBR) approach as proposed by Reeves ( 2006 ). This paper contributes to DBR literature by providing insight into the process followed to articulate validated design principles while also sharing reflections. The paper commences with a review of the development of digital innovation skills. Subsequently, the application of DBR and the methodology applied are presented. The results of the three iterations are then discussed by focussing on a portion of the analysis applied in each iteration. The paper concludes with the recommended design principles that can be applied to teach digital innovation skills.

Developing digital innovation skills

The focus of the study was on digital innovation and not digital transformation due to the ability to implement digital innovations in individual areas of an organisation whereas digital transformation refers to systemwide, long-term change. Digital innovation foregrounds new digital products and services, whilst digital transformation is the application of digital technologies to transform an organisation's operations and culture.

A digital innovator can see new possibilities created by advances in technology that meet organisational or societal needs (Fichman et al., 2014 ). Digital innovation is the recombination of something that already exists with new technology and follows a nonlinear pattern of innovation diffusion to change the ways products and services are developed, produced and used (Bogers et al., 2022 ).

In the study, the first objective was to identify the skills required by IS students to develop the competence to be digital innovators. The umbrella term used to describe the type of skills that students will need in the digital economy is “21st-century skills” (21st CS). Many different frameworks identify 21st CS, but the one deemed to be the most comprehensive was a study by Kereluik et al. ( 2013 ). They compared more than 15 frameworks to identify the types of knowledge claimed to be integral to 21st CS. This framework was used to test different skill sets and to identify the most prominent skills required to develop digital innovation capability. Figure  1 illustrates this framework.

Synthesis of 15 different 21st-century learning frameworks into one visual image (Kereluik et al., 2013 )

The development of certain 21st-century skills to teach digital business innovation can be enhanced by an authentic learning environment in which students must be “engaged in an inventive and realistic task that provides opportunities for complex collaborative activities” (Herrington et al., 2010 , p. 1). The principles of authentic learning as outlined in Herrington et al. ( 2010 ) are described in Table 1 with a practical application attached to each principle as applicable to the cultivation of digital innovation skills.

• Design-based research

The study is situated within the paradigm of design research. Design science research in the field of Information Systems (IS) is characterised by the use of human creativity to develop innovative artefacts that address problems in digital environments (Hevner & Chatterjee, 2010 ). This type of research employs design cycles to test and refine solutions, with an emphasis on the design and implementation of novel artefacts that contribute to the advancement of the field of digital innovation (Hevner et al., 2019 ).

Design research applied in an educational setting follows a similar process of iterative development of solutions to complex and practical educational issues (McKenney & Reeves, 2019 ). The fundamental premise of design research is to apply theory to ground the design process, with the ultimate goal of expanding scientific understanding. This approach is collaborative, with input from multiple stakeholders representing different disciplines within iterative cycles of design, development, testing, and revision (McKenney & Reeves, 2019 ).

Design-based research (DBR) is an iterative approach that involves continuous design cycles within authentic learning settings to test and refine theories and advance practice. DBR studies use a mixed-methods design and involve multiple parties such as designers, researchers, and practitioners with diverse expertise to guide the design, conduct, and reporting of the research (McKenney & Reeves, 2021 ). This approach allows for the development of practical and effective solutions to real-world problems within educational settings and is an effective methodology for advancing educational practice (McKenney & Reeves, 2019 ; Reeves, 2006 ).

The four phases applied to this study are illustrated in Fig.  2

Adapted from Design-based research approaches in educational technology research (Reeves, 2006 )

Phase 1: stakeholder consultation and a review of the literature

In a DBR study, the design principles are collaboratively developed among various stakeholders and underpinned by a review of the literature. The problems are explored by parties who deal with them on a day-to-day basis via a consultative process such as participant observation and conversation, interviews, focus groups or reflective journals and blogs (Herrington & Reeves, 2011 ; Herrington et al., 2010 ). In this study, consultations with industry participants, students and higher education practitioners in IS were conducted via interviews and focus groups see Table 1 , Phase 1.

Phase 2: development of solutions

During this phase, the literature review was extended to find additional theories and existing design principles that address similar problems. This was further expanded to create the draft design principles.

Phase 3: iterative cycles of testing and refinement

According to Herrington and Reeves ( 2011 ), a single implementation cannot gather enough evidence about the success of the intervention prompting which prompted three iterations over three consecutive years. After each iteration, changes were made to improve the design to better address the problem in the subsequent iteration. Phase 1 and 2 required empirical research to be conducted and the data collection and analysis are depicted in Tables 2 , 3 .

Phase 4: design principles

Once a learning design or intervention had been implemented, evaluated and refined in cycles, the last phase was to reflect on the entire process to produce design principles that could inform future development and implementation decisions. The aim is to provide at least three useful outcomes namely the design principles, a representation of the learning environment and societal outputs, such as professional development and learning (Herrington & Reeves, 2011 ).

Phase 1: analysis of practical problems

In the first phase, consultation with industry partners who participated in the student project during previous years took place. The purpose was to understand their perception regarding the impact of digital innovation on organisations in South Africa, and further the type of skills required by IS students. The aspects highlighted included the development of social intelligence, creative thinking, and an innovative approach to problem-solving. Industry participants valued students who have been exposed to a “real work” environment via, for example, internships or projects in collaboration with the industry.

Six lecturers in the IS department at the University of the Western Cape (UWC) were consulted. They were concerned about the exact meaning and extent of digital skills requirements and how to prepare students for a digital economy. Limitations in the current IS curriculum were also highlighted as an issue to address. The practitioners further expressed their concerns about their practice and how this meets the requirements of a changing landscape.

A review of student perception related to skills requirements for a digital society took place before the first iteration. The students expressed the need to become more technically flexible. They also expressed the requirement to have more exposure to the industry and work on projects that deal with real-world problems.

The challenges faced by the different stakeholders are depicted in Fig.  3 .

Challenges identified by students, teachers and industry

Phase 2: draft design principles

The draft principles informed the development of the proposed solution, and the technological affordances identified also formed part of the process for drafting design principles. A mapping of the curriculum design principles to the actual learning environment, including the skill sets and authentic learning elements required (see Table 2 ), is depicted in Table 4

A Design-Based Research (DBR) approach was adopted to enhance the intervention through iterative cycles of data gathering, testing, and verification. After each iteration, the design was refined based on the findings. A review of each draft design principle was conducted by examining the authentic learning elements and skill sets associated with each principle (refer to Table 4 ) to analyse the outcomes. The analytical approach involved applying quantitative analysis first to test the skill sets acquired by learners (S1 to S8), followed by qualitative analysis to assess the presence of authentic learning elements in the course (A1 to A9). Subsequently, the content presented and the technology applied were analysed to determine the overall outcome of each draft design principle. The principles were updated and refined after each iteration to improve the intervention's overall effectiveness and refine the framework.

Given the extensive data analysis conducted over three years, it is not possible to present the results for each iteration for each section, such as quantitative, qualitative, project artefacts, industry feedback, and facilitator reflections. Instead, the focus will be on the quantitative analysis in iteration 1, the qualitative analysis in iteration 2, and project artefacts and industry feedback in iteration 3. The reflections on how the design principles were reviewed and updated are visually presented for the first two iterations.

Iteration 1

The first iteration took place during a first-semester course with a group of 40 postgraduate IS students. The objectives of the course were the identification, creation and implementation of digital innovation within a client’s business (industry partner). Groups could choose clients within the creative industries sector in Cape Town. As stipulated, the quantitative analysis is portrayed for the first iteration followed by the qualitative analysis in iteration 2 and project artefacts and industry feedback in iteration 3.

Quantitative results iteration 1

A regression analysis was conducted to determine the relationships between the variables (skills as indicated in Table 3 coded S1–S9) obtained in the survey results. The regression analysis applied student assessment scores as the dependent variable to test the importance of the different skills. The analysis tested the reliance on certain skills during an initial assessment and again at the end of the course. The analysis helped to identify the skills that were statistically significant using the p-value to test the null hypothesis. The collected data were analysed using Excel and the Statistical Package for Social Science (SPSS). The quality of results was verified with a hypothesis test where the null hypothesis was all the slope coefficients of the model equalling zero and the attentive hypnosis was that at least one of the slope coefficients is not equal to zero. The hypothesis is rejected if at least one of the independent variables explains the value of the dependent variable by reviewing the p-value. If the p-value is less than the level of significance, the null hypothesis that the coefficient equals zero is rejected; the variable is therefore statistically significant (Anderson, 2014 ). Typically you would like to produce a high R-value, thus a low p-value with a high R will indicate that the results explain the response variability. However, when one predicts human behaviour, lower R 2 values are acceptable because humans are harder to predict (Anderson, 2014 ). For this analysis, the p-value, therefore, was examined more closely.

The second statistical test applied was a one-way ANOVA to test the differences between the students’ scores for their initial skills survey and the scores obtained for the second survey upon completion of the module. The purpose of one-way ANOVA is to test whether the means of different groups are common or different. The quantitative results are depicted in Table 5 below.

The dependent variable in the regression analysis was student assessment scores using the initial assessments (first blog post, initial presentations and peer reviews) for the first (pre) survey and the final assessments (final blog posts, industry presentations and reports) for the final (post) survey. Table 6 depicts the summary of findings applied to each iteration to review the overall results about skills development.

This process was repeated in each iteration and the results were analysed to review the skill sets that showed an improvement and to highlight areas where further interventions were required. See Table 7 for an example of the review of the multiple regression over the three iterations to show the progress.

The overall findings after each iteration were reviewed and the areas that were deemed to be satisfactory were highlighted in green, the areas that required improvement in amber, and the areas that needed intervention and new strategies in the following iterations were highlighted in red. These reflections are summarised in Fig.  4 below.

Outcome of iteration 1 and additional design principle added

Figure  4 depicts the dashboard applied to summarise the overall outcomes, as seen several areas required further interventions in the following iteration. Only two of the draft design principles were successfully integrated and the following iteration required changes in the design of the projects with industry, the group formation, the assessment of students, coaching and scaffolding and the quality of feedback. After iteration 1, a new principle was added to allow all tasks to be funnelled into a comprehensive capstone project that applies agile methods. The application of Agile allows changes to group formation, regular feedback and the assessment of the overall process and not merely the end solution.

Iteration 2

The second iteration took place during a second-semester course with a group of 42 Information Systems students in their final year. The design of the project changed to incorporate an Agile methodology with clear team roles and regular opportunities for feedback by the facilitator and other teams. The assessments were designed to measure the outcomes per week to allow teams to make changes and improvements. A similar process of data collection was applied. As stipulated, the qualitative analysis is discussed in the second iteration but a similar process was applied to the analysis of the skills as described in iteration 1.

Qualitative results iteration 2

The steps prescribed by Miles and Huberman ( 1994 ) to systematically organise the data were applied in the qualitative data analysis. Firstly, data was organised and emerging patterns were identified from the different sources. Thereafter, data was coded about the key pedagogical and design principles identified in the literature and sorted into potential themes (see Table 2 ). The data analysis phases indicated in the DBR approach were followed through the iterative cycles.

During each iteration, students were tasked to subscribe to a blog and submit three blogs during the semester. The blogs were reviewed to find evidence of authentic learning elements to which the students responded positively in their learning.

The learning environment needs to enhance the ability of students to apply critical thinking to develop the capabilities to become digital innovators. This requires an authentic context (A1) to enable students to apply their knowledge as they would in real life to find their own solutions in the implementation of digital innovations. It also encourages interdisciplinary skills development in the implementation of the capstone project (updated principle). For example:

The course was very phenomenal because it teaches about the current issues facing the technological sector and how to improve business processes for an organisation using advance technology. (LJ2)

However, students also felt that the engagement with industry partners was not sufficient and that their projects were not “real” enough. An area that needed to be redesigned in the next iteration was the type of industry partners with whom students engage. Students ought to be partnered with industry partners that are active in the community for them to see a real change in terms of their digital innovations for example:

I would like to suggest that in the next group, the lecturer must identify companies to work with and actually make sure that relations are built beforehand because companies are disinterested in projects that are consultative. (St23)

Authentic tasks (A2) stimulate collaboration and consist of ill-defined activities that create a polished product with real-world relevance. They are complex and performed over some time to promote competing solutions and a diversity of outcomes. For example, as quoted by a student:

I found the incremental steps in developing a product interesting. It made me realise that all the small parts come together to form a product or final solution. (PG2)

Peer reviews and online feedback were utilised to achieve access to expert performance (A3). During iteration 1, students felt uncomfortable with this and more coaching was done to encourage participation. Students needed guidance on how to give and receive feedback, and this was an aspect that had to be built into the rubric to test the peer review process. Positive outcomes were achieved for example:

What I learnt from the exercise was that one does not see their mistakes but quickly notices them in someone else. What I mean by this is that the groups spotted what the other groups did not do or did wrong in their assignments but in actual fact, they also did not do the same mistake but they did not take note of it. (BN2)

An area in which students felt that they lacked expert performance was particularly their technical ability. Students will need more assistance from experts in the rollout of their projects, particularly in areas in which they are not that comfortable. As remarked by students:

More practical sessions to train students more about how to create a website from scratch or through using platforms such as WIX. (St24)

The provision of multiple roles and perspectives (A4) was explored more during the second iteration to try to encourage students to explore different avenues. More time was spent in class during which students had to work in their teams and analyse their chosen business from different perspectives. They were given a set of questions to answer and present to their peers regarding the industry forces that have an impact on their business, as well as the market forces and key trends that they envisage.

In the second iteration, the creation of a collaborative learning environment (A5) was expanded through the use of Google Drive. This was expressed by a student:

What I have noticed and learnt is that this module is presented digitally well I guess it has to be because, after all, it is digital business innovation. (BN2)

However, group work and collaboration are a challenge and this needs to be monitored throughout to facilitate conflict resolution and teach students the necessary skills to cope in a group environment. As remarked:

Put more exercises which focuses [sic] on the individual because I don’t think personal development occurs much in group assignments. (ST2)

The importance of incorporating individual reflection (A6) was stressed in the literature and incorporated into the course from the first iteration. During the first and second iterations, this area was challenging for students as they were not familiar with reflective exercises, but the usefulness thereof was grasped by some students towards the end. This is expressed in the following quotes:

I did not enjoy being marked by my peers. (LJ7) I didn't enjoy doing many self-evaluations. (St38) I must say, blogging has really made me look deeper into topics and buzzwords in the world of science and technology. And this has for the first time challenged me as a Technology student t think outside the box and share my ideas about the coming future. (KW2)

Articulation to enable tacit knowledge to be made explicit (A7) was emphasised via regular presentations in class as remarked by a student:

The effective presentation and scrum session in class were very fundamental to help me to grow my understanding in the course. I have strong understanding now of how operations of the business function especially how to apply the knowledge from the course. (LJ18)

The principle of coaching and scaffolding (A8) can be achieved through assessment tasks that facilitate student engagement over time, with feedback generated by various sources. The facilitator needs to carefully coach the teams and put just enough scaffolding in place to enable teams to construct their understanding. It is always a careful balance though as some students need more and others less for example:

I wish we had more time with the lecturer, and that she made reference to projects she has done and part-took in, in the past and what she did when she was faced with hurdles and what happens when things do not go according to plan. (ST2) Lecturer was somewhat repetitive when relaying learning material during group discussions which were distracting . (BM2)

The utilisation of authentic assessment (A9) is recommended to be integrated throughout the entire assessment process. In the second iteration, assessments were updated with more detailed rubrics to evaluate students on various aspects, such as their collaboration, communication, and content knowledge, at the end of the semester.

Figure  5 presents a summary of the review of iteration 2, following a similar process to that of iteration 1. It was observed that a new design principle was required, emphasizing the cultivation of “social change-makers” who implement digital innovations that benefit both businesses and society, as this was felt to be lacking.

Outcome of iteration 2 and additional design principle added

Iteration 3

The third iteration took place during a second-semester course with a group of 31 students enrolled in a third-year IS course. After the first two iterations, it became apparent that the students did not learn enough from their industry partners and a different approach to partnering with entrepreneurs in the start-up phase of their business was tried. A start-up phase is more open to change and the opinion was that students would be able to propose initiatives to support digital innovations in the business.

The organisations that the teams worked with included a hair salon that produces natural hair products, a fashion designer, a recruitment agency, a guest house, a quantity surveyor and a winemaker. Table 8 highlights the overview and the results are depicted in (Table 9 ).

There was an overall improvement in the results obtained from the industry in the third iteration on the conduct of teams. In their engagement with entrepreneurs, the teams were better able to sell the benefits of the digital economy, according to the findings from the last section.

Phase 4: updated design principles

In the fourth phase, the draft design principles developed and updated during the three iterations were updated as portrayed in Table 10 . It is important to note that the iterative process that culminated in the final design principles occurred over three years. As noted, the comprehensive data analysis that took place cannot be discussed in a single paper. The purpose of this article is to provide a view of the overall process that can be applied during a DBR study noting that “communicating the processes and outcomes of EDR studies can be challenging because these studies are typically large and complex and because their value to non-stakeholders is not always articulated” (McKenney & Reeves, 2021 . p. 89).

Within the study, the students were active participants or co-creators of the research. Ethical approval was obtained and participation in the study was voluntary. The identities of students were protected and there were no risks to them for their participation in the project.

This was a collaborative effort, and much of the insights obtained came from continuous engagement among all the parties involved. Also, the engagement with industry participants required continuous interventions and tweaking of project results. It was not possible to pre-empt any interpersonal issues that occurred in the teams or requirement limitations experienced with the industry partners. The actions taken were different for the different groups and resulted in different reactions. However, it is difficult to gauge whether the positive and negative outcomes were a result of the interventions, or other, external factors.

The updated course design is depicted in Table 10 with more detail about the design principles and a summary of the updated design principles in Table 11 .

The updated design principles were implemented in the curriculum design of exit-level IS courses at the university. The principles continue to be tested and refined within the courses. Student projects are now fully interdisciplinary with groups from different universities and different disciplines working collaboratively to implement digital innovations. The student projects also include a strong emphasis on sustainable development goals (SDGs) during the scoping. The application of technology within the learning environment is also continuously updated and refined with a strong emphasis on the different technology tools to enhance interdisciplinary learning in a blended environment (van den Berg & Verster, 2022 ). It is expected that the model will continue to evolve as more practitioners implement it within their teaching and learning environments. One of the purposes of DBR is to circulate information to the broader educational community to inform both theory and practice.

The digitalisation of society has brought about significant change, with digital innovation being a major driver of transformation. As such, higher education institutions must provide effective, innovative, and high-quality learning experiences that equip students with the necessary skills for a changing labour market. This paper aimed to define the skills requirements of IS students to become competent digital innovators and to develop design principles to teach digital innovation, ensuring industry-informed curriculum design that is future-proof within a digital economy. To achieve this, a four-phased design-based research approach was utilised, and the study was conducted in a South African university, involving students, industry practitioners, and researchers.

The study aimed to answer the research question of how digital business innovation skills should be taught to South African Information Systems students. The findings yielded nine design principles that ensure a future-oriented, industry-informed curriculum design that is relevant to the digital economy. These principles include collaboration, experimentation, the application of design thinking, interdisciplinary problem-solving, regular reflection, project-based learning and industry participation.

This study contributes to the literature by providing valuable insights into the process of articulating validated design principles. However, the study is not without limitations. It was conducted in a single university, limiting the generalisability of the findings, and the study’s design and approach may not apply to other contexts or disciplines. Future research should explore the effectiveness of the design principles in other educational settings and examine their transferability across different domains. Ultimately, the study's findings have implications for the development of future-oriented, industry-informed curricula, ensuring that students are equipped with the skills necessary to become competent digital innovators in a rapidly changing digital economy.

Alexander, B., Ashford-Rowe, K., Barajas-Murph, N., Dobbin, G., Knott, J., McCormack, M., Pomerantz, J., Seilhamer, R. and Weber, N., 2019. EDUCAUSE Horizon Report 2019 Higher Education Edition (pp. 3–41). EDU19. Retrieved 20 October 2021 from https://www.learntechlib.org/p/208644/

Anderson, A. (2014). Business statistics for dummies . Wiley and Sons.

Google Scholar  

Bogers, M. L. A. M., Garud, R., Thomas, L. D. W., & Yoo, Y. (2022). Digital innovation: Transforming research and practice. Innovation, 24 (1), 4–12. https://doi.org/10.1080/14479338.2021.2005465

Article   Google Scholar  

Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods research (2nd ed.). Sage Publications.

Djankov, S., Saliola, F., Chen, R., Connon, D., & Cusolito, A. (2019). World Development Report 2019: The Changing Nature of Work: Main Report . World Bank Group.

Fichman, R. G., Dos Santos, B. L., & Zheng, Z. (Eric). (2014). Digital Innovation As a Fundamental and Powerful Concept in the Information Systems Curriculum. MIS Quarterly , 38(2), 329-A15. https://doi.org/10.25300/MISQ/2014/38.2.0

George, D. (2015). Student success kit guide. Retrieved September 7, 2015, from studylib.net/doc/9717312/student-success-kit-student-guide

Goulart, V. G., & Liboni, L. B. (2022). Balancing skills in the digital transformation era: The future of jobs and the role of higher education. Industry and Higher Education, 36 (2), 118–127. https://doi.org/10.1177/09504222211029796

Herrington, J. & Reeves, T.C. 2011. Using design principles to improve pedagogical practice and promote student engagement. ASCILITE 2011— The Australasian Society for Computers in Learning in Tertiary Education . 594–601.

Herrington, J., Reeves, T. C., & Oliver, R. (2010). A guide to authentic e-learning . Routledge.

Hevner, A., & Chatterjee, S. (2010). Design research in information systems: theory and practice (Vol. 22). Boston: Springer Science & Business Media.

Hevner, A., von Brocke, J., & Alexander, M. (2019). Roles of digital innovation in design science research. Business & Information Systems Engineering, 61 (1), 3–8. https://doi.org/10.1007/s12599-018-0571-z

Kereluik, K., Mishra, P., Fahnoe, C., & Terry, L. (2013). What knowledge is of most worth? Journal of Digital Learning in Teacher Education, 29 (4), 127–140. https://doi.org/10.1080/21532974.2013.10784716

McKenney, S., & Reeves, T. C. (2019). Conducting educational design research (2nd ed.). New York: Routledge

McKenney, S. E., & Reeves, T. C. (2021). Educational design research: Portraying, conducting, and enhancing scholarship that serves both theory and practice. Medical Education, 55 (1), 82–92. https://doi.org/10.1111/medu.14280

Miles, M., & Huberman, A. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Sage.

Nambisan, S., Lyytinen, K., Majchrzak, A., & Song, M. (2017). Digital innovation management: Reinventing innovation management research in a digital world. MIS Quarterly, 14 (1), 223–238. https://doi.org/10.25300/MISQ/2017/41:1.03

Reeves, T. C. (2006). Design research from the technology perspective. In J. V. Akker, K. Gravemeijer, S. E. McKenney, & N. Nieveen (Eds.), Educational design research (pp. 86–109). Routledge.

Van den Berg, C. (2019). Teaching innovation to strengthen knowledge creation in a digital world. Electronic Journal of Knowledge Management, 17 (2), 144–157.

Van den Berg, C., & Verster, B. (2022). Exploring Interdisciplinary Collaborative Online Learning Spaces through Sociomateriality in Cape Town, South Africa. The Electronic Journal of e-Learning, 20 (4), 467–482. https://doi.org/10.34190/ejel.20.4.2359

Download references

Open access funding provided by University of the Western Cape. The study was supported by funding from the National Research Foundation in South Africa. Grant: Support for Completing PhD Studies Part-time 2017 APDS160607168920A.

Author information

Authors and affiliations.

Department of Information Systems, University of the Western Cape, Cape Town, South Africa

Carolien van den Berg

Women’s and Gender Studies, University of the Western Cape, Cape Town, South Africa

Vivienne Bozalek

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Carolien van den Berg .

Ethics declarations

Conflict of interest.

No potential conflict of interest was reported by the authors.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

van den Berg, C., Bozalek, V. Design principles to develop digital innovation skills: a design-based research approach. Education Tech Research Dev (2023). https://doi.org/10.1007/s11423-023-10308-y

Download citation

Accepted : 10 October 2023

Published : 10 November 2023

DOI : https://doi.org/10.1007/s11423-023-10308-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Digital innovation
• Authentic learning
• Future work
• Design principles
• Find a journal
• Publish with us
• Track your research