design based research process model

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• CBE Life Sci Educ
• v.19(3); Fall 2020

Design-Based Research: A Methodology to Extend and Enrich Biology Education Research

Emily e. scott.

† Department of Biology, University of Washington, Seattle, WA 98195

Mary Pat Wenderoth

Jennifer h. doherty.

Recent calls in biology education research (BER) have recommended that researchers leverage learning theories and methodologies from other disciplines to investigate the mechanisms by which students to develop sophisticated ideas. We suggest design-based research from the learning sciences is a compelling methodology for achieving this aim. Design-based research investigates the “learning ecologies” that move student thinking toward mastery. These “learning ecologies” are grounded in theories of learning, produce measurable changes in student learning, generate design principles that guide the development of instructional tools, and are enacted using extended, iterative teaching experiments. In this essay, we introduce readers to the key elements of design-based research, using our own research into student learning in undergraduate physiology as an example of design-based research in BER. Then, we discuss how design-based research can extend work already done in BER and foster interdisciplinary collaborations among cognitive and learning scientists, biology education researchers, and instructors. We also explore some of the challenges associated with this methodological approach.

INTRODUCTION

There have been recent calls for biology education researchers to look toward other fields of educational inquiry for theories and methodologies to advance, and expand, our understanding of what helps students learn to think like biologists ( Coley and Tanner, 2012 ; Dolan, 2015 ; Peffer and Renken, 2016 ; Lo et al. , 2019 ). These calls include the recommendations that biology education researchers ground their work in learning theories from the cognitive and learning sciences ( Coley and Tanner, 2012 ) and begin investigating the underlying mechanisms by which students to develop sophisticated biology ideas ( Dolan, 2015 ; Lo et al. , 2019 ). Design-based research from the learning sciences is one methodology that seeks to do both by using theories of learning to investigate how “learning ecologies”—that is, complex systems of interactions among instructors, students, and environmental components—support the process of student learning ( Brown, 1992 ; Cobb et al. , 2003 ; Collins et al. , 2004 ; Peffer and Renken, 2016 ).

The purpose of this essay is twofold. First, we want to introduce readers to the key elements of design-based research, using our research into student learning in undergraduate physiology as an example of design-based research in biology education research (BER). Second, we will discuss how design-based research can extend work already done in BER and explore some of the challenges of its implementation. For a more in-depth review of design-based research, we direct readers to the following references: Brown (1992) , Barab and Squire (2004) , and Collins et al. (2004) , as well as commentaries by Anderson and Shattuck (2012) and McKenney and Reeves (2013) .

WHAT IS DESIGN-BASED RESEARCH?

Design-based research is a methodological approach that aligns with research methods from the fields of engineering or applied physics, where products are designed for specific purposes ( Brown, 1992 ; Joseph, 2004 ; Middleton et al. , 2008 ; Kelly, 2014 ). Consequently, investigators using design-based research approach educational inquiry much as an engineer develops a new product: First, the researchers identify a problem that needs to be addressed (e.g., a particular learning challenge that students face). Next, they design a potential “solution” to the problem in the form of instructional tools (e.g., reasoning strategies, worksheets; e.g., Reiser et al. , 2001 ) that theory and previous research suggest will address the problem. Then, the researchers test the instructional tools in a real-world setting (i.e., the classroom) to see if the tools positively impact student learning. As testing proceeds, researchers evaluate the instructional tools with emerging evidence of their effectiveness (or lack thereof) and progressively revise the tools— in real time —as necessary ( Collins et al. , 2004 ). Finally, the researchers reflect on the outcomes of the experiment, identifying the features of the instructional tools that were successful at addressing the initial learning problem, revising those aspects that were not helpful to learning, and determining how the research informed the theory underlying the experiment. This leads to another research cycle of designing, testing, evaluating, and reflecting to refine the instructional tools in support of student learning. We have characterized this iterative process in Figure 1 after Sandoval (2014) . Though we have portrayed four discrete phases to design-based research, there is often overlap of the phases as the research progresses (e.g., testing and evaluating can occur simultaneously).

The four phases of design-based research experienced in an iterative cycle (A). We also highlight the main features of each phase of our design-based research project investigating students’ use of flux in physiology (B).

Design-based research has no specific requirements for the form that instructional tools must take or the manner in which the tools are evaluated ( Bell, 2004 ; Anderson and Shattuck, 2012 ). Instead, design-based research has what Sandoval (2014) calls “epistemic commitments” 1 that inform the major goals of a design-based research project as well as how it is implemented. These epistemic commitments are: 1) Design based research should be grounded in theories of learning (e.g., constructivism, knowledge-in-pieces, conceptual change) that both inform the design of the instructional tools and are improved upon by the research ( Cobb et al. , 2003 ; Barab and Squire, 2004 ). This makes design-based research more than a method for testing whether or not an instructional tool works; it also investigates why the design worked and how it can be generalized to other learning environments ( Cobb et al. , 2003 ). 2) Design-based research should aim to produce measurable changes in student learning in classrooms around a particular learning problem ( Anderson and Shattuck, 2012 ; McKenney and Reeves, 2013 ). This requirement ensures that theoretical research into student learning is directly applicable, and impactful, to students and instructors in classroom settings ( Hoadley, 2004 ). 3) Design-based research should generate design principles that guide the development and implementation of future instructional tools ( Edelson, 2002 ). This commitment makes the research findings broadly applicable for use in a variety of classroom environments. 4) Design-based research should be enacted using extended, iterative teaching experiments in classrooms. By observing student learning over an extended period of time (e.g., throughout an entire term or across terms), researchers are more likely to observe the full effects of how the instructional tools impact student learning compared with short-term experiments ( Brown, 1992 ; Barab and Squire, 2004 ; Sandoval and Bell, 2004 ).

HOW IS DESIGN-BASED RESEARCH DIFFERENT FROM AN EXPERIMENTAL APPROACH?

Many BER studies employ experimental approaches that align with traditional scientific methods of experimentation, such as using treatment versus control groups, randomly assigning treatments to different groups, replicating interventions across multiple spatial or temporal periods, and using statistical methods to guide the kinds of inferences that arise from an experiment. While design-based research can similarly employ these strategies for educational inquiry, there are also some notable differences in its approach to experimentation ( Collins et al. , 2004 ; Hoadley, 2004 ). In this section, we contrast the differences between design-based research and what we call “experimental approaches,” although both paradigms represent a form of experimentation.

The first difference between an experimental approach and design-based research regards the role participants play in the experiment. In an experimental approach, the researcher is responsible for making all the decisions about how the experiment will be implemented and analyzed, while the instructor facilitates the experimental treatments. In design-based research, both researchers and instructors are engaged in all stages of the research from conception to reflection ( Collins et al. , 2004 ). In BER, a third condition frequently arises wherein the researcher is also the instructor. In this case, if the research questions being investigated produce generalizable results that have the potential to impact teaching broadly, then this is consistent with a design-based research approach ( Cobb et al. , 2003 ). However, when the research questions are self-reflective about how a researcher/instructor can improve his or her own classroom practices, this aligns more closely with “action research,” which is another methodology used in education research (see Stringer, 2013 ).

A second difference between experimental research and design-based research is the form that hypotheses take and the manner in which they are investigated ( Collins et al. , 2004 ; Sandoval, 2014 ). In experimental approaches, researchers develop a hypothesis about how a specific instructional intervention will impact student learning. The intervention is then tested in the classroom(s) while controlling for other variables that are not part of the study in order to isolate the effects of the intervention. Sometimes, researchers designate a “control” situation that serves as a comparison group that does not experience the intervention. For example, Jackson et al. (2018) were interested in comparing peer- and self-grading of weekly practice exams to if they were equally effective forms of deliberate practice for students in a large-enrollment class. To test this, the authors (including authors of this essay J.H.D., M.P.W.) designed an experiment in which lab sections of students in a large lecture course were randomly assigned to either a peer-grading or self-grading treatment so they could isolate the effects of each intervention. In design-based research, a hypothesis is conceptualized as the “design solution” rather than a specific intervention; that is, design-based researchers hypothesize that the designed instructional tools, when implemented in the classroom, will create a learning ecology that improves student learning around the identified learning problem ( Edelson, 2002 ; Bell, 2004 ). For example, Zagallo et al. (2016) developed a laboratory curriculum (i.e., the hypothesized “design solution”) for molecular and cellular biology majors to address the learning problem that students often struggle to connect scientific models and empirical data. This curriculum entailed: focusing instruction around a set of target biological models; developing small-group activities in which students interacted with the models by analyzing data from scientific papers; using formative assessment tools for student feedback; and providing students with a set of learning objectives they could use as study tools. They tested their curriculum in a novel, large-enrollment course of upper-division students over several years, making iterative changes to the curriculum as the study progressed.

By framing the research approach as an iterative endeavor of progressive refinement rather than a test of a particular intervention when all other variables are controlled, design-based researchers recognize that: 1) classrooms, and classroom experiences, are unique at any given time, making it difficult to truly “control” the environment in which an intervention occurs or establish a “control group” that differs only in the features of an intervention; and 2) many aspects of a classroom experience may influence the effectiveness of an intervention, often in unanticipated ways, which should be included in the research team’s analysis of an intervention’s success. Consequently, the research team is less concerned with controlling the research conditions—as in an experimental approach—and instead focuses on characterizing the learning environment ( Barab and Squire, 2004 ). This involves collecting data from multiple sources as the research progresses, including how the instructional tools were implemented, aspects of the implementation process that failed to go as planned, and how the instructional tools or implementation process was modified. These characterizations can provide important insights into what specific features of the instructional tools, or the learning environment, were most impactful to learning ( DBR Collective, 2003 ).

A third difference between experimental approaches and design-based research is when the instructional interventions can be modified. In experimental research, the intervention is fixed throughout the experimental period, with any revisions occurring only after the experiment has concluded. This is critical for ensuring that the results of the study provide evidence of the efficacy of a specific intervention. By contrast, design-based research takes a more flexible approach that allows instructional tools to be modified in situ as they are being implemented ( Hoadley, 2004 ; Barab, 2014 ). This flexibility allows the research team to modify instructional tools or strategies that prove inadequate for collecting the evidence necessary to evaluate the underlying theory and ensures a tight connection between interventions and a specific learning problem ( Collins et al. , 2004 ; Hoadley, 2004 ).

Finally, and importantly, experimental approaches and design-based research differ in the kinds of conclusions they draw from their data. Experimental research can “identify that something meaningful happened; but [it is] not able to articulate what about the intervention caused that story to unfold” ( Barab, 2014 , p. 162). In other words, experimental methods are robust for identifying where differences in learning occur, such as between groups of students experiencing peer- or self-grading of practice exams ( Jackson et al. , 2018 ) or receiving different curricula (e.g., Chi et al. , 2012 ). However, these methods are not able to characterize the underlying learning process or mechanism involved in the different learning outcomes. By contrast, design-based research has the potential to uncover mechanisms of learning, because it investigates how the nature of student thinking changes as students experience instructional interventions ( Shavelson et al. , 2003 ; Barab, 2014 ). According to Sandoval (2014) , “Design research, as a means of uncovering causal processes, is oriented not to finding effects but to finding functions , to understanding how desired (and undesired) effects arise through interactions in a designed environment” (p. 30). In Zagallo et al. (2016) , the authors found that their curriculum supported students’ data-interpretation skills, because it stimulated students’ spontaneous use of argumentation during which group members coconstructed evidence-based claims from the data provided. Students also worked collaboratively to decode figures and identify data patterns. These strategies were identified from the researchers’ qualitative data analysis of in-class recordings of small-group discussions, which allowed them to observe what students were doing to support their learning. Because design-based research is focused on characterizing how learning occurs in classrooms, it can begin to answer the kinds of mechanistic questions others have identified as central to advancing BER ( National Research Council [NRC], 2012 ; Dolan, 2015 ; Lo et al. , 2019 ).

DESIGN-BASED RESEARCH IN ACTION: AN EXAMPLE FROM UNDERGRADUATE PHYSIOLOGY

To illustrate how design-based research could be employed in BER, we draw on our own research that investigates how students learn physiology. We will characterize one iteration of our design-based research cycle ( Figure 1 ), emphasizing how our project uses Sandoval’s four epistemic commitments (i.e., theory driven, practically applied, generating design principles, implemented in an iterative manner) to guide our implementation.

Identifying the Learning Problem

Understanding physiological phenomena is challenging for students, given the wide variety of contexts (e.g., cardiovascular, neuromuscular, respiratory; animal vs. plant) and scales involved (e.g., using molecular-level interactions to explain organism functioning; Wang, 2004 ; Michael, 2007 ; Badenhorst et al. , 2016 ). To address these learning challenges, Modell (2000) identified seven “general models” that undergird most physiology phenomena (i.e., control systems, conservation of mass, mass and heat flow, elastic properties of tissues, transport across membranes, cell-to-cell communication, molecular interactions). Instructors can use these models as a “conceptual framework” to help students build intellectual coherence across phenomena and develop a deeper understanding of physiology ( Modell, 2000 ; Michael et al. , 2009 ). This approach aligns with theoretical work in the learning sciences that indicates that providing students with conceptual frameworks improves their ability to integrate and retrieve knowledge ( National Academies of Sciences, Engineering, and Medicine, 2018 ).

Before the start of our design-based project, we had been using Modell’s (2000) general models to guide our instruction. In this essay, we will focus on how we used the general models of mass and heat flow and transport across membranes in our instruction. These two models together describe how materials flow down gradients (e.g., pressure gradients, electrochemical gradients) against sources of resistance (e.g., tube diameter, channel frequency). We call this flux reasoning. We emphasized the fundamental nature and broad utility of flux reasoning in lecture and lab and frequently highlighted when it could be applied to explain a phenomenon. We also developed a conceptual scaffold (the Flux Reasoning Tool) that students could use to reason about physiological processes involving flux.

Although these instructional approaches had improved students’ understanding of flux phenomena, we found that students often demonstrated little commitment to using flux broadly across physiological contexts. Instead, they considered flux to be just another fact to memorize and applied it to narrow circumstances (e.g., they would use flux to reason about ions flowing across membranes—the context where flux was first introduced—but not the bulk flow of blood in a vessel). Students also struggled to integrate the various components of flux (e.g., balancing chemical and electrical gradients, accounting for variable resistance). We saw these issues reflected in students’ lower than hoped for exam scores on the cumulative final of the course. From these experiences, and from conversations with other physiology instructors, we identified a learning problem to address through design-based research: How do students learn to use flux reasoning to explain material flows in multiple physiology contexts?

The process of identifying a learning problem usually emerges from a researcher’s own experiences (in or outside a classroom) or from previous research that has been described in the literature ( Cobb et al. , 2003 ). To remain true to Sandoval’s first epistemic commitment, a learning problem must advance a theory of learning ( Edelson, 2002 ; McKenney and Reeves, 2013 ). In our work, we investigated how conceptual frameworks based on fundamental scientific concepts (i.e., Modell’s general models) could help students reason productively about physiology phenomena (National Academies of Sciences, Engineering, and Medicine, 2018; Modell, 2000 ). Our specific theoretical question was: Can we characterize how students’ conceptual frameworks around flux change as they work toward robust ideas? Sandoval’s second epistemic commitment stated that a learning problem must aim to improve student learning outcomes. The practical significance of our learning problem was: Does using the concept of flux as a foundational idea for instructional tools increase students’ learning of physiological phenomena?

We investigated our learning problem in an introductory biology course at a large R1 institution. The introductory course is the third in a biology sequence that focuses on plant and animal physiology. The course typically serves between 250 and 600 students in their sophomore or junior years each term. Classes have the following average demographics: 68% male, 21% from lower-income situations, 12% from an underrepresented minority, and 26% first-generation college students.

Design-Based Research Cycle 1, Phase 1: Designing Instructional Tools

The first phase of design-based research involves developing instructional tools that address both the theoretical and practical concerns of the learning problem ( Edelson, 2002 ; Wang and Hannafin, 2005 ). These instructional tools can take many forms, such as specific instructional strategies, classroom worksheets and practices, or technological software, as long as they embody the underlying learning theory being investigated. They must also produce classroom experiences or materials that can be evaluated to determine whether learning outcomes were met ( Sandoval, 2014 ). Indeed, this alignment between theory, the nature of the instructional tools, and the ways students are assessed is central to ensuring rigorous design-based research ( Hoadley, 2004 ; Sandoval, 2014 ). Taken together, the instructional tools instantiate a hypothesized learning environment that will advance both the theoretical and practical questions driving the research ( Barab, 2014 ).

In our work, the theoretical claim that instruction based on fundamental scientific concepts would support students’ flux reasoning was embodied in our instructional approach by being the central focus of all instructional materials, which included: a revised version of the Flux Reasoning Tool ( Figure 2 ); case study–based units in lecture that explicitly emphasized flux phenomena in real-world contexts ( Windschitl et al. , 2012 ; Scott et al. , 2018 ; Figure 3 ); classroom activities in which students practiced using flux to address physiological scenarios; links to online videos describing key flux-related concepts; constructed-response assessment items that cued students to use flux reasoning in their thinking; and pretest/posttest formative assessment questions that tracked student learning ( Figure 4 ).

The Flux Reasoning Tool given to students at the beginning of the quarter.

An example flux case study that is presented to students at the beginning of the neurophysiology unit. Throughout the unit, students learn how ion flows into and out of cells, as mediated by chemical and electrical gradients and various ion/molecular channels, sends signals throughout the body. They use this information to better understand why Jaime experiences persistent neuropathy. Images from: uz.wikipedia.org/wiki/Fayl:Blausen_0822_SpinalCord.png and commons.wikimedia.org/wiki/File:Figure_38_01_07.jpg.

An example flux assessment question about ion flows given in a pre-unit/post-unit formative assessment in the neurophysiology unit.

Phase 2: Testing the Instructional Tools

In the second phase of design-based research, the instructional tools are tested by implementing them in classrooms. During this phase, the instructional tools are placed “in harm’s way … in order to expose the details of the process to scrutiny” ( Cobb et al. , 2003 , p. 10). In this way, researchers and instructors test how the tools perform in real-world settings, which may differ considerably from the design team’s initial expectations ( Hoadley, 2004 ). During this phase, if necessary, the design team may make adjustments to the tools as they are being used to account for these unanticipated conditions ( Collins et al. , 2004 ).

We implemented the instructional tools during the Autumn and Spring quarters of the 2016–2017 academic year. Students were taught to use the Flux Reasoning Tool at the beginning of the term in the context of the first case study unit focused on neurophysiology. Each physiology unit throughout the term was associated with a new concept-based case study (usually about flux) that framed the context of the teaching. Embedded within the daily lectures were classroom activities in which students could practice using flux. Students were also assigned readings from the textbook and videos related to flux to watch during each unit. Throughout the term, students took five exams that each contained some flux questions as well as some pre- and post-unit formative assessment questions. During Winter quarter, we conducted clinical interviews with students who would take our course in the Spring term (i.e., “pre” data) as well as students who had just completed our course in Autumn (i.e., “post” data).

Phase 3: Evaluating the Instructional Tools

The third phase of a design-based research cycle involves evaluating the effectiveness of instructional tools using evidence of student learning ( Barab and Squire, 2004 ; Anderson and Shattuck, 2012 ). This can be done using products produced by students (e.g., homework, lab reports), attitudinal gains measured with surveys, participation rates in activities, interview testimonials, classroom discourse practices, and formative assessment or exam data (e.g., Reiser et al. , 2001 ; Cobb et al. , 2003 ; Barab and Squire, 2004 ; Mohan et al. , 2009 ). Regardless of the source, evidence must be in a form that supports a systematic analysis that could be scrutinized by other researchers ( Cobb et al. , 2003 ; Barab, 2014 ). Also, because design-based research often involves multiple data streams, researchers may need to use both quantitative and qualitative analytical methods to produce a rich picture of how the instructional tools affected student learning ( Collins et al. , 2004 ; Anderson and Shattuck, 2012 ).

In our work, we used the quality of students’ written responses on exams and formative assessment questions to determine whether students improved their understanding of physiological phenomena involving flux. For each assessment question, we analyzed a subset of student’s pretest answers to identify overarching patterns in students’ reasoning about flux, characterized these overarching patterns, then ordinated the patterns into different levels of sophistication. These became our scoring rubrics, which identified five different levels of student reasoning about flux. We used the rubrics to code the remainder of students’ responses, with a code designating the level of student reasoning associated with a particular reasoning pattern. We used this ordinal rubric format because it would later inform our theoretical understanding of how students build flux conceptual frameworks (see phase 4). This also allowed us to both characterize the ideas students held about flux phenomena and identify the frequency distribution of those ideas in a class.

By analyzing changes in the frequency distributions of students’ ideas across the rubric levels at different time points in the term (e.g., pre-unit vs. post-unit), we could track both the number of students who gained more sophisticated ideas about flux as the term progressed and the quality of those ideas. If the frequency of students reasoning at higher levels increased from pre-unit to post-unit assessments, we could conclude that our instructional tools as a whole were supporting students’ development of sophisticated flux ideas. For example, on one neuromuscular ion flux assessment question in the Spring of 2017, we found that relatively more students were reasoning at the highest levels of our rubric (i.e., levels 4 and 5) on the post-unit test compared with the pre-unit test. This meant that more students were beginning to integrate sophisticated ideas about flux (i.e., they were balancing concentration and electrical gradients) in their reasoning about ion movement.

To help validate this finding, we drew on three additional data streams: 1) from in-class group recordings of students working with flux items, we noted that students increasingly incorporated ideas about gradients and resistance when constructing their explanations as the term progressed; 2) from plant assessment items in the latter part of the term, we began to see students using flux ideas unprompted; and 3) from interviews, we observed that students who had already taken the course used flux ideas in their reasoning.

Through these analyses, we also noticed an interesting pattern in the pre-unit test data for Spring 2017 when compared with the frequency distribution of students’ responses with a previous term (Autumn 2016). In Spring 2017, 42% of students reasoned at level 4 or 5 on the pre-unit test, indicating these students already had sophisticated ideas about ion flux before they took the pre-unit assessment. This was surprising, considering only 2% of students reasoned at these levels for this item on the Autumn 2016 pre-unit test.

Phase 4: Reflecting on the Instructional Tools and Their Implementation

The final phase of a design-based research cycle involves a retrospective analysis that addresses the epistemic commitments of this methodology: How was the theory underpinning the research advanced by the research endeavor (theoretical outcome)? Did the instructional tools support student learning about the learning problem (practical outcome)? What were the critical features of the design solution that supported student learning (design principles)? ( Cobb et al. , 2003 ; Barab and Squire, 2004 ).

Theoretical Outcome (Epistemic Commitment 1).

Reflecting on how a design-based research experiment advances theory is critical to our understanding of how students learn in educational settings ( Barab and Squire, 2004 ; Mohan et al. , 2009 ). In our work, we aimed to characterize how students’ conceptual frameworks around flux change as they work toward robust ideas. To do this, we drew on learning progression research as our theoretical framing ( NRC, 2007 ; Corcoran et al. , 2009 ; Duschl et al. , 2011 ; Scott et al. , 2019 ). Learning progression frameworks describe empirically derived patterns in student thinking that are ordered into levels representing cognitive shifts in the ways students conceive a topic as they work toward mastery ( Gunckel et al. , 2012 ). We used our ion flux scoring rubrics to create a preliminary five-level learning progression framework ( Table 1 ). The framework describes how students’ ideas about flux often start with teleological-driven accounts at the lowest level (i.e., level 1), shift to focusing on driving forces (e.g., concentration gradients, electrical gradients) in the middle levels, and arrive at complex ideas that integrate multiple interacting forces at the higher levels. We further validated these reasoning patterns with our student interviews. However, our flux conceptual framework was largely based on student responses to our ion flux assessment items. Therefore, to further validate our learning progression framework, we needed a greater diversity of flux assessment items that investigated student thinking more broadly (i.e., about bulk flow, water movement) across physiological systems.

The preliminary flux learning progression framework characterizing the patterns of reasoning students may exhibit as they work toward mastery of flux reasoning. The student exemplars are from the ion flux formative assessment question presented in Figure 4 . The “/” divides a student’s answers to the first and second parts of the question. Level 5 represents the most sophisticated ideas about flux phenomena.

Practical Outcome (Epistemic Commitment 2).

In design-based research, learning theories must “do real work” by improving student learning in real-world settings ( DBR Collective, 2003 ). Therefore, design-based researchers must reflect on whether or not the data they collected show evidence that the instructional tools improved student learning ( Cobb et al. , 2003 ; Sharma and McShane, 2008 ). We determined whether our flux-based instructional approach aided student learning by analyzing the kinds of answers students provided to our assessment questions. Specifically, we considered students who reasoned at level 4 or above as demonstrating productive flux reasoning. Because almost half of students were reasoning at level 4 or 5 on the post-unit assessment after experiencing the instructional tools in the neurophysiology unit (in Spring 2017), we concluded that our tools supported student learning in physiology. Additionally, we noticed that students used language in their explanations that directly tied to the Flux Reasoning Tool ( Figure 2 ), which instructed them to use arrows to indicate the magnitude and direction of gradient-driving forces. For example, in a posttest response to our ion flux item ( Figure 4 ), one student wrote:

Ion movement is a function of concentration and electrical gradients . Which arrow is stronger determines the movement of K+. We can make the electrical arrow bigger and pointing in by making the membrane potential more negative than Ek [i.e., potassium’s equilibrium potential]. We can make the concentration arrow bigger and pointing in by making a very strong concentration gradient pointing in.

Given that almost half of students reasoned at level 4 or above, and that students used language from the Flux Reasoning Tool, we concluded that using fundamental concepts was a productive instructional approach for improving student learning in physiology and that our instructional tools aided student learning. However, some students in the 2016–2017 academic year continued to apply flux ideas more narrowly than intended (i.e., for ion and simple diffusion cases, but not water flux or bulk flow). This suggested that students had developed nascent flux conceptual frameworks after experiencing the instructional tools but could use more support to realize the broad applicability of this principle. Also, although our cross-sectional interview approach demonstrated how students’ ideas, overall, could change after experiencing the instructional tools, it did not provide information about how a student developed flux reasoning.

Reflecting on practical outcomes also means interpreting any learning gains in the context of the learning ecology. This reflection allowed us to identify whether there were particular aspects of the instructional tools that were better at supporting learning than others ( DBR Collective, 2003 ). Indeed, this was critical for our understanding why 42% of students scored at level 3 and above on the pre-unit ion assessment in the Spring of 2017, while only 2% of students scored level 3 and above in Autumn of 2016. When we reviewed notes of the Spring 2017 implementation scheme, we saw that the pretest was due at the end of the first day of class after students had been exposed to ion flux ideas in class and in a reading/video assignment about ion flow, which may be one reason for the students’ high performance on the pretest. Consequently, we could not tell whether students’ initial high performance was due to their learning from the activities in the first day of class or for other reasons we did not measure. It also indicated we needed to close pretests before the first day of class for a more accurate measure of students’ incoming ideas and the effectiveness of the instructional tools employed at the beginning of the unit.

Design Principles (Epistemic Commitment 3).

Although design-based research is enacted in local contexts (i.e., a particular classroom), its purpose is to inform learning ecologies that have broad applications to improve learning and teaching ( Edelson, 2002 ; Cobb et al. , 2003 ). Therefore, design-based research should produce design principles that describe characteristics of learning environments that researchers and instructors can use to develop instructional tools specific to their local contexts (e.g., Edelson, 2002 ; Subramaniam et al. , 2015 ). Consequently, the design principles must balance specificity with adaptability so they can be used broadly to inform instruction ( Collins et al. , 2004 ; Barab, 2014 ).

From our first cycle of design-based research, we developed the following design principles: 1) Key scientific concepts should provide an overarching framework for course organization. This way, the individual components that make up a course, like instructional units, activities, practice problems, and assessments, all reinforce the centrality of the key concept. 2) Instructional tools should explicitly articulate the principle of interest, with specific guidance on how that principle is applied in context. This stresses the applied nature of the principle and that it is more than a fact to be memorized. 3) Instructional tools need to show specific instances of how the principle is applied in multiple contexts to combat students’ narrow application of the principle to a limited number of contexts.

Design-Based Research Cycle 2, Phase 1: Redesign and Refine the Experiment

The last “epistemic commitment” Sandoval (2014) articulated was that design-based research be an iterative process with an eye toward continually refining the instructional tools, based on evidence of student learning, to produce more robust learning environments. By viewing educational inquiry as formative research, design-based researchers recognize the difficulty in accounting for all variables that could impact student learning, or the implementation of the instructional tools, a priori ( Collins et al. , 2004 ). Robust instructional designs are the products of trial and error, which are strengthened by a systematic analysis of how they perform in real-world settings.

To continue to advance our work investigating student thinking using the principle of flux, we began a second cycle of design-based research that continued to address the learning problem of helping students reason with fundamental scientific concepts. In this cycle, we largely focused on broadening the number of physiological systems that had accompanying formative assessment questions (i.e., beyond ion flux), collecting student reasoning from a more diverse population of students (e.g., upper division, allied heath, community college), and refining and validating the flux learning progression with both written and interview data in a student through time. We developed a suite of constructed-response flux assessment questions that spanned neuromuscular, cardiovascular, respiratory, renal, and plant physiological contexts and asked students about several kinds of flux: ion movement, diffusion, water movement, and bulk flow (29 total questions; available at beyondmultiplechoice.org). This would provide us with rich qualitative data that we could use to refine the learning progression. We decided to administer written assessments and conduct interviews in a pretest/posttest manner at the beginning and end of each unit both as a way to increase our data about student reasoning and to provide students with additional practice using flux reasoning across contexts.

From this second round of designing instructional tools (i.e., broader range of assessment items), testing them in the classroom (i.e., administering the assessment items to diverse student populations), evaluating the tools (i.e., developing learning progression–aligned rubrics across phenomena from student data, tracking changes in the frequency distribution of students across levels through time), and reflecting on the tools’ success, we would develop a more thorough and robust characterization of how students use flux across systems that could better inform our creation of new instructional tools to support student learning.

HOW CAN DESIGN-BASED RESEARCH EXTEND AND ENRICH BER?

While design-based research has primarily been used in educational inquiry at the K–12 level (see Reiser et al. , 2001 ; Mohan et al. , 2009 ; Jin and Anderson, 2012 ), other science disciplines at undergraduate institutions have begun to employ this methodology to create robust instructional approaches (e.g., Szteinberg et al. , 2014 in chemistry; Hake, 2007 , and Sharma and McShane, 2008 , in physics; Kelly, 2014 , in engineering). Our own work, as well as that by Zagallo et al. (2016) , provides two examples of how design-based research could be implemented in BER. Below, we articulate some of the ways incorporating design-based research into BER could extend and enrich this field of educational inquiry.

Design-Based Research Connects Theory with Practice

One critique of BER is that it does not draw heavily enough on learning theories from other disciplines like cognitive psychology or the learning sciences to inform its research ( Coley and Tanner, 2012 ; Dolan, 2015 ; Peffer and Renken, 2016 ; Davidesco and Milne, 2019 ). For example, there has been considerable work in BER developing concept inventories as formative assessment tools that identify concepts students often struggle to learn (e.g., Marbach-Ad et al. , 2009 ; McFarland et al. , 2017 ; Summers et al. , 2018 ). However, much of this work is detached from a theoretical understanding of why students hold misconceptions in the first place, what the nature of their thinking is, and the learning mechanisms that would move students to a more productive understanding of domain ideas ( Alonzo, 2011 ). Using design-based research to understand the basis of students’ misconceptions would ground these practical learning problems in a theoretical understanding of the nature of student thinking (e.g., see Coley and Tanner, 2012 , 2015 ; Gouvea and Simon, 2018 ) and the kinds of instructional tools that would best support the learning process.

Design-Based Research Fosters Collaborations across Disciplines

Recently, there have been multiple calls across science, technology, engineering, and mathematics education fields to increase collaborations between BER and other disciplines so as to increase the robustness of science education research at the collegiate level ( Coley and Tanner, 2012 ; NRC, 2012 ; Talanquer, 2014 ; Dolan, 2015 ; Peffer and Renken, 2016 ; Mestre et al. , 2018 ; Davidesco and Milne, 2019 ). Engaging in design-based research provides both a mechanism and a motivation for fostering interdisciplinary collaborations, as it requires the design team to have theoretical knowledge of how students learn, domain knowledge of practical learning problems, and instructional knowledge for how to implement instructional tools in the classroom ( Edelson, 2002 ; Hoadley, 2004 ; Wang and Hannafin, 2005 ; Anderson and Shattuck, 2012 ). For example, in our current work, our research team consists of two discipline-based education learning scientists from an R1 institution, two physiology education researchers/instructors (one from an R1 institution the other from a community college), several physiology disciplinary experts/instructors, and a K–12 science education expert.

Design-based research collaborations have several distinct benefits for BER: first, learning or cognitive scientists could provide theoretical and methodological expertise that may be unfamiliar to biology education researchers with traditional science backgrounds ( Lo et al. , 2019 ). This would both improve the rigor of the research project and provide biology education researchers with the opportunity to explore ideas and methods from other disciplines. Second, collaborations between researchers and instructors could help increase the implementation of evidence-based teaching practices by instructors/faculty who are not education researchers and would benefit from support while shifting their instructional approaches ( Eddy et al. , 2015 ). This may be especially true for community college and primarily undergraduate institution faculty who often do not have access to the same kinds of resources that researchers and instructors at research-intensive institutions do ( Schinske et al. , 2017 ). Third, making instructors an integral part of a design-based research project ensures they are well versed in the theory and learning objectives underlying the instructional tools they are implementing in the classroom. This can improve the fidelity of implementation of the instructional tools, because the instructors understand the tools’ theoretical and practical purposes, which has been cited as one reason there have been mixed results on the impact of active learning across biology classes ( Andrews et al. , 2011 ; Borrego et al. , 2013 ; Lee et al. , 2018 ; Offerdahl et al. , 2018 ). It also gives instructors agency to make informed adjustments to the instructional tools during implementation that improve their practical applications while remaining true to the goals of the research ( Hoadley, 2004 ).

Design-Based Research Invites Using Mixed Methods to Analyze Data

The diverse nature of the data that are often collected in design-based research can require both qualitative and quantitative methodologies to produce a rich picture of how the instructional tools and their implementation influenced student learning ( Anderson and Shattuck, 2012 ). Using mixed methods may be less familiar to biology education researchers who were primarily trained in quantitative methods as biologists ( Lo et al. , 2019 ). However, according to Warfa (2016 , p. 2), “Integration of research findings from quantitative and qualitative inquiries in the same study or across studies maximizes the affordances of each approach and can provide better understanding of biology teaching and learning than either approach alone.” Although the number of BER studies using mixed methods has increased over the past decade ( Lo et al. , 2019 ), engaging in design-based research could further this trend through its collaborative nature of bringing social scientists together with biology education researchers to share research methodologies from different fields. By leveraging qualitative and quantitative methods, design-based researchers unpack “mechanism and process” by characterizing the nature of student thinking rather than “simply reporting that differences did or did not occur” ( Barab, 2014 , p. 158), which is important for continuing to advance our understanding of student learning in BER ( Dolan, 2015 ; Lo et al. , 2019 ).

CHALLENGES TO IMPLEMENTING DESIGN-BASED RESEARCH IN BER

As with any methodological approach, there can be challenges to implementing design-based research. Here, we highlight three that may be relevant to BER.

Collaborations Can Be Difficult to Maintain

While collaborations between researchers and instructors offer many affordances (as discussed earlier), the reality of connecting researchers across departments and institutions can be challenging. For example, Peffer and Renken (2016) noted that different traditions of scholarship can present barriers to collaboration where there is not mutual respect for the methods and ideas that are part and parcel to each discipline. Additionally, Schinske et al. (2017) identified several constraints that community college faculty face for engaging in BER, such as limited time or support (e.g., infrastructural, administrative, and peer support), which could also impact their ability to form the kinds of collaborations inherent in design-based research. Moreover, the iterative nature of design-based research requires these collaborations to persist for an extended period of time. Attending to these challenges is an important part of forming the design team and identifying the different roles researchers and instructors will play in the research.

Design-Based Research Experiments Are Resource Intensive

The focus of design-based research on studying learning ecologies to uncover mechanisms of learning requires that researchers collect multiple data streams through time, which often necessitates significant temporal and financial resources ( Collins et al., 2004 ; O’Donnell, 2004 ). Consequently, researchers must weigh both practical as well as methodological considerations when formulating their experimental design. For example, investigating learning mechanisms requires that researchers collect data at a frequency that will capture changes in student thinking ( Siegler, 2006 ). However, researchers may be constrained in the number of data-collection events they can anticipate depending on: the instructor’s ability to facilitate in-class collection events or solicit student participation in extracurricular activities (e.g., interviews); the cost of technological devices to record student conversations; the time and logistical considerations needed to schedule and conduct student interviews; the financial resources available to compensate student participants; the financial and temporal costs associated with analyzing large amounts of data.

Identifying learning mechanisms also requires in-depth analyses of qualitative data as students experience various instructional tools (e.g., microgenetic methods; Flynn et al. , 2006 ; Siegler, 2006 ). The high intensity of these in-depth analyses often limits the number of students who can be evaluated in this way, which must be balanced with the kinds of generalizations researchers wish to make about the effectiveness of the instructional tools ( O’Donnell, 2004 ). Because of the large variety of data streams that could be collected in a design-based research experiment—and the resources required to collect and analyze them—it is critical that the research team identify a priori how specific data streams, and the methods of their analysis, will provide the evidence necessary to address the theoretical and practical objectives of the research (see the following section on experimental rigor; Sandoval, 2014 ). These are critical management decisions because of the need for a transparent, systematic analysis of the data that others can scrutinize to evaluate the validity of the claims being made ( Cobb et al. , 2003 ).

Concerns with Experimental Rigor

The nature of design-based research, with its use of narrative to characterize versus control experimental environments, has drawn concerns about the rigor of this methodological approach. Some have challenged its ability to produce evidence-based warrants to support its claims of learning that can be replicated and critiqued by others ( Shavelson et al. , 2003 ; Hoadley, 2004 ). This is a valid concern that design-based researchers, and indeed all education researchers, must address to ensure their research meets established standards for education research ( NRC, 2002 ).

One way design-based researchers address this concern is by “specifying theoretically salient features of a learning environment design and mapping out how they are predicted to work together to produce desired outcomes” ( Sandoval, 2014 , p. 19). Through this process, researchers explicitly show before they begin the work how their theory of learning is embodied in the instructional tools to be tested, the specific data the tools will produce for analysis, and what outcomes will be taken as evidence for success. Moreover, by allowing instructional tools to be modified during the testing phase as needed, design-based researchers acknowledge that it is impossible to anticipate all aspects of the classroom environment that might impact the implementation of instructional tools, “as dozens (if not millions) of factors interact to produce the measureable outcomes related to learning” ( Hoadley, 2004 , p. 204; DBR Collective, 2003 ). Consequently, modifying instructional tools midstream to account for these unanticipated factors can ensure they retain their methodological alignment with the underlying theory and predicted learning outcomes so that inferences drawn from the design experiment accurately reflect what was being tested ( Edelson, 2002 ; Hoadley, 2004 ). Indeed, Barab (2014) states, “the messiness of real-world practice must be recognized, understood, and integrated as part of the theoretical claims if the claims are to have real-world explanatory value” (p. 153).

CONCLUSIONS

In this essay, we have highlighted some of the ways design-based research can advance—and expand upon—research done in biology education. These ways include:

• providing a methodology that integrates theories of learning with practical experiences in classrooms,
• using a range of analytical approaches that allow for researchers to uncover the underlying mechanisms of student thinking and learning,
• fostering interdisciplinary collaborations among researchers and instructors, and
• characterizing learning ecologies that account for the complexity involved in student learning

By employing this methodology from the learning sciences, biology education researchers can enrich our current understanding of what is required to help biology students achieve their personal and professional aims during their college experience. It can also stimulate new ideas for biology education that can be discussed and debated in our research community as we continue to explore and refine how best to serve the students who pass through our classroom doors.

Acknowledgments

We thank the UW Biology Education Research Group’s (BERG) feedback on drafts of this essay as well as Dr. L. Jescovich for last-minute analyses. This work was supported by a National Science Foundation award (NSF DUE 1661263/1660643). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF. All procedures were conducted in accordance with approval from the Institutional Review Board at the University of Washington (52146) and the New England Independent Review Board (120160152).

1 “Epistemic commitment” is defined as engaging in certain practices that generate knowledge in an agreed-upon way.

• Alonzo, A. C. (2011). Learning progressions that support formative assessment practices . Measurement , 9 ( 2/3 ), 124–129. [ Google Scholar ]
• Anderson, T., Shattuck, J. (2012). Design-based research: A decade of progress in education research? Educational Researcher , 41 ( 1 ), 16–25. [ Google Scholar ]
• Andrews, T. M., Leonard, M. J., Colgrove, C. A., Kalinowski, S. T. (2011). Active learning not associated with student learning in a random sample of college biology courses . CBE—Life Sciences Education , 10 ( 4 ), 394–405. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Badenhorst, E., Hartman, N., Mamede, S. (2016). How biomedical misconceptions may arise and affect medical students’ learning: A review of theoretical perspectives and empirical evidence . Health Professions Education , 2 ( 1 ), 10–17. [ Google Scholar ]
• Barab, S. (2014). Design-based research: A methodological toolkit for engineering change . In The Cambridge handbook of the learning sciences (2nd ed., pp. 151–170). Cambridge University Press. 10.1017/CBO9781139519526.011 [ CrossRef ] [ Google Scholar ]
• Barab, S., Squire, K. (2004). Design-based research: Putting a stake in the ground . Journal of the Learning Sciences , 13 ( 1 ), 1–14. [ Google Scholar ]
• Bell, P. (2004). On the theoretical breadth of design-based research in education . Educational Psychologist , 39 ( 4 ), 243–253. [ Google Scholar ]
• Borrego, M., Cutler, S., Prince, M., Henderson, C., Froyd, J. E. (2013). Fidelity of implementation of research-based instructional strategies (RBIS) in engineering science courses . Journal of Engineering Education , 102 ( 3 ), 394–425. [ Google Scholar ]
• Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings . Journal of the Learning Sciences , 2 ( 2 ), 141–178. [ Google Scholar ]
• Chi, M. T. H., Roscoe, R. D., Slotta, J. D., Roy, M., Chase, C. C. (2012). Misconceived causal explanations for emergent processes . Cognitive Science , 36 ( 1 ), 1–61. [ PubMed ] [ Google Scholar ]
• Cobb, P., Confrey, J., diSessa, A., Lehrer, R., Schauble, L. (2003). Design experiments in educational research . Educational Researcher , 32 ( 1 ), 9–13. [ Google Scholar ]
• Coley, J. D., Tanner, K. D. (2012). Common origins of diverse misconceptions: Cognitive principles and the development of biology thinking . CBE—Life Sciences Education , 11 ( 3 ), 209–215. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Coley, J. D., Tanner, K. (2015). Relations between intuitive biological thinking and biological misconceptions in biology majors and nonmajors . CBE—Life Sciences Education , 14 ( 1 ). 10.1187/cbe.14-06-0094 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Collins, A., Joseph, D., Bielaczyc, K. (2004). Design research: Theoretical and methodological issues . Journal of the Learning Sciences , 13 ( 1 ), 15–42. [ Google Scholar ]
• Corcoran, T., Mosher, F. A., Rogat, A. D. (2009). Learning progressions in science: An evidence-based approach to reform (CPRE Research Report No. RR-63) . Philadelphia, PA: Consortium for Policy Research in Education. [ Google Scholar ]
• Davidesco, I., Milne, C. (2019). Implementing cognitive science and discipline-based education research in the undergraduate science classroom . CBE—Life Sciences Education , 18 ( 3 ), es4. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry . Educational Researcher , 32 ( 1 ), 5–8. [ Google Scholar ]
• Dolan, E. L. (2015). Biology education research 2.0 . CBE—Life Sciences Education , 14 ( 4 ), ed1. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Duschl, R., Maeng, S., Sezen, A. (2011). Learning progressions and teaching sequences: A review and analysis . Studies in Science Education , 47 ( 2 ), 123–182. [ Google Scholar ]
• Eddy, S. L., Converse, M., Wenderoth, M. P. (2015). PORTAAL: A classroom observation tool assessing evidence-based teaching practices for active learning in large science, technology, engineering, and mathematics classes . CBE—Life Sciences Education , 14 ( 2 ), ar23. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Edelson, D. C. (2002). Design research: What we learn when we engage in design . Journal of the Learning Sciences , 11 ( 1 ), 105–121. [ Google Scholar ]
• Flynn, E., Pine, K., Lewis, C. (2006). The microgenetic method—Time for change? The Psychologist , 19 ( 3 ), 152–155. [ Google Scholar ]
• Gouvea, J. S., Simon, M. R. (2018). Challenging cognitive construals: A dynamic alternative to stable misconceptions . CBE—Life Sciences Education , 17 ( 2 ), ar34. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Gunckel, K. L., Mohan, L., Covitt, B. A., Anderson, C. W. (2012). Addressing challenges in developing learning progressions for environmental science literacy . In Alonzo, A. C., Gotwals, A. W. (Eds.), Learning progressions in science: Current challenges and future directions (pp. 39–75). Rotterdam: SensePublishers. 10.1007/978-94-6091-824-7_4 [ CrossRef ] [ Google Scholar ]
• Hake, R. R. (2007). Design-based research in physics education research: A review . In Kelly, A. E., Lesh, R. A., Baek, J. Y. (Eds.), Handbook of design research methods in mathematics, science, and technology education (p. 24). New York: Routledge. [ Google Scholar ]
• Hoadley, C. M. (2004). Methodological alignment in design-based research . Educational Psychologist , 39 ( 4 ), 203–212. [ Google Scholar ]
• Jackson, M., Tran, A., Wenderoth, M. P., Doherty, J. H. (2018). Peer vs. self-grading of practice exams: Which is better? CBE—Life Sciences Education , 17 ( 3 ), es44. 10.1187/cbe.18-04-0052 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Jin, H., Anderson, C. W. (2012). A learning progression for energy in socio-ecological systems . Journal of Research in Science Teaching , 49 ( 9 ), 1149–1180. [ Google Scholar ]
• Joseph, D. (2004). The practice of design-based research: Uncovering the interplay between design, research, and the real-world context . Educational Psychologist , 39 ( 4 ), 235–242. [ Google Scholar ]
• Kelly, A. E. (2014). Design-based research in engineering education . In Cambridge handbook of engineering education research (pp. 497–518). New York, NY: Cambridge University Press. 10.1017/CBO9781139013451.032 [ CrossRef ] [ Google Scholar ]
• Lee, C. J., Toven-Lindsey, B., Shapiro, C., Soh, M., Mazrouee, S., Levis-Fitzgerald, M., Sanders, E. R. (2018). Error-discovery learning boosts student engagement and performance, while reducing student attrition in a bioinformatics course . CBE—Life Sciences Education , 17 ( 3 ), ar40. 10.1187/cbe.17-04-0061 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Lo, S. M., Gardner, G. E., Reid, J., Napoleon-Fanis, V., Carroll, P., Smith, E., Sato, B. K. (2019). Prevailing questions and methodologies in biology education research: A longitudinal analysis of research in CBE — life sciences education and at the society for the advancement of biology education research . CBE—Life Sciences Education , 18 ( 1 ), ar9. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Marbach-Ad, G., Briken, V., El-Sayed, N. M., Frauwirth, K., Fredericksen, B., Hutcheson, S., Smith, A. C. (2009). Assessing student understanding of host pathogen interactions using a concept inventory . Journal of Microbiology & Biology Education , 10 ( 1 ), 43–50. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• McFarland, J. L., Price, R. M., Wenderoth, M. P., Martinková, P., Cliff, W., Michael, J., … & Wright, A. (2017). Development and validation of the homeostasis concept inventory . CBE—Life Sciences Education , 16 ( 2 ), ar35. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• McKenney, S., Reeves, T. C. (2013). Systematic review of design-based research progress: Is a little knowledge a dangerous thing? Educational Researcher , 42 ( 2 ), 97–100. [ Google Scholar ]
• Mestre, J. P., Cheville, A., Herman, G. L. (2018). Promoting DBER-cognitive psychology collaborations in STEM education . Journal of Engineering Education , 107 ( 1 ), 5–10. [ Google Scholar ]
• Michael, J. A. (2007). What makes physiology hard for students to learn? Results of a faculty survey . AJP: Advances in Physiology Education , 31 ( 1 ), 34–40. [ PubMed ] [ Google Scholar ]
• Michael, J. A., Modell, H., McFarland, J., Cliff, W. (2009). The “core principles” of physiology: What should students understand? Advances in Physiology Education , 33 ( 1 ), 10–16. [ PubMed ] [ Google Scholar ]
• Middleton, J., Gorard, S., Taylor, C., Bannan-Ritland, B. (2008). The “compleat” design experiment: From soup to nuts . In Kelly, A. E., Lesh, R. A., Baek, J. Y. (Eds.), Handbook of design research methods in education: Innovations in science, technology, engineering, and mathematics learning and teaching (pp. 21–46). New York, NY: Routledge. [ Google Scholar ]
• Modell, H. I. (2000). How to help students understand physiology? Emphasize general models . Advances in Physiology Education , 23 ( 1 ), S101–S107. [ PubMed ] [ Google Scholar ]
• Mohan, L., Chen, J., Anderson, C. W. (2009). Developing a multi-year learning progression for carbon cycling in socio-ecological systems . Journal of Research in Science Teaching , 46 ( 6 ), 675–698. [ Google Scholar ]
• National Academies of Sciences, Engineering, and Medicine. (2018). How People Learn II: Learners, Contexts, and Cultures . Washington, DC: National Academies Press. Retrieved June 24, 2019, from 10.17226/24783 [ CrossRef ] [ Google Scholar ]
• National Research Council (NRC). (2002). Scientific research in education . Washington, DC: National Academies Press. Retrieved January 31, 2019, from 10.17226/10236 [ CrossRef ] [ Google Scholar ]
• NRC. (2007). Taking science to school: Learning and teaching science in grades K–8 . Washington, DC: National Academies Press. Retrieved March 22, 2019, from www.nap.edu/catalog/11625/taking-science-to-school-learning-and-teaching-science-in-grades . 10.17226/11625 [ CrossRef ] [ Google Scholar ]
• NRC. (2012). Discipline-based education research: Understanding and improving learning in undergraduate science and engineering . Washington, DC: National Academies Press. Retrieved from www.nap.edu/catalog/13362/discipline-based-education-research-understanding-and-improving-learning-in-undergraduate . 10.17226/13362 [ CrossRef ] [ Google Scholar ]
• NRC. (2018). How people learn II: Learners, contexts, and cultures . Washington, DC: National Academies Press. Retrieved from www.nap.edu/read/24783/chapter/7 . 10.17226/24783 [ CrossRef ] [ Google Scholar ]
• O’Donnell, A. M. (2004). A commentary on design research . Educational Psychologist , 39 ( 4 ), 255–260. [ Google Scholar ]
• Offerdahl, E. G., McConnell, M., Boyer, J. (2018). Can I have your recipe? Using a fidelity of implementation (FOI) framework to identify the key ingredients of formative assessment for learning . CBE—Life Sciences Education , 17 ( 4 ), es16. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Peffer, M., Renken, M. (2016). Practical strategies for collaboration across discipline-based education research and the learning sciences . CBE—Life Sciences Education , 15 ( 4 ), es11. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Reiser, B. J., Smith, B. K., Tabak, I., Steinmuller, F., Sandoval, W. A., Leone, A. J. (2001). BGuILE: Strategic and conceptual scaffolds for scientific inquiry in biology classrooms . In Carver, S. M., Klahr, D. (Eds.), Cognition and instruction: Twenty-five years of progress (pp. 263–305). Mahwah, NJ: Lawrence Erlbaum Associates, Inc. [ Google Scholar ]
• Sandoval, W. (2014). Conjecture mapping: An approach to systematic educational design research . Journal of the Learning Sciences , 23 ( 1 ), 18–36. [ Google Scholar ]
• Sandoval, W. A., Bell, P. (2004). Design-based research methods for studying learning in context: Introduction . Educational Psychologist , 39 ( 4 ), 199–201. [ Google Scholar ]
• Schinske, J. N., Balke, V. L., Bangera, M. G., Bonney, K. M., Brownell, S. E., Carter, R. S., … & Corwin, L. A. (2017). Broadening participation in biology education research: Engaging community college students and faculty . CBE—Life Sciences Education , 16 ( 2 ), mr1. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Scott, E., Anderson, C. W., Mashood, K. K., Matz, R. L., Underwood, S. M., Sawtelle, V. (2018). Developing an analytical framework to characterize student reasoning about complex processes . CBE—Life Sciences Education , 17 ( 3 ), ar49. 10.1187/cbe.17-10-0225 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Scott, E., Wenderoth, M. P., Doherty, J. H. (2019). Learning progressions: An empirically grounded, learner-centered framework to guide biology instruction . CBE—Life Sciences Education , 18 ( 4 ), es5. 10.1187/cbe.19-03-0059 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sharma, M. D., McShane, K. (2008). A methodological framework for understanding and describing discipline-based scholarship of teaching in higher education through design-based research . Higher Education Research & Development , 27 ( 3 ), 257–270. [ Google Scholar ]
• Shavelson, R. J., Phillips, D. C., Towne, L., Feuer, M. J. (2003). On the science of education design studies . Educational Researcher , 32 ( 1 ), 25–28. [ Google Scholar ]
• Siegler, R. S. (2006). Microgenetic analyses of learning . In Damon, W., Lerner, R. M. (Eds.), Handbook of child psychology (pp. 464–510). Hoboken, NJ: John Wiley & Sons, Inc. 10.1002/9780470147658.chpsy0211 [ CrossRef ] [ Google Scholar ]
• Stringer, E. T. (2013). Action research . Thousand Oaks, CA: Sage Publications, Inc. [ Google Scholar ]
• Subramaniam, M., Jean, B. S., Taylor, N. G., Kodama, C., Follman, R., Casciotti, D. (2015). Bit by bit: Using design-based research to improve the health literacy of adolescents . JMIR Research Protocols , 4 ( 2 ), e62. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Summers, M. M., Couch, B. A., Knight, J. K., Brownell, S. E., Crowe, A. J., Semsar, K., … & Batzli, J. (2018). EcoEvo-MAPS: An ecology and evolution assessment for introductory through advanced undergraduates . CBE—Life Sciences Education , 17 ( 2 ), ar18. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Szteinberg, G., Balicki, S., Banks, G., Clinchot, M., Cullipher, S., Huie, R., … & Sevian, H. (2014). Collaborative professional development in chemistry education research: Bridging the gap between research and practice . Journal of Chemical Education , 91 ( 9 ), 1401–1408. [ Google Scholar ]
• Talanquer, V. (2014). DBER and STEM education reform: Are we up to the challenge? Journal of Research in Science Teaching , 51 ( 6 ), 809–819. [ Google Scholar ]
• Wang, F., Hannafin, M. J. (2005). Design-based research and technology-enhanced learning environments . Educational Technology Research and Development , 53 ( 4 ), 5–23. [ Google Scholar ]
• Wang, J.-R. (2004). Development and validation of a Two-tier instrument to examine understanding of internal transport in plants and the human circulatory system . International Journal of Science and Mathematics Education , 2 ( 2 ), 131–157. [ Google Scholar ]
• Warfa, A.-R. M. (2016). Mixed-methods design in biology education research: Approach and uses . CBE—Life Sciences Education , 15 ( 4 ), rm5. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Windschitl, M., Thompson, J., Braaten, M., Stroupe, D. (2012). Proposing a core set of instructional practices and tools for teachers of science . Science Education , 96 ( 5 ), 878–903. [ Google Scholar ]
• Zagallo, P., Meddleton, S., Bolger, M. S. (2016). Teaching real data interpretation with models (TRIM): Analysis of student dialogue in a large-enrollment cell and developmental biology course . CBE—Life Sciences Education , 15 ( 2 ), ar17. [ PMC free article ] [ PubMed ] [ Google Scholar ]

• Introduction
• Acknowledgements
• 1. Groundwork
• 1.1. Research
• 1.2. Knowing
• 1.3. Theories
• 1.4. Ethics
• 2. Paradigms
• 2.1. Inferential Statistics
• 2.2. Sampling
• 2.3. Qualitative Rigor
• 2.4. Design-Based Research
• 2.5. Mixed Methods
• 3. Learning Theories
• 3.1. Behaviorism
• 3.2. Cognitivism
• 3.3. Constructivism
• 3.4. Socioculturalism
• 3.5. Connectivism
• Appendix A. Supplements
• Appendix B. Example Studies
• Example Study #1. Public comment sentiment on educational videos
• Example Study #2. Effects of open textbook adoption on teachers' open practices
• Appendix C. Historical Readings
• Manifesto of the Communist Party (1848)
• On the Origin of Species (1859)
• Science and the Savages (1905)
• Theories of Knowledge (1916)
• Theories of Morals (1916)
• Translations

Design-Based Research

Choose a sign-in option.

Tools and Settings

Questions and Tasks

Citation and Embed Code

In an educational setting, design-based research is a research approach that engages in iterative designs to develop knowledge that improves educational practices. This chapter will provide a brief overview of the origin, paradigms, outcomes, and processes of design-based research (DBR). In these sections we explain that (a) DBR originated because some researchers believed that traditional research methods failed to improve classroom practices, (b) DBR places researchers as agents of change and research subjects as collaborators, (c) DBR produces both new designs and theories, and (d) DBR consists of an iterative process of design and evaluation to develop knowledge.

Origin of DBR

DBR originated as researchers like Allan Collins (1990) and Ann Brown (1992) recognized that educational research often failed to improve classroom practices. They perceived that much of educational research was conducted in controlled, laboratory-like settings. They believed that this laboratory research was not as helpful as possible for practitioners.

Proponents of DBR claim that educational research is often detached from practice (The Design-Based Research Collective, 2002). There are at least two problems that arise from this detachment: (a) practitioners do not benefit from researchers’ work and (b) research results may be inaccurate because they fail to account for context (The Design-Based Research Collective, 2002).

Practitioners do not benefit from researchers’ work if the research is detached from practice. Practitioners are able to benefit from research when they see how the research can inform and improve their designs and practices. Some practitioners believe that educational research is often too abstract or sterilized to be useful in real contexts (The Design-Based Research Collective, 2002).

Not only is lack of relevance a problem, but research results can also be inaccurate by failing to account for context. Findings and theories based on lab results may not accurately reflect what happens in real-world educational settings.

Conversely, a problem that arises from an overemphasis on practice is that while individual practices may improve, the general body of theory and knowledge does not increase. Scholars like Collins (1990) and Brown (1992) believed that the best way to conduct research would be to achieve the right balance between theory-building and practical impact.

Paradigms of DBR

Proponents of DBR believe that conducting research in context, rather than in a controlled laboratory setting, and iteratively designing interventions yields authentic and useful knowledge. Sasha Barab (2004) says that the goal of DBR is to “directly impact practice while advancing theory that will be of use to others” (p. 8). This implies “a pragmatic philosophical underpinning, one in which the value of a theory lies in its ability to produce changes in the world” (p. 6). The aims of DBR and the role of researchers and subjects are informed by this philosophical underpinning.

Aims of DBR

Traditional, experimental research is conducted by theorists focused on isolating variables to test and refine theory. DBR is conducted by designers focused on (a) understanding contexts, (b) designing effective systems, and (c) making meaningful changes for the subjects of their studies (Barab & Squire, 2004; Collins, 1990). Traditional methods of research generate refined understandings of how the world works, which may indirectly affect practice. In DBR there is an intentionality in the research process to both refine theory and practice (Collins et al., 2004).

Role of DBR Researcher

In DBR, researchers assume the roles of “curriculum designers, and implicitly, curriculum theorists” (Barab & Squire, 2004, p.2). As curriculum designers, DBR researchers come into their contexts as informed experts with the purpose of creating, “test[ing] and refin[ing] educational designs based on principles derived from prior research” (Collins et al., 2004, p. 15). These educational designs may include curricula, practices, software, or tangible objects beneficial to the learning process (Barab & Squire, 2004). As curriculum theorists, DBR researchers also come into their research contexts with the purpose to refine extant theories about learning (Brown, 1992).

This duality of roles for DBR researchers contributes to a greater sense of responsibility and accountability within the field. Traditional, experimental researchers isolate themselves from the subjects of their study (Barab & Squire, 2004). This separation is seen as a virtue, allowing researchers to make dispassionate observations as they test and refine their understandings of the world around them. In comparison, design-based researchers “bring agendas to their work,” see themselves as necessary agents of change and see themselves as accountable for the work they do (Barab & Squire, 2004, p. 2).

Role of DBR Subjects

Within DBR, research subjects are seen as key contributors and collaborators in the research process. Classic experimentalism views the subjects of research as things to be observed or experimented on, suggesting a unidirectional relationship between researcher and research subject. The role of the research subject is to be available and genuine so that the researcher can make meaningful observations and collect accurate data. In contrast, design-based researchers view the subjects of their research (e.g., students, teachers, schools) as “co-participants” (Barab & Squire, 2004, p. 3) and “co-investigators” (Collins, 1990, p. 4). Research subjects are seen as necessary in “helping to formulate the questions,” “making refinements in the designs,” “evaluating the effects of...the experiment,” and “reporting the results of the experiment to other teachers and researchers” (Collins, 1990, pp. 4-5). Research subjects are co-workers with the researcher in iteratively pushing the study forward.

Outcomes of DBR

DBR educational research develops knowledge through this collaborative, iterative research process. The knowledge developed by DBR can be separated into two categories: (a) tangible, practical outcomes and (b) intangible, theoretical outcomes.

Tangibles Outcomes

A major goal of design-based research is producing meaningful interventions and practices. Within educational research these interventions may “involve the development of technological tools [and] curricula” (Barab & Squire, 2004, p. 1). But more than just producing meaningful educational products for a specific context, DBR aims to produce meaningful, effective educational products that can be transferred and adapted (Barab & Squire, 2004). As expressed by Brown (1992), “an effective intervention should be able to migrate from our experimental classroom to average classrooms operated by and for average students and teachers” (p.143).

Intangible Outcomes

It is important to recognize that DBR is not only concerned with improving practice but also aims to advance theory and understanding (Collins et al., 2004). DBR’s emphasis on the importance of context enhances the knowledge claims of the research. “Researchers investigate cognition in context...with the broad goal of developing evidence-based claims derived from both laboratory-based and naturalistic investigations that result in knowledge about how people learn” (Barab & Squire, 2004, p.1). This new knowledge about learning then drives future research and practice.

Process of DBR

A hallmark of DBR is the iterative nature of its interventions. As each iteration progresses, researchers refine and rework the intervention drawing on a variety of research methods that best fit the context. This flexibility allows the end result to take precedence over the process. While each researcher may use different methods, McKenny and Reeves (2012) outlined three core processes of DBR: (a) analysis and exploration, (b) design and construction, and (c) evaluation and reflection. To put these ideas in context, we will refer to a recent DBR study completed by Siko and Barbour regarding the use of PowerPoint games in the classroom.

Analysis and Exploration

Analysis is a critical aspect of DBR and must be used throughout the entire process. At the start of a DBR project, it is critical to understand and define which problem will be addressed. In collaboration with practitioners, researchers seek to understand all aspects of a problem. Additionally, they “seek out and learn from how others have viewed and solved similar problems ” (McKenny & Reeves, 2012, p. 85). This analysis helps to provide an understanding of the context within which to execute an intervention.

Since theories cannot account for the variety of variables in a learning situation, exploration is needed to fill the gaps. DBR researchers can draw from a number of disciplines and methodologies as they execute an intervention. The decision of which methodologies to use should be driven by the research context and goals.

Siko and Barbour (2016) used the DBR process to address a gap they found in research regarding the effectiveness of having students create their own PowerPoint games to review for a test. In analyzing existing research, they found studies that stated teaching students to create their own PowerPoint games did not improve content retention. Siko and Barbour wanted to “determine if changes to the implementation protocol would lead to improved performance” (Siko & Barbour, 2016, p. 420). They chose to test their theory in three different phases and adapt the curriculum following each phase.

Design and Construction

Informed by the analysis and exploration, researchers design and construct interventions, which may be a specific technology or “less concrete aspects such as activity structures, institutions, scaffolds, and curricula” (Design-Based Research Collective, 2003, pp. 5–6). This process involves laying out a variety of options for a solution and then creating the idea with the most promise.

Within Siko and Barbour’s design, they planned to observe three phases of a control group and a test group. Each phase would use t-tests to compare two unit tests for each group. They worked with teachers to implement time for playing PowerPoint games as well as a discussion on what makes games successful. The first implementation was a control phase that replicated past research and established a baseline. Once they finished that phase, they began to evaluate.

Evaluation and Reflection

Researchers can evaluate their designs both before and after use. The cyclical process involves careful, constant evaluation for each iteration so that improvements can be made. While tests and quizzes are a standard way of evaluating educational progress, interviews and observations also play a key role, as they allow for better understanding of how teachers and students might see the learning situation.

Reflection allows the researcher to make connections between actions and results. Researchers must take the time to analyze what changes allowed them to have success or failure so that theory and practice at large can be benefited. Collins (1990) states:

It is important to analyze the reasons for failure and to take steps to fix them. It is critical to document the nature of the failures and the attempted revisions, as well as the overall results of the experiment, because this information informs the path to success. (pg. 5)

As researchers reflect on each change they made, they find what is most useful to the field at large, whether it be a failure or a success.

After evaluating results of the first phase, Siko and Barbour revisited the literature of instructional games. Based on that research, they first tried extending the length of time students spent creating the games. They also discovered that the students struggled to design effective test questions, so the researchers tried working with teachers to spend more time explaining how to ask good questions. As they explored these options, researchers were able to see unit test scores improve.

Reflection on how the study was conducted allowed the researchers to properly place their experiences within the context of existing research. They recognized that while they found positive impacts as a result of their intervention, there were a number of limitations with the study. This is an important realization for the research and allows readers to not misinterpret the scope of the findings.

This chapter has provided a brief overview of the origin, paradigms, outcomes, and processes of Design-Based Research (DBR). We explained that (a) DBR originated because some researchers believed that traditional research methods failed to improve classroom practices, (b) DBR places researchers as agents of change and research subjects as collaborators, (c) DBR produces both new designs and theories, and (d) DBR consists of an iterative process of design and evaluation to develop knowledge.

Barab, S., & Squire, K. (2004). Design-based research: putting a stake in the ground. Journal of the Learning Sciences, 13(1), 1–14.

Brown, A. L. (1992). Design experiments: theoretical and methodological challenges in creating complex interventions in classroom settings. Journal of the Learning Sciences, 2(2), 141–178.

Collins, A. (1990). Toward a design science of education (Report No. 1). Washington, DC: Center for Technology in Education.

Collins, A., Joseph, D., & Bielaczyc, K. (2004). Design research: Theoretical and methodological issues. Journal of the Learning Sciences, 13(1), 15–42.

Mckenney, S., & Reeves, T.C. (2012) Conducting Educational Design Research. New York, NY: Routledge.

Siko, J. P., & Barbour, M. K. (2016). Building a better mousetrap: how design-based research was used to improve homemade PowerPoint games. TechTrends, 60(5), 419–424.

The Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5–8.

This content is provided to you freely by BYU Open Learning Network.

Access it online or download it at https://open.byu.edu/education_research/design_based_research .

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Kimberly Christensen and Richard E. West

Design-Based Research (DBR) is one of the most exciting evolutions in research methodology of our time, as it allows for the potential knowledge gained through the intimate connections designers have with their work to be combined with the knowledge derived from research. These two sources of knowledge can inform each other, leading to improved design interventions as well as improved local and generalizable theory. However, these positive outcomes are not easily attained, as DBR is also a difficult method to implement well. The good news is that we can learn much from other disciplines who are also seeking to find effective strategies for intertwining design and research. In this chapter, we will review the history of DBR as well as Interdisciplinary Design Research (IDR) and then discuss potential implications for our field.

Shared Origins with IDR

These two types of design research, both DBR and IDR, share a common genesis among the design revolution of the 1960s, where designers, researchers, and scholars sought to elevate design from mere practice to an independent scholarly discipline, with its own research and distinct theoretical and methodological underpinnings. A scholarly focus on design methods, they argued, would foster the development of design theories, which would in turn improve the quality of design and design practice (Margolin, 2010). Research on design methods, termed design research, would be the foundation of this new discipline.

Design research had existed in primitive form—as market research and process analysis—since before the turn of the 20th century, and, although it had served to improve processes and marketing, it had not been applied as scientific research. John Chris Jones, Bruce Archer, and Herbert Simon were among the first to shift the focus from research for design (e.g., research with the intent of gathering data to support product development) to research on design (e.g., research exploring the design process). Their efforts framed the initial development of design research and science.

John Chris Jones

An engineer, Jones (1970) felt that the design process was ambiguous and often too abstruse to discuss effectively. One solution, he offered, was to define and discuss design in terms of methods. By identifying and discussing design methods, researchers would be able to create transparency in the design process, combating perceptions of design being more or less mysteriously inspired. This discussion of design methods, Jones proposed, would in turn raise the level of discourse and practice in design.

Bruce Archer

Archer, also an engineer, worked with Jones and likewise supported the adoption of research methods from other disciplines. Archer (1965) proposed that applying systematic methods would improve the assessment of design problems and foster the development of effective solutions. Archer recognized, however, that improved practice alone would not enable design to achieve disciplinary status. In order to become a discipline, design required a theoretical foundation to support its practice. Archer (1981) advocated that design research was the primary means by which theoretical knowledge could be developed. He suggested that the application of systematic inquiry, such as existed in engineering, would yield knowledge about not only product and practice, but also the theory that guided each.

Herbert Simon

It was multidisciplinary social scientist Simon, however, that issued the clarion call for transforming design into design science (Buchanan, 2007; Collins, 1992; Collins, Joseph, & Bielaczyc, 2004; Cross, 1999; Cross, 2007; Friedman, 2003; Jonas, 2007; Willemien, 2009). In The Sciences of the Artificial, Simon (1969) reasoned that the rigorous inquiry and discussion surrounding naturally occurring processes and phenomena was just as necessary for man-made products and processes. He particularly called for “[bodies] of intellectually tough, analytic, partly formalizable, partly empirical, teachable doctrine about the design process” (p. 132). This call for more scholarly discussion and practice resonated with designers across disciplines in design and engineering (Buchanan, 2007; Cross, 1999; Cross, 2007; Friedman, 2003; Jonas, 2007; Willemien, 2009). IDR sprang directly from this early movement and has continued to gain momentum, producing an interdisciplinary body of research encompassing research efforts in engineering, design, and technology.

Years later, in the 1980s, Simon’s work inspired the first DBR efforts in education (Collins et al., 2004). Much of the DBR literature attributes its beginnings to the work of Ann Brown and Allan Collins (Cobb, Confrey, diSessa, Lehrer, & Schauble, 2003; Collins et al., 2004; Kelly, 2003; McCandliss, Kalchman, & Bryant, 2003; Oh & Reeves, 2010; Reeves, 2006; Shavelson, Phillips, Towne, & Feuer, 2003; Tabak, 2004; van den Akker, 1999). Their work, focusing on research and development in authentic contexts, drew heavily on research approaches and development practices in the design sciences, including the work of early design researchers such as Simon (Brown, 1992; Collins, 1992; Collins et al., 2004). However, over generations of research, this connection has been all but forgotten, and DBR, although similarly inspired by the early efforts of Simon, Archer, and Jones, has developed into an isolated and discipline-specific body of design research, independent from its interdisciplinary cousin.

Current Issues in DBR

The initial obstacle to understanding and engaging in DBR is understanding what DBR is. What do we call it? What does it entail? How do we do it? Many of the current challenges facing DBR concern these questions. Specifically, there are three issues that influence how DBR is identified, implemented, and discussed. First, proliferation of terminology among scholars and inconsistent use of these terms have created a sprawling body of literature, with various splinter DBR groups hosting scholarly conversations regarding their particular brand of DBR. Second, DBR, as a field, is characterized by a lack of definition, in terms of its purpose, its characteristics, and the steps or processes of which it is comprised. Third, the one consistent element of DBR across the field is an unwieldy set of considerations incumbent upon the researcher.

Because it is so difficult to define and conceptualize DBR, it is similarly difficult to replicate authentically. Lack of scholarly agreement on the characteristics and outcomes that define DBR withholds a structure by which DBR studies can be identified and evaluated and, ultimately, limits the degree to which the field can progress. The following sections will identify and explore the three greatest challenges facing DBR today: proliferation of terms, lack of definition, and competing demands.

Proliferation of terminology

One of the most challenging characteristics of DBR is the quantity and use of terms that identify DBR in the research literature. There are seven common terms typically associated with DBR: design experiments, design research, design-based research, formative research, development research, developmental research, and design-based implementation research.

Synonymous terms

Collins and Brown first termed their efforts design experiments (Brown, 1992; Collins, 1992). Subsequent literature stemming from or relating to Collins’ and Brown’s work used design research and design experiments synonymously (Anderson & Shattuck, 2012; Collins et al., 2004). Design-based research was introduced to distinguish DBR from other research approaches. Sandoval and Bell (2004) best summarized this as follows:

We have settled on the term design-based research over the other commonly used phrases “design experimentation,” which connotes a specific form of controlled experimentation that does not capture the breadth of the approach, or “design research,” which is too easily confused with research design and other efforts in design fields that lack in situ research components. (p. 199)

Variations by discipline

Terminology across disciplines refers to DBR approaches as formative research, development research, design experiments, and developmental research. According to van den Akker (1999), the use of DBR terminology also varies by educational sub-discipline, with areas such as (a) curriculum, (b) learning and instruction, (c) media and technology, and (d) teacher education and didactics favoring specific terms that reflect the focus of their research (Figure 1).

Figure 1. Variations in DBR terminology across educational sub-disciplines.

Lack of definition

This variation across disciplines, with design researchers tailoring design research to address discipline-specific interests and needs, has created a lack of definition in the field overall. In addition, in the literature, DBR has been conceptualized at various levels of granularity. Here, we will discuss three existing approaches to defining DBR: (a) statements of the overarching purpose, (b) lists of defining characteristics, and (c) models of the steps or processes involved.

General purpose

In literature, scholars and researchers have made multiple attempts to isolate the general purpose of design research in education, with each offering a different insight and definition. According to van den Akker (1999), design research is distinguished from other research efforts by its simultaneous commitment to (a) developing a body of design principles and methods that are based in theory and validated by research and (b) offering direct contributions to practice. This position was supported by Sandoval and Bell (2004), who suggested that the general purpose of DBR was to address the “tension between the desire for locally usable knowledge, on the one hand, and scientifically sound, generalizable knowledge on the other” (p. 199). Cobb et al. (2003) particularly promoted the theory-building focus, asserting “design experiments are conducted to develop theories, not merely to empirically tune ‘what works’” (p. 10). Shavelson et al. (2003) recognized the importance of developing theory but emphasized that the testing and building of instructional products was an equal focus of design research rather than the means to a theoretical end.

The aggregate of these definitions suggests that the purpose of DBR involves theoretical and practical design principles and active engagement in the design process. However, DBR continues to vary in its prioritization of these components, with some focusing largely on theory, others emphasizing practice or product, and many examining neither but all using the same terms.

Specific characteristics

Another way to define DBR is by identifying the key characteristics that both unite and define the approach. Unlike other research approaches, DBR can take the form of multiple research methodologies, both qualitative and quantitative, and thus cannot be recognized strictly by its methods. Identifying characteristics, therefore, concern the research process, context, and focus. This section will discuss the original characteristics of DBR, as introduced by Brown and Collins, and then identify the seven most common characteristics suggested by DBR literature overall.

Brown’s concept of DBR. Brown (1992) defined design research as having five primary characteristics that distinguished it from typical design or research processes. First, a design is engineered in an authentic, working environment. Second, the development of research and the design are influenced by a specific set of inputs: classroom environment, teachers and students as researchers, curriculum, and technology. Third, the design and development process includes multiple cycles of testing, revision, and further testing. Fourth, the design research process produces an assessment of the design’s quality as well as the effectiveness of both the design and its theoretical underpinnings. Finally, the overall process should make contributions to existing learning theory.

Collins’s concept of DBR. Collins (1990, 1992) posed a similar list of design research characteristics. Collins echoed Brown’s specifications of authentic context, cycles of testing and revision, and design and process evaluation. Additionally, Collins provided greater detail regarding the characteristics of the design research processes—specifically, that design research should include the comparison of multiple sample groups, be systematic in both its variation within the experiment and in the order of revisions (i.e., by testing the innovations most likely to succeed first), and involve an interdisciplinary team of experts including not just the teacher and designer, but technologists, psychologists, and developers as well. Unlike Brown, however, Collins did not refer to theory building as an essential characteristic.

Current DBR characteristics. The DBR literature that followed expanded, clarified, and revised the design research characteristics identified by Brown and Collins. The range of DBR characteristics discussed in the field currently is broad but can be distilled to seven most frequently referenced identifying characteristics of DBR: design driven, situated, iterative, collaborative, theory building, practical, and productive.

Design driven.  All literature identifies DBR as focusing on the evolution of a design (Anderson & Shattuck, 2012; Brown, 1992; Cobb et al., 2003; Collins, 1992; Design-Based Research Collective, 2003). While the design can range from an instructional artifact to an intervention, engagement in the design process is what yields the experience, data, and insight necessary for inquiry.

Situated.  Recalling Brown’s (1992) call for more authentic research contexts, nearly all definitions of DBR situate the aforementioned design process in a real-world context, such as a classroom (Anderson & Shattuck, 2012; Barab & Squire, 2004; Cobb et al., 2003).

Iterative. Literature also appears to agree that a DBR process does not consist of a linear design process, but rather multiple cycles of design, testing, and revision (Anderson & Shattuck, 2012; Barab & Squire, 2004; Brown, 1992; Design-Based Research Collective, 2003; Shavelson et al., 2003). These iterations must also represent systematic adjustment of the design, with each adjustment and subsequent testing serving as a miniature experiment (Barab & Squire, 2004; Collins, 1992).

Collaborative.  While the literature may not always agree on the roles and responsibilities of those engaged in DBR, collaboration between researchers, designers, and educators appears to be key (Anderson & Shattuck, 2012; Barab & Squire, 2004; McCandliss et al., 2003). Each collaborator enters the project with a unique perspective and, as each engages in research, forms a role-specific view of phenomena. These perspectives can then be combined to create a more holistic view of the design process, its context, and the developing product.

Theory building.  Design research focuses on more than creating an effective design; DBR should produce an intimate understanding of both design and theory (Anderson & Shattuck, 2012; Barab & Squire, 2004; Brown, 1992; Cobb et al., 2003; Design-Based Research Collective, 2003; Joseph, 2004; Shavelson et al., 2003). According to Barab & Squire (2004), “Design-based research requires more than simply showing a particular design works but demands that the researcher . . . generate evidence-based claims about learning that address contemporary theoretical issues and further the theoretical knowledge of the field” (p. 6). DBR needs to build and test theory, yielding findings that can be generalized to both local and broad theory (Hoadley, 2004).

Practical.  While theoretical contributions are essential to DBR, the results of DBR studies “must do real work” (Cobb et al., 2003, p. 10) and inform instructional, research, and design practice (Anderson & Shattuck, 2012; Barab & Squire, 2004; Design-Based Research Collective, 2003; McCandliss et al., 2003).

Productive.  Not only should design research produce theoretical and practical insights, but also the design itself must produce results, measuring its success in terms of how well the design meets its intended outcomes (Barab & Squire, 2004; Design-Based Research Collective, 2003; Joseph, 2004; McCandliss et al., 2003).

Steps and processes

The third way DBR could possibly be defined is to identify the steps or processes involved in implementing it. The sections below illustrate the steps outlined by Collins (1990) and Brown (1992) as well as models by Bannan-Ritland (2003), Reeves (2006), and an aggregate model presented by Anderson & Shattuck (2012).

Collins’s design experimentation steps.  In his technical report, Collins (1990) presented an extensive list of 10 steps in design experimentation (Figure 2). While Collins’s model provides a guide for experimentally testing and developing new instructional programs, it does not include multiple iterative stages or any evaluation of the final product. Because Collins was interested primarily in development, research was not given much attention in his model.

Brown’s design research example.  The example of design research Brown (1992) included in her article was limited and less clearly delineated than Collins’s model (Figure 2). Brown focused on the development of educational interventions, including additional testing with minority populations. Similar to Collins, Brown also omitted any summative evaluation of intervention quality or effectiveness and did not specify the role of research through the design process.

Bannan-Ritland’s DBR model.  Bannan-Ritland (2003) reviewed design process models in fields such as product development, instructional design, and engineering to create a more sophisticated model of design-based research. In its simplest form, Bannan-Ritland’s model is comprised of multiple processes subsumed under four broad stages: (a) informed exploration, (b) enactment, (c) evaluation of local impact, and (d) evaluation of broad impact. Unlike Collins and Brown, Bannan-Ritland dedicated large portions of the model to evaluation in terms of the quality and efficacy of the final product as well as the implications for theory and practice.

Reeves’s development research model.  Reeves (2006) provided a simplified model consisting of just four steps (Figure 2). By condensing DBR into just a few steps, Reeves highlighted what he viewed as the most essential processes, ending with a general reflection on both the process and product generated in order to develop theoretical and practical insights.

Anderson and Shattuck’s aggregate model.  Anderson and Shattuck (2012) reviewed design-based research abstracts over the past decade and, from their review, presented an eight-step aggregate model of DBR (Figure 2). As an aggregate of DBR approaches, this model was their attempt to unify approaches across DBR literature, and includes similar steps to Reeves’s model. However, unlike Reeves, Anderson and Shattuck did not include summative reflection and insight development.

Comparison of models. Following in Figure 2, we provide a comparison of all these models side-by-side.

Competing demands and roles

The third challenge facing DBR is the variety of roles researchers are expected to fulfill, with researchers often acting simultaneously as project managers, designers, and evaluators. However, with most individuals able to focus on only one task at a time, these competing demands on resources and researcher attention and faculties can be challenging to balance, and excess focus on one role can easily jeopardize others. The literature has recognized four major roles that a DBR professional must perform simultaneously: researcher, project manager, theorist, and designer.

Researcher as researcher

Planning and carrying out research is already comprised of multiple considerations, such as controlling variables and limiting bias. The nature of DBR, with its collaboration and situated experimentation and development, innately intensifies some of these issues (Hoadley, 2004). While simultaneously designing the intervention, a design-based researcher must also ensure that high-quality research is accomplished, per typical standards of quality associated with quantitative or qualitative methods.

However, research is even more difficult in DBR because the nature of the method leads to several challenges. First, it can be difficult to control the many variables at play in authentic contexts (Collins et al., 2004). Many researchers may feel torn between being able to (a) isolate critical variables or (b) study the comprehensive, complex nature of the design experience (van den Akker, 1999). Second, because many DBR studies are qualitative, they produce large amounts of data, resulting in demanding data collection and analysis (Collins et al., 2004). Third, according to Anderson and Shattuck (2012), the combination of demanding data analysis and highly invested roles of the researchers leaves DBR susceptible to multiple biases during analysis. Perhaps best expressed by Barab and Squire (2004), “if a researcher is intimately involved in the conceptualization, design, development, implementation, and researching of a pedagogical approach, then ensuring that researchers can make credible and trustworthy assertions is a challenge” (p. 10). Additionally, the assumption of multiple roles invests much of the design and research in a single person, diminishing the likelihood of replicability (Hoadley, 2004). Finally, it is impossible to document or account for all discrete decisions made by the collaborators that influenced the development and success of the design (Design-Based Research Collective, 2003).

Quality research, though, was never meant to be easy! Despite these challenges, DBR has still been shown to be effective in simultaneously developing theory through research as well as interventions that can benefit practice—the two simultaneous goals of any instructional designer.

Researcher as project manager

The collaborative nature of DBR lends the approach one of its greatest strengths: multiple perspectives. While this can be a benefit, collaboration between researchers, developers, and practitioners needs to be highly coordinated (Collins et al., 2004), because it is difficult to manage interdisciplinary teams and maintain a productive, collaborative partnership (Design-Based Research Collective, 2003).

Researcher as theorist

For many researchers in DBR, the development or testing of theory is a foundational component and primary focus of their work. However, the iterative and multi-tasking nature of a DBR process may not be well-suited to empirically testing or building theory. According to Hoadley (2004), “the treatment’s fidelity to theory [is] initially, and sometimes continually, suspect” (p. 204). This suggests that researchers, despite intentions to test or build theory, may not design or implement their solution in alignment with theory or provide enough control to reliably test the theory in question.

Researcher as designer

Because DBR is simultaneously attempting to satisfy the needs of both design and research, there is a tension between the responsibilities of the researcher and the responsibilities of the designer (van den Akker, 1999). Any design decision inherently alters the research. Similarly, research decisions place constraints on the design. Skilled design-based researchers seek to balance these competing demands effectively.

What we can learn from IDR

IDR has been encumbered by similar issues that currently exist in DBR. While IDR is by no means a perfect field and is still working to hone and clarify its methods, it has been developing for two decades longer than DBR. The history of IDR and efforts in the field to address similar issues can yield possibilities and insights for the future of DBR. The following sections address efforts in IDR to define the field that hold potential for application in DBR, including how professionals in IDR have focused their efforts to increase unity and worked to define sub-approaches more clearly.

Defining Approaches

Similar to DBR, IDR has been subject to competing definitions as varied as the fields in which design research has been applied (i.e., product design, engineering, manufacturing, information technology, etc.) (Findeli, 1998; Jonas, 2007; Schneider, 2007). Typically, IDR scholars have focused on the relationship between design and research, as well as the underlying purpose, to define the approach. This section identifies three defining conceptualizations of IDR—the prepositional approach trinity, Cross’s -ologies, and Buchanan’s strategies of productive science—and discusses possible implications for DBR.

The approach trinity

One way of defining different purposes of design research is by identifying the preposition in the relationship between research and design: research into design, research for design, and research through design (Buchanan, 2007; Cross, 1999; Findeli, 1998; Jonas, 2007; Schneider, 2007).

Jonas (2007) identified research into design as the most prevalent—and straightforward—form of IDR. This approach separates research from design practice; the researcher observes and studies design practice from without, commonly addressing the history, aesthetics, theory, or nature of design (Schneider, 2007). Research into design generally yields little or no contribution to broader theory (Findeli, 1998).

Research for design applies to complex, sophisticated projects, where the purpose of research is to foster product research and development, such as in market and user research (Findeli, 1998; Jonas, 2007). Here, the role of research is to build and improve the design, not contribute to theory or practice.

According to Jonas’s (2007) description, research through design bears the strongest resemblance to DBR and is where researchers work to shape their design (i.e., the research object) and establish connections to broader theory and practice. This approach begins with the identification of a research question and carries through the design process experimentally, improving design methods and finding novel ways of controlling the design process (Schneider, 2007). According to Findeli (1998), because this approach adopts the design process as the research method, it helps to develop authentic theories of design.

Cross’s -ologies

Cross (1999) conceived of IDR approaches based on the early drive toward a science of design and identified three bodies of scientific inquiry: epistemology, praxiology, and phenomenology. Design epistemology primarily concerns what Cross termed “designerly ways of knowing” or how designers think and communicate about design (Cross, 1999; Cross, 2007). Design praxiology deals with practices and processes in design or how to develop and improve artifacts and the processes used to create them. Design phenomenology examines the form, function, configuration, and value of artifacts, such as exploring what makes a cell phone attractive to a user or how changes in a software interface affect user’s activities within the application.

Buchanan’s strategies of productive science

Like Cross, Buchanan (2007) viewed IDR through the lens of design science and identified four research strategies that frame design inquiry: design science, dialectic inquiry, rhetorical inquiry, and productive science (Figure 2). Design science focuses on designing and decision-making, addressing human and consumer behavior. According to Buchanan (2007), dialectic inquiry examines the “social and cultural context of design; typically [drawing] attention to the limitations of the individual designer in seeking sustainable solutions to problems” (p.57). Rhetorical inquiry focuses on the design experience as well as the designer’s process to create products that are usable, useful, and desirable. Productive science studies how the potential of a design is realized through the refinement of its parts, including materials, form, and function. Buchanan (2007) conceptualized a design research—what he termed design inquiry—that includes elements of all four strategies, looking at the designer, the design, the design context, and the refinement process as a holistic experience.

Implications for DBR

While the literature has yet to accept any single approach to defining types of IDR, it may still be helpful for DBR to consider similar ways of limiting and defining sub-approaches in the field. The challenges brought on by collaboration, multiple researcher roles, and lack of sufficient focus on the design product could be addressed and relieved by identifying distinct approaches to DBR. This idea is not new. Bell and Sandoval (2004) opposed the unification of DBR, specifically design-based research, across educational disciplines (such as developmental psychology, cognitive science, and instructional design). However, they did not suggest any potential alternatives. Adopting an IDR approach, such as the approach trinity, could serve to both unite studies across DBR and clearly distinguish the purpose of the approach and its primary functions. Research into design could focus on the design process and yield valuable insights on design thinking and practice. Research for design could focus on the development of an effective product, which development is missing from many DBR approaches. Research through design would use the design process as a vehicle to test and develop theory, reducing the set of expected considerations. Any approach to dividing or defining DBR efforts could help to limit the focus of the study, helping to prevent the diffusion of researcher efforts and findings.

In this chapter we have reviewed the historical development of both design-based research and interdisciplinary design research in an effort to identify strategies in IDR that could benefit DBR development. Following are a few conclusions, leading to recommendations for the DBR field.

Improve interdisciplinary collaboration

Overall, one key advantage that IDR has had—and that DBR presently lacks—is communication and collaboration with other fields. Because DBR has remained so isolated, only rarely referencing or exploring approaches from other design disciplines, it can only evolve within the constraints of educational inquiry. IDR’s ability to conceive solutions to issues in the field is derived, in part, from a wide variety of disciplines that contribute to the body of research. Engineers, developers, artists, and a range of designers interpose their own ideas and applications, which are in turn adopted and modified by others. Fostering collaboration between DBR and IDR, while perhaps not the remedy to cure all scholarly ills, could yield valuable insights for both fields, particularly in terms of refining methodologies and promoting the development of theory.

Simplify terminology and improve consistency in use

As we identified in this paper, a major issue facing DBR is the proliferation of terminology among scholars and the inconsistency in usage. From IDR comes the useful acknowledgement that there can be research into design, for design, and through design (Buchanan, 2007; Cross, 1999; Findeli, 1998; Jonas, 2007; Schneider, 2007). This framework was useful for scholars in our conversations at the conference. A resulting recommendation, then, is that, in published works, scholars begin articulating which of these approaches they are using in that particular study. This can simplify the requirements on DBR researchers, because instead of feeling the necessity of doing all three in every paper, they can emphasize one. This will also allow us to communicate our research better with IDR scholars.

Describe DBR process in publications

Oftentimes authors publish DBR studies using the same format as regular research studies, making it difficult to recognize DBR research and learn how other DBR scholars mitigate the challenges we have discussed in this chapter. Our recommendation is that DBR scholars publish the messy findings resulting from their work and pull back the curtain to show how they balanced competing concerns to arrive at their results. We believe it would help if DBR scholars adopted more common frameworks for publishing studies. In our review of the literature, we identified the following characteristics, which are the most frequently used to identify DBR:

• DBR is design driven and intervention focused
• DBR is situated within an actual teaching/learning context
• DBR is iterative
• DBR is collaborative between researchers, designers, and practitioners
• DBR builds theory but also needs to be practical and result in useful interventions

One recommendation is that DBR scholars adopt these as the characteristics of their work that they will make explicit in every published paper so that DBR articles can be recognized by readers and better aggregated together to show the value of DBR over time. One suggestion is that DBR scholars in their methodology sections could adopt these characteristics as subheadings. So in addition to discussing data collection and data analysis, they would also discuss Design Research Type (research into, through, or of design), Description of the Design Process and Product, Design and Learning Context, Design Collaborations, and a discussion explicitly of the Design Iterations, perhaps by listing each iteration and then the data collection and analysis for each. Also in the concluding sections, in addition to discussing research results, scholars would discuss Applications to Theory (perhaps dividing into Local Theory and Outcomes and Transferable Theory and Findings) and Applications for Practice. Papers that are too big could be broken up with different papers reporting on different iterations but using this same language and formatting to make it easier to connect the ideas throughout the papers. Not all papers would have both local and transferable theory (the latter being more evident in later iterations), so it would be sufficient to indicate in a paper that local theory and outcomes were developed and met with some ideas for transferable theory that would be developed in future iterations. The important thing would be to refer to each of these main characteristics in each paper so that scholars can recognize the work as DBR, situate it appropriately, and know what to look for in terms of quality during the review process.

Application Exercises

• According to the authors, what are the major issues facing DBR and what are some things that can be done to address this problem?
• Imagine you have designed a new learning app for use in public schools. How would you go about testing it using design-based research?

Anderson, T., & Shattuck, J. (2012). Design-based research: A decade of progress in education research? Educational Researcher, 41 (1), 16–25.

Archer, L.B. (1965). Systematic method for designers. In N. Cross (ed.), Developments in design methodology. London, England: John Wiley, 1984, pp. 57–82.

Archer, L. B. (1981). A view of the nature of design research. In R. Jacques & J.A. Powell (Eds.), Design: Science: Method (pp. 36-39). Guilford, England: Westbury House.

Bannan-Ritland, B. (2003). The role of design in research: The integrative learning design framework. Educational Researcher, 32 (1), 21 –24. doi:10.3102/0013189X032001021

Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. The Journal of the Learning Sciences, 13 (1), 1–14.

Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Learning Sciences, 2 (2), 141–178.

Buchanan, R. (2007). Strategies of design research: Productive science and rhetorical inquiry. In R. Michel (Ed.), Design research now (pp. 55–66). Basel, Switzerland: Birkhäuser Verlag AG.

Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32 (1), 9–13. doi:10.3102/0013189X032001009

Collins, A. (1990). Toward a Design Science of Education. Technical Report No. 1.

Collins, A. (1992). Toward a design science of education. In E. Scanlon & T. O’Shea (Eds.), New directions in educational technology. Berlin, Germany: Springer-Verlag.

Collins, A., Joseph, D., & Bielaczyc, K. (2004). Design research: Theoretical and methodological issues. The Journal of the Learning Sciences, 13 (1), 15–42.

Cross, N. (1999). Design research: A disciplined conversation. Design Issues, 15 (2), 5–10. doi:10.2307/1511837

Cross, N. (2007). Forty years of design research. Design Studies, 28 (1), 1–4. doi:10.1016/j.destud.2006.11.004

Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32 (1), 5–8. doi:10.3102/0013189X032001005

Findeli, A. (1998). A quest for credibility: Doctoral education and research in design at the University of Montreal. Doctoral Education in Design, Ohio, 8–11 October 1998.

Friedman, K. (2003). Theory construction in design research: Criteria: approaches, and methods. Design Studies, 24 (6), 507–522.

Hoadley, C. M. (2004). Methodological alignment in design-based research. Educational Psychologist, 39 (4), 203–212.

Jonas, W. (2007). Design research and its meaning to the methodological development of the discipline. In R. Michel (Ed.), Design research now (pp. 187–206). Basel, Switzerland: Birkhäuser Verlag AG.

Jones, J. C. (1970). Design methods: Seeds of human futures. New York, NY: John Wiley & Sons Ltd.

Joseph, D. (2004). The practice of design-based research: uncovering the interplay between design, research, and the real-world context. Educational Psychologist, 39 (4), 235–242.

Kelly, A. E. (2003). Theme issue: The role of design in educational research. Educational Researcher, 32 (1), 3–4. doi:10.3102/0013189X032001003

Margolin, V. (2010). Design research: Towards a history. Presented at the Design Research Society Annual Conference on Design & Complexity, Montreal, Canada. Retrieved from http://www.drs2010.umontreal.ca/data/PDF/080.pdf

McCandliss, B. D., Kalchman, M., & Bryant, P. (2003). Design experiments and laboratory approaches to learning: Steps toward collaborative exchange. Educational Researcher, 32 (1), 14–16. doi:10.3102/0013189X032001014

Michel, R. (Ed.). (2007). Design research now. Basel, Switzerland: Birkhäuser Verlag AG

Oh, E., & Reeves, T. C. (2010). The implications of the differences between design research and instructional systems design for educational technology researchers and practitioners. Educational Media International, 47 (4), 263–275.

Reeves, T. C. (2006). Design research from a technology perspective. In J. van den Akker, K. Gravemeijer, S. McKenney, & N. Nieveen (Eds.), Educational design research (Vol. 1, pp. 52–66). London, England: Routledge.

Reigeluth, C. M., & Frick, T. W. (1999). Formative research: A methodology for creating and improving design theories. In C. Reigeluth (Ed.), Instructional-design theories and models. A new paradigm of instructional theory (Vol. 2) (pp. 633–651), Mahwah, NJ: Lawrence Erlbaum Associates.

Richey, R. C., & Nelson, W. A. (1996). Developmental research. In D. Jonassen (Ed.), Handbook of research for educational communications and technology (pp. 1213–1245), London, England: Macmillan.

Sandoval, W. A., & Bell, P. (2004). Design-based research methods for studying learning in context: Introduction. Educational Psychologist, 39 (4), 199–201.

Schneider, B. (2007). Design as practice, science and research. In R. Michel (Ed.), Design research now (pp. 207–218). Basel, Switzerland: Birkhäuser Verlag AG.

Shavelson, R. J., Phillips, D. C., Towne, L., & Feuer, M. J. (2003). On the science of education design studies. Educational Researcher, 32 (1), 25–28. doi:10.3102/0013189X032001025

Simon, H. A. (1969). The sciences of the artificial. Cambridge, MA: The MIT Press.

Tabak, I. (2004). Reconstructing context: Negotiating the tension between exogenous and endogenous educational design. Educational Psychologist, 39 (4), 225–233.

van den Akker, J. (1999). Principles and methods of development research. In J. van den Akker, R. M. Branch, K. Gustafson, N. Nieveen, & T. Plomp (Eds.), Design approaches and tools in education and training (pp. 1–14). Norwell, MA: Kluwer Academic Publishers.

van den Akker, J., & Plomp, T. (1993). Development research in curriculum: Propositions and experiences. Paper presented at the annual meeting of the American Educational Research Association, April 12–14, Atlanta, GA.

Walker, D.F., (1992). Methodological issues in curriculum research, In Jackson, P. (Ed.), Handbook of research on curriculum (pp. 98–118). New York, NY: Macmillan.

Walker, D. & Bresler, L. (1993). Development research: Definitions, methods, and criteria.  Paper presented at the annual meeting of the American Educational Research Association, April 12–16, Atlanta, GA.

Willemien, V. (2009). Design: One, but in different forms. Design Studies, 30 (3), 187–223. doi:10.1016/j.destud.2008.11.004

Further Video Resource

Video available at  http://bit.ly/WestDBRX

Foundations of Learning and Instructional Design Technology Copyright © 2018 by Kimberly Christensen and Richard E. West is licensed under a Creative Commons Attribution 4.0 International License , except where otherwise noted.

Share This Book

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is a Research Design | Types, Guide & Examples

What Is a Research Design | Types, Guide & Examples

Published on June 7, 2021 by Shona McCombes . Revised on November 20, 2023 by Pritha Bhandari.

A research design is a strategy for answering your   research question  using empirical data. Creating a research design means making decisions about:

• Your overall research objectives and approach
• Whether you’ll rely on primary research or secondary research
• Your sampling methods or criteria for selecting subjects
• Your data collection methods
• The procedures you’ll follow to collect data
• Your data analysis methods

A well-planned research design helps ensure that your methods match your research objectives and that you use the right kind of analysis for your data.

Table of contents

Step 1: consider your aims and approach, step 2: choose a type of research design, step 3: identify your population and sampling method, step 4: choose your data collection methods, step 5: plan your data collection procedures, step 6: decide on your data analysis strategies, other interesting articles, frequently asked questions about research design.

• Introduction

Before you can start designing your research, you should already have a clear idea of the research question you want to investigate.

There are many different ways you could go about answering this question. Your research design choices should be driven by your aims and priorities—start by thinking carefully about what you want to achieve.

The first choice you need to make is whether you’ll take a qualitative or quantitative approach.

Qualitative research designs tend to be more flexible and inductive , allowing you to adjust your approach based on what you find throughout the research process.

Quantitative research designs tend to be more fixed and deductive , with variables and hypotheses clearly defined in advance of data collection.

It’s also possible to use a mixed-methods design that integrates aspects of both approaches. By combining qualitative and quantitative insights, you can gain a more complete picture of the problem you’re studying and strengthen the credibility of your conclusions.

Practical and ethical considerations when designing research

As well as scientific considerations, you need to think practically when designing your research. If your research involves people or animals, you also need to consider research ethics .

• How much time do you have to collect data and write up the research?
• Will you be able to gain access to the data you need (e.g., by travelling to a specific location or contacting specific people)?
• Do you have the necessary research skills (e.g., statistical analysis or interview techniques)?
• Will you need ethical approval ?

At each stage of the research design process, make sure that your choices are practically feasible.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Within both qualitative and quantitative approaches, there are several types of research design to choose from. Each type provides a framework for the overall shape of your research.

Types of quantitative research designs

Quantitative designs can be split into four main types.

• Experimental and   quasi-experimental designs allow you to test cause-and-effect relationships
• Descriptive and correlational designs allow you to measure variables and describe relationships between them.

With descriptive and correlational designs, you can get a clear picture of characteristics, trends and relationships as they exist in the real world. However, you can’t draw conclusions about cause and effect (because correlation doesn’t imply causation ).

Experiments are the strongest way to test cause-and-effect relationships without the risk of other variables influencing the results. However, their controlled conditions may not always reflect how things work in the real world. They’re often also more difficult and expensive to implement.

Types of qualitative research designs

Qualitative designs are less strictly defined. This approach is about gaining a rich, detailed understanding of a specific context or phenomenon, and you can often be more creative and flexible in designing your research.

The table below shows some common types of qualitative design. They often have similar approaches in terms of data collection, but focus on different aspects when analyzing the data.

Your research design should clearly define who or what your research will focus on, and how you’ll go about choosing your participants or subjects.

In research, a population is the entire group that you want to draw conclusions about, while a sample is the smaller group of individuals you’ll actually collect data from.

Defining the population

A population can be made up of anything you want to study—plants, animals, organizations, texts, countries, etc. In the social sciences, it most often refers to a group of people.

For example, will you focus on people from a specific demographic, region or background? Are you interested in people with a certain job or medical condition, or users of a particular product?

The more precisely you define your population, the easier it will be to gather a representative sample.

• Sampling methods

Even with a narrowly defined population, it’s rarely possible to collect data from every individual. Instead, you’ll collect data from a sample.

To select a sample, there are two main approaches: probability sampling and non-probability sampling . The sampling method you use affects how confidently you can generalize your results to the population as a whole.

Probability sampling is the most statistically valid option, but it’s often difficult to achieve unless you’re dealing with a very small and accessible population.

For practical reasons, many studies use non-probability sampling, but it’s important to be aware of the limitations and carefully consider potential biases. You should always make an effort to gather a sample that’s as representative as possible of the population.

Case selection in qualitative research

In some types of qualitative designs, sampling may not be relevant.

For example, in an ethnography or a case study , your aim is to deeply understand a specific context, not to generalize to a population. Instead of sampling, you may simply aim to collect as much data as possible about the context you are studying.

In these types of design, you still have to carefully consider your choice of case or community. You should have a clear rationale for why this particular case is suitable for answering your research question .

For example, you might choose a case study that reveals an unusual or neglected aspect of your research problem, or you might choose several very similar or very different cases in order to compare them.

Data collection methods are ways of directly measuring variables and gathering information. They allow you to gain first-hand knowledge and original insights into your research problem.

You can choose just one data collection method, or use several methods in the same study.

Survey methods

Surveys allow you to collect data about opinions, behaviors, experiences, and characteristics by asking people directly. There are two main survey methods to choose from: questionnaires and interviews .

Observation methods

Observational studies allow you to collect data unobtrusively, observing characteristics, behaviors or social interactions without relying on self-reporting.

Observations may be conducted in real time, taking notes as you observe, or you might make audiovisual recordings for later analysis. They can be qualitative or quantitative.

Other methods of data collection

There are many other ways you might collect data depending on your field and topic.

If you’re not sure which methods will work best for your research design, try reading some papers in your field to see what kinds of data collection methods they used.

Secondary data

If you don’t have the time or resources to collect data from the population you’re interested in, you can also choose to use secondary data that other researchers already collected—for example, datasets from government surveys or previous studies on your topic.

With this raw data, you can do your own analysis to answer new research questions that weren’t addressed by the original study.

Using secondary data can expand the scope of your research, as you may be able to access much larger and more varied samples than you could collect yourself.

However, it also means you don’t have any control over which variables to measure or how to measure them, so the conclusions you can draw may be limited.

As well as deciding on your methods, you need to plan exactly how you’ll use these methods to collect data that’s consistent, accurate, and unbiased.

Planning systematic procedures is especially important in quantitative research, where you need to precisely define your variables and ensure your measurements are high in reliability and validity.

Operationalization

Some variables, like height or age, are easily measured. But often you’ll be dealing with more abstract concepts, like satisfaction, anxiety, or competence. Operationalization means turning these fuzzy ideas into measurable indicators.

If you’re using observations , which events or actions will you count?

If you’re using surveys , which questions will you ask and what range of responses will be offered?

You may also choose to use or adapt existing materials designed to measure the concept you’re interested in—for example, questionnaires or inventories whose reliability and validity has already been established.

Reliability and validity

Reliability means your results can be consistently reproduced, while validity means that you’re actually measuring the concept you’re interested in.

For valid and reliable results, your measurement materials should be thoroughly researched and carefully designed. Plan your procedures to make sure you carry out the same steps in the same way for each participant.

If you’re developing a new questionnaire or other instrument to measure a specific concept, running a pilot study allows you to check its validity and reliability in advance.

Sampling procedures

As well as choosing an appropriate sampling method , you need a concrete plan for how you’ll actually contact and recruit your selected sample.

That means making decisions about things like:

• How many participants do you need for an adequate sample size?
• What inclusion and exclusion criteria will you use to identify eligible participants?
• How will you contact your sample—by mail, online, by phone, or in person?

If you’re using a probability sampling method , it’s important that everyone who is randomly selected actually participates in the study. How will you ensure a high response rate?

If you’re using a non-probability method , how will you avoid research bias and ensure a representative sample?

Data management

It’s also important to create a data management plan for organizing and storing your data.

Will you need to transcribe interviews or perform data entry for observations? You should anonymize and safeguard any sensitive data, and make sure it’s backed up regularly.

Keeping your data well-organized will save time when it comes to analyzing it. It can also help other researchers validate and add to your findings (high replicability ).

On its own, raw data can’t answer your research question. The last step of designing your research is planning how you’ll analyze the data.

Quantitative data analysis

In quantitative research, you’ll most likely use some form of statistical analysis . With statistics, you can summarize your sample data, make estimates, and test hypotheses.

Using descriptive statistics , you can summarize your sample data in terms of:

• The distribution of the data (e.g., the frequency of each score on a test)
• The central tendency of the data (e.g., the mean to describe the average score)
• The variability of the data (e.g., the standard deviation to describe how spread out the scores are)

The specific calculations you can do depend on the level of measurement of your variables.

Using inferential statistics , you can:

• Make estimates about the population based on your sample data.
• Test hypotheses about a relationship between variables.

Regression and correlation tests look for associations between two or more variables, while comparison tests (such as t tests and ANOVAs ) look for differences in the outcomes of different groups.

Your choice of statistical test depends on various aspects of your research design, including the types of variables you’re dealing with and the distribution of your data.

Qualitative data analysis

In qualitative research, your data will usually be very dense with information and ideas. Instead of summing it up in numbers, you’ll need to comb through the data in detail, interpret its meanings, identify patterns, and extract the parts that are most relevant to your research question.

Two of the most common approaches to doing this are thematic analysis and discourse analysis .

There are many other ways of analyzing qualitative data depending on the aims of your research. To get a sense of potential approaches, try reading some qualitative research papers in your field.

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A research design is a strategy for answering your   research question . It defines your overall approach and determines how you will collect and analyze data.

A well-planned research design helps ensure that your methods match your research aims, that you collect high-quality data, and that you use the right kind of analysis to answer your questions, utilizing credible sources . This allows you to draw valid , trustworthy conclusions.

Quantitative research designs can be divided into two main categories:

• Correlational and descriptive designs are used to investigate characteristics, averages, trends, and associations between variables.
• Experimental and quasi-experimental designs are used to test causal relationships .

Qualitative research designs tend to be more flexible. Common types of qualitative design include case study , ethnography , and grounded theory designs.

The priorities of a research design can vary depending on the field, but you usually have to specify:

• Your research questions and/or hypotheses
• Your overall approach (e.g., qualitative or quantitative )
• The type of design you’re using (e.g., a survey , experiment , or case study )
• Your data collection methods (e.g., questionnaires , observations)
• Your data collection procedures (e.g., operationalization , timing and data management)
• Your data analysis methods (e.g., statistical tests  or thematic analysis )

A sample is a subset of individuals from a larger population . Sampling means selecting the group that you will actually collect data from in your research. For example, if you are researching the opinions of students in your university, you could survey a sample of 100 students.

In statistics, sampling allows you to test a hypothesis about the characteristics of a population.

Operationalization means turning abstract conceptual ideas into measurable observations.

For example, the concept of social anxiety isn’t directly observable, but it can be operationally defined in terms of self-rating scores, behavioral avoidance of crowded places, or physical anxiety symptoms in social situations.

Before collecting data , it’s important to consider how you will operationalize the variables that you want to measure.

A research project is an academic, scientific, or professional undertaking to answer a research question . Research projects can take many forms, such as qualitative or quantitative , descriptive , longitudinal , experimental , or correlational . What kind of research approach you choose will depend on your topic.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, November 20). What Is a Research Design | Types, Guide & Examples. Scribbr. Retrieved March 26, 2024, from https://www.scribbr.com/methodology/research-design/

Is this article helpful?

Shona McCombes

Other students also liked, guide to experimental design | overview, steps, & examples, how to write a research proposal | examples & templates, ethical considerations in research | types & examples, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

Change Password

Your password must have 8 characters or more and contain 3 of the following:.

• a lower case character, 
• an upper case character, 
• a special character 

Password Changed Successfully

Your password has been changed

• Sign in / Register

Request Username

Can't sign in? Forgot your username?

Enter your email address below and we will send you your username

If the address matches an existing account you will receive an email with instructions to retrieve your username

Design-Based Research: A Methodology to Extend and Enrich Biology Education Research

• Emily E. Scott
• Mary Pat Wenderoth
• Jennifer H. Doherty

*Address correspondence to: Emily E. Scott ( E-mail Address: [email protected] ).

Department of Biology, University of Washington, Seattle, WA 98195

Search for more papers by this author

Recent calls in biology education research (BER) have recommended that researchers leverage learning theories and methodologies from other disciplines to investigate the mechanisms by which students to develop sophisticated ideas. We suggest design-based research from the learning sciences is a compelling methodology for achieving this aim. Design-based research investigates the “learning ecologies” that move student thinking toward mastery. These “learning ecologies” are grounded in theories of learning, produce measurable changes in student learning, generate design principles that guide the development of instructional tools, and are enacted using extended, iterative teaching experiments. In this essay, we introduce readers to the key elements of design-based research, using our own research into student learning in undergraduate physiology as an example of design-based research in BER. Then, we discuss how design-based research can extend work already done in BER and foster interdisciplinary collaborations among cognitive and learning scientists, biology education researchers, and instructors. We also explore some of the challenges associated with this methodological approach.

INTRODUCTION

There have been recent calls for biology education researchers to look toward other fields of educational inquiry for theories and methodologies to advance, and expand, our understanding of what helps students learn to think like biologists ( Coley and Tanner, 2012 ; Dolan, 2015 ; Peffer and Renken, 2016 ; Lo et al. , 2019 ). These calls include the recommendations that biology education researchers ground their work in learning theories from the cognitive and learning sciences ( Coley and Tanner, 2012 ) and begin investigating the underlying mechanisms by which students to develop sophisticated biology ideas ( Dolan, 2015 ; Lo et al. , 2019 ). Design-based research from the learning sciences is one methodology that seeks to do both by using theories of learning to investigate how “learning ecologies”—that is, complex systems of interactions among instructors, students, and environmental components—support the process of student learning ( Brown, 1992 ; Cobb et al. , 2003 ; Collins et al. , 2004 ; Peffer and Renken, 2016 ).

The purpose of this essay is twofold. First, we want to introduce readers to the key elements of design-based research, using our research into student learning in undergraduate physiology as an example of design-based research in biology education research (BER). Second, we will discuss how design-based research can extend work already done in BER and explore some of the challenges of its implementation. For a more in-depth review of design-based research, we direct readers to the following references: Brown (1992) , Barab and Squire (2004) , and Collins et al. (2004) , as well as commentaries by Anderson and Shattuck (2012) and McKenney and Reeves (2013) .

WHAT IS DESIGN-BASED RESEARCH?

Design-based research is a methodological approach that aligns with research methods from the fields of engineering or applied physics, where products are designed for specific purposes ( Brown, 1992 ; Joseph, 2004 ; Middleton et al. , 2008 ; Kelly, 2014 ). Consequently, investigators using design-based research approach educational inquiry much as an engineer develops a new product: First, the researchers identify a problem that needs to be addressed (e.g., a particular learning challenge that students face). Next, they design a potential “solution” to the problem in the form of instructional tools (e.g., reasoning strategies, worksheets; e.g., Reiser et al. , 2001 ) that theory and previous research suggest will address the problem. Then, the researchers test the instructional tools in a real-world setting (i.e., the classroom) to see if the tools positively impact student learning. As testing proceeds, researchers evaluate the instructional tools with emerging evidence of their effectiveness (or lack thereof) and progressively revise the tools— in real time —as necessary ( Collins et al. , 2004 ). Finally, the researchers reflect on the outcomes of the experiment, identifying the features of the instructional tools that were successful at addressing the initial learning problem, revising those aspects that were not helpful to learning, and determining how the research informed the theory underlying the experiment. This leads to another research cycle of designing, testing, evaluating, and reflecting to refine the instructional tools in support of student learning. We have characterized this iterative process in Figure 1 after Sandoval (2014) . Though we have portrayed four discrete phases to design-based research, there is often overlap of the phases as the research progresses (e.g., testing and evaluating can occur simultaneously).

FIGURE 1. The four phases of design-based research experienced in an iterative cycle (A). We also highlight the main features of each phase of our design-based research project investigating students’ use of flux in physiology (B).

Design-based research has no specific requirements for the form that instructional tools must take or the manner in which the tools are evaluated ( Bell, 2004 ; Anderson and Shattuck, 2012 ). Instead, design-based research has what Sandoval (2014) calls “epistemic commitments” 1 that inform the major goals of a design-based research project as well as how it is implemented. These epistemic commitments are: 1) Design based research should be grounded in theories of learning (e.g., constructivism, knowledge-in-pieces, conceptual change) that both inform the design of the instructional tools and are improved upon by the research ( Cobb et al. , 2003 ; Barab and Squire, 2004 ). This makes design-based research more than a method for testing whether or not an instructional tool works; it also investigates why the design worked and how it can be generalized to other learning environments ( Cobb et al. , 2003 ). 2) Design-based research should aim to produce measurable changes in student learning in classrooms around a particular learning problem ( Anderson and Shattuck, 2012 ; McKenney and Reeves, 2013 ). This requirement ensures that theoretical research into student learning is directly applicable, and impactful, to students and instructors in classroom settings ( Hoadley, 2004 ). 3) Design-based research should generate design principles that guide the development and implementation of future instructional tools ( Edelson, 2002 ). This commitment makes the research findings broadly applicable for use in a variety of classroom environments. 4) Design-based research should be enacted using extended, iterative teaching experiments in classrooms. By observing student learning over an extended period of time (e.g., throughout an entire term or across terms), researchers are more likely to observe the full effects of how the instructional tools impact student learning compared with short-term experiments ( Brown, 1992 ; Barab and Squire, 2004 ; Sandoval and Bell, 2004 ).

HOW IS DESIGN-BASED RESEARCH DIFFERENT FROM AN EXPERIMENTAL APPROACH?

Many BER studies employ experimental approaches that align with traditional scientific methods of experimentation, such as using treatment versus control groups, randomly assigning treatments to different groups, replicating interventions across multiple spatial or temporal periods, and using statistical methods to guide the kinds of inferences that arise from an experiment. While design-based research can similarly employ these strategies for educational inquiry, there are also some notable differences in its approach to experimentation ( Collins et al. , 2004 ; Hoadley, 2004 ). In this section, we contrast the differences between design-based research and what we call “experimental approaches,” although both paradigms represent a form of experimentation.

The first difference between an experimental approach and design-based research regards the role participants play in the experiment. In an experimental approach, the researcher is responsible for making all the decisions about how the experiment will be implemented and analyzed, while the instructor facilitates the experimental treatments. In design-based research, both researchers and instructors are engaged in all stages of the research from conception to reflection ( Collins et al. , 2004 ). In BER, a third condition frequently arises wherein the researcher is also the instructor. In this case, if the research questions being investigated produce generalizable results that have the potential to impact teaching broadly, then this is consistent with a design-based research approach ( Cobb et al. , 2003 ). However, when the research questions are self-reflective about how a researcher/instructor can improve his or her own classroom practices, this aligns more closely with “action research,” which is another methodology used in education research (see Stringer, 2013 ).

A second difference between experimental research and design-based research is the form that hypotheses take and the manner in which they are investigated ( Collins et al. , 2004 ; Sandoval, 2014 ). In experimental approaches, researchers develop a hypothesis about how a specific instructional intervention will impact student learning. The intervention is then tested in the classroom(s) while controlling for other variables that are not part of the study in order to isolate the effects of the intervention. Sometimes, researchers designate a “control” situation that serves as a comparison group that does not experience the intervention. For example, Jackson et al. (2018) were interested in comparing peer- and self-grading of weekly practice exams to if they were equally effective forms of deliberate practice for students in a large-enrollment class. To test this, the authors (including authors of this essay J.H.D., M.P.W.) designed an experiment in which lab sections of students in a large lecture course were randomly assigned to either a peer-grading or self-grading treatment so they could isolate the effects of each intervention. In design-based research, a hypothesis is conceptualized as the “design solution” rather than a specific intervention; that is, design-based researchers hypothesize that the designed instructional tools, when implemented in the classroom, will create a learning ecology that improves student learning around the identified learning problem ( Edelson, 2002 ; Bell, 2004 ). For example, Zagallo et al. (2016) developed a laboratory curriculum (i.e., the hypothesized “design solution”) for molecular and cellular biology majors to address the learning problem that students often struggle to connect scientific models and empirical data. This curriculum entailed: focusing instruction around a set of target biological models; developing small-group activities in which students interacted with the models by analyzing data from scientific papers; using formative assessment tools for student feedback; and providing students with a set of learning objectives they could use as study tools. They tested their curriculum in a novel, large-enrollment course of upper-division students over several years, making iterative changes to the curriculum as the study progressed.

By framing the research approach as an iterative endeavor of progressive refinement rather than a test of a particular intervention when all other variables are controlled, design-based researchers recognize that: 1) classrooms, and classroom experiences, are unique at any given time, making it difficult to truly “control” the environment in which an intervention occurs or establish a “control group” that differs only in the features of an intervention; and 2) many aspects of a classroom experience may influence the effectiveness of an intervention, often in unanticipated ways, which should be included in the research team’s analysis of an intervention’s success. Consequently, the research team is less concerned with controlling the research conditions—as in an experimental approach—and instead focuses on characterizing the learning environment ( Barab and Squire, 2004 ). This involves collecting data from multiple sources as the research progresses, including how the instructional tools were implemented, aspects of the implementation process that failed to go as planned, and how the instructional tools or implementation process was modified. These characterizations can provide important insights into what specific features of the instructional tools, or the learning environment, were most impactful to learning ( DBR Collective, 2003 ).

A third difference between experimental approaches and design-based research is when the instructional interventions can be modified. In experimental research, the intervention is fixed throughout the experimental period, with any revisions occurring only after the experiment has concluded. This is critical for ensuring that the results of the study provide evidence of the efficacy of a specific intervention. By contrast, design-based research takes a more flexible approach that allows instructional tools to be modified in situ as they are being implemented ( Hoadley, 2004 ; Barab, 2014 ). This flexibility allows the research team to modify instructional tools or strategies that prove inadequate for collecting the evidence necessary to evaluate the underlying theory and ensures a tight connection between interventions and a specific learning problem ( Collins et al. , 2004 ; Hoadley, 2004 ).

Finally, and importantly, experimental approaches and design-based research differ in the kinds of conclusions they draw from their data. Experimental research can “identify that something meaningful happened; but [it is] not able to articulate what about the intervention caused that story to unfold” ( Barab, 2014 , p. 162). In other words, experimental methods are robust for identifying where differences in learning occur, such as between groups of students experiencing peer- or self-grading of practice exams ( Jackson et al. , 2018 ) or receiving different curricula (e.g., Chi et al. , 2012 ). However, these methods are not able to characterize the underlying learning process or mechanism involved in the different learning outcomes. By contrast, design-based research has the potential to uncover mechanisms of learning, because it investigates how the nature of student thinking changes as students experience instructional interventions ( Shavelson et al. , 2003 ; Barab, 2014 ). According to Sandoval (2014) , “Design research, as a means of uncovering causal processes, is oriented not to finding effects but to finding functions , to understanding how desired (and undesired) effects arise through interactions in a designed environment” (p. 30). In Zagallo et al. (2016) , the authors found that their curriculum supported students’ data-interpretation skills, because it stimulated students’ spontaneous use of argumentation during which group members coconstructed evidence-based claims from the data provided. Students also worked collaboratively to decode figures and identify data patterns. These strategies were identified from the researchers’ qualitative data analysis of in-class recordings of small-group discussions, which allowed them to observe what students were doing to support their learning. Because design-based research is focused on characterizing how learning occurs in classrooms, it can begin to answer the kinds of mechanistic questions others have identified as central to advancing BER ( National Research Council [NRC], 2012 ; Dolan, 2015 ; Lo et al. , 2019 ).

DESIGN-BASED RESEARCH IN ACTION: AN EXAMPLE FROM UNDERGRADUATE PHYSIOLOGY

To illustrate how design-based research could be employed in BER, we draw on our own research that investigates how students learn physiology. We will characterize one iteration of our design-based research cycle ( Figure 1 ), emphasizing how our project uses Sandoval’s four epistemic commitments (i.e., theory driven, practically applied, generating design principles, implemented in an iterative manner) to guide our implementation.

Identifying the Learning Problem

Understanding physiological phenomena is challenging for students, given the wide variety of contexts (e.g., cardiovascular, neuromuscular, respiratory; animal vs. plant) and scales involved (e.g., using molecular-level interactions to explain organism functioning; Wang, 2004 ; Michael, 2007 ; Badenhorst et al. , 2016 ). To address these learning challenges, Modell (2000) identified seven “general models” that undergird most physiology phenomena (i.e., control systems, conservation of mass, mass and heat flow, elastic properties of tissues, transport across membranes, cell-to-cell communication, molecular interactions). Instructors can use these models as a “conceptual framework” to help students build intellectual coherence across phenomena and develop a deeper understanding of physiology ( Modell, 2000 ; Michael et al. , 2009 ). This approach aligns with theoretical work in the learning sciences that indicates that providing students with conceptual frameworks improves their ability to integrate and retrieve knowledge ( National Academies of Sciences, Engineering, and Medicine, 2018 ).

Before the start of our design-based project, we had been using Modell’s (2000) general models to guide our instruction. In this essay, we will focus on how we used the general models of mass and heat flow and transport across membranes in our instruction. These two models together describe how materials flow down gradients (e.g., pressure gradients, electrochemical gradients) against sources of resistance (e.g., tube diameter, channel frequency). We call this flux reasoning. We emphasized the fundamental nature and broad utility of flux reasoning in lecture and lab and frequently highlighted when it could be applied to explain a phenomenon. We also developed a conceptual scaffold (the Flux Reasoning Tool) that students could use to reason about physiological processes involving flux.

Although these instructional approaches had improved students’ understanding of flux phenomena, we found that students often demonstrated little commitment to using flux broadly across physiological contexts. Instead, they considered flux to be just another fact to memorize and applied it to narrow circumstances (e.g., they would use flux to reason about ions flowing across membranes—the context where flux was first introduced—but not the bulk flow of blood in a vessel). Students also struggled to integrate the various components of flux (e.g., balancing chemical and electrical gradients, accounting for variable resistance). We saw these issues reflected in students’ lower than hoped for exam scores on the cumulative final of the course. From these experiences, and from conversations with other physiology instructors, we identified a learning problem to address through design-based research: How do students learn to use flux reasoning to explain material flows in multiple physiology contexts?

The process of identifying a learning problem usually emerges from a researcher’s own experiences (in or outside a classroom) or from previous research that has been described in the literature ( Cobb et al. , 2003 ). To remain true to Sandoval’s first epistemic commitment, a learning problem must advance a theory of learning ( Edelson, 2002 ; McKenney and Reeves, 2013 ). In our work, we investigated how conceptual frameworks based on fundamental scientific concepts (i.e., Modell’s general models) could help students reason productively about physiology phenomena (National Academies of Sciences, Engineering, and Medicine, 2018; Modell, 2000 ). Our specific theoretical question was: Can we characterize how students’ conceptual frameworks around flux change as they work toward robust ideas? Sandoval’s second epistemic commitment stated that a learning problem must aim to improve student learning outcomes. The practical significance of our learning problem was: Does using the concept of flux as a foundational idea for instructional tools increase students’ learning of physiological phenomena?

We investigated our learning problem in an introductory biology course at a large R1 institution. The introductory course is the third in a biology sequence that focuses on plant and animal physiology. The course typically serves between 250 and 600 students in their sophomore or junior years each term. Classes have the following average demographics: 68% male, 21% from lower-income situations, 12% from an underrepresented minority, and 26% first-generation college students.

Design-Based Research Cycle 1, Phase 1: Designing Instructional Tools

The first phase of design-based research involves developing instructional tools that address both the theoretical and practical concerns of the learning problem ( Edelson, 2002 ; Wang and Hannafin, 2005 ). These instructional tools can take many forms, such as specific instructional strategies, classroom worksheets and practices, or technological software, as long as they embody the underlying learning theory being investigated. They must also produce classroom experiences or materials that can be evaluated to determine whether learning outcomes were met ( Sandoval, 2014 ). Indeed, this alignment between theory, the nature of the instructional tools, and the ways students are assessed is central to ensuring rigorous design-based research ( Hoadley, 2004 ; Sandoval, 2014 ). Taken together, the instructional tools instantiate a hypothesized learning environment that will advance both the theoretical and practical questions driving the research ( Barab, 2014 ).

In our work, the theoretical claim that instruction based on fundamental scientific concepts would support students’ flux reasoning was embodied in our instructional approach by being the central focus of all instructional materials, which included: a revised version of the Flux Reasoning Tool ( Figure 2 ); case study–based units in lecture that explicitly emphasized flux phenomena in real-world contexts ( Windschitl et al. , 2012 ; Scott et al. , 2018 ; Figure 3 ); classroom activities in which students practiced using flux to address physiological scenarios; links to online videos describing key flux-related concepts; constructed-response assessment items that cued students to use flux reasoning in their thinking; and pretest/posttest formative assessment questions that tracked student learning ( Figure 4 ).

FIGURE 2. The Flux Reasoning Tool given to students at the beginning of the quarter.

FIGURE 3. An example flux case study that is presented to students at the beginning of the neurophysiology unit. Throughout the unit, students learn how ion flows into and out of cells, as mediated by chemical and electrical gradients and various ion/molecular channels, sends signals throughout the body. They use this information to better understand why Jaime experiences persistent neuropathy. Images from: uz.wikipedia.org/wiki/Fayl:Blausen_0822_SpinalCord.png and commons.wikimedia.org/wiki/File:Figure_38_01_07.jpg.

FIGURE 4. An example flux assessment question about ion flows given in a pre-unit/post-unit formative assessment in the neurophysiology unit.

Phase 2: Testing the Instructional Tools

In the second phase of design-based research, the instructional tools are tested by implementing them in classrooms. During this phase, the instructional tools are placed “in harm’s way … in order to expose the details of the process to scrutiny” ( Cobb et al. , 2003 , p. 10). In this way, researchers and instructors test how the tools perform in real-world settings, which may differ considerably from the design team’s initial expectations ( Hoadley, 2004 ). During this phase, if necessary, the design team may make adjustments to the tools as they are being used to account for these unanticipated conditions ( Collins et al. , 2004 ).

We implemented the instructional tools during the Autumn and Spring quarters of the 2016–2017 academic year. Students were taught to use the Flux Reasoning Tool at the beginning of the term in the context of the first case study unit focused on neurophysiology. Each physiology unit throughout the term was associated with a new concept-based case study (usually about flux) that framed the context of the teaching. Embedded within the daily lectures were classroom activities in which students could practice using flux. Students were also assigned readings from the textbook and videos related to flux to watch during each unit. Throughout the term, students took five exams that each contained some flux questions as well as some pre- and post-unit formative assessment questions. During Winter quarter, we conducted clinical interviews with students who would take our course in the Spring term (i.e., “pre” data) as well as students who had just completed our course in Autumn (i.e., “post” data).

Phase 3: Evaluating the Instructional Tools

The third phase of a design-based research cycle involves evaluating the effectiveness of instructional tools using evidence of student learning ( Barab and Squire, 2004 ; Anderson and Shattuck, 2012 ). This can be done using products produced by students (e.g., homework, lab reports), attitudinal gains measured with surveys, participation rates in activities, interview testimonials, classroom discourse practices, and formative assessment or exam data (e.g., Reiser et al. , 2001 ; Cobb et al. , 2003 ; Barab and Squire, 2004 ; Mohan et al. , 2009 ). Regardless of the source, evidence must be in a form that supports a systematic analysis that could be scrutinized by other researchers ( Cobb et al. , 2003 ; Barab, 2014 ). Also, because design-based research often involves multiple data streams, researchers may need to use both quantitative and qualitative analytical methods to produce a rich picture of how the instructional tools affected student learning ( Collins et al. , 2004 ; Anderson and Shattuck, 2012 ).

In our work, we used the quality of students’ written responses on exams and formative assessment questions to determine whether students improved their understanding of physiological phenomena involving flux. For each assessment question, we analyzed a subset of student’s pretest answers to identify overarching patterns in students’ reasoning about flux, characterized these overarching patterns, then ordinated the patterns into different levels of sophistication. These became our scoring rubrics, which identified five different levels of student reasoning about flux. We used the rubrics to code the remainder of students’ responses, with a code designating the level of student reasoning associated with a particular reasoning pattern. We used this ordinal rubric format because it would later inform our theoretical understanding of how students build flux conceptual frameworks (see phase 4). This also allowed us to both characterize the ideas students held about flux phenomena and identify the frequency distribution of those ideas in a class.

By analyzing changes in the frequency distributions of students’ ideas across the rubric levels at different time points in the term (e.g., pre-unit vs. post-unit), we could track both the number of students who gained more sophisticated ideas about flux as the term progressed and the quality of those ideas. If the frequency of students reasoning at higher levels increased from pre-unit to post-unit assessments, we could conclude that our instructional tools as a whole were supporting students’ development of sophisticated flux ideas. For example, on one neuromuscular ion flux assessment question in the Spring of 2017, we found that relatively more students were reasoning at the highest levels of our rubric (i.e., levels 4 and 5) on the post-unit test compared with the pre-unit test. This meant that more students were beginning to integrate sophisticated ideas about flux (i.e., they were balancing concentration and electrical gradients) in their reasoning about ion movement.

To help validate this finding, we drew on three additional data streams: 1) from in-class group recordings of students working with flux items, we noted that students increasingly incorporated ideas about gradients and resistance when constructing their explanations as the term progressed; 2) from plant assessment items in the latter part of the term, we began to see students using flux ideas unprompted; and 3) from interviews, we observed that students who had already taken the course used flux ideas in their reasoning.

Through these analyses, we also noticed an interesting pattern in the pre-unit test data for Spring 2017 when compared with the frequency distribution of students’ responses with a previous term (Autumn 2016). In Spring 2017, 42% of students reasoned at level 4 or 5 on the pre-unit test, indicating these students already had sophisticated ideas about ion flux before they took the pre-unit assessment. This was surprising, considering only 2% of students reasoned at these levels for this item on the Autumn 2016 pre-unit test.

Phase 4: Reflecting on the Instructional Tools and Their Implementation

The final phase of a design-based research cycle involves a retrospective analysis that addresses the epistemic commitments of this methodology: How was the theory underpinning the research advanced by the research endeavor (theoretical outcome)? Did the instructional tools support student learning about the learning problem (practical outcome)? What were the critical features of the design solution that supported student learning (design principles)? ( Cobb et al. , 2003 ; Barab and Squire, 2004 ).

Theoretical Outcome (Epistemic Commitment 1).

Reflecting on how a design-based research experiment advances theory is critical to our understanding of how students learn in educational settings ( Barab and Squire, 2004 ; Mohan et al. , 2009 ). In our work, we aimed to characterize how students’ conceptual frameworks around flux change as they work toward robust ideas. To do this, we drew on learning progression research as our theoretical framing ( NRC, 2007 ; Corcoran et al. , 2009 ; Duschl et al. , 2011 ; Scott et al. , 2019 ). Learning progression frameworks describe empirically derived patterns in student thinking that are ordered into levels representing cognitive shifts in the ways students conceive a topic as they work toward mastery ( Gunckel et al. , 2012 ). We used our ion flux scoring rubrics to create a preliminary five-level learning progression framework ( Table 1 ). The framework describes how students’ ideas about flux often start with teleological-driven accounts at the lowest level (i.e., level 1), shift to focusing on driving forces (e.g., concentration gradients, electrical gradients) in the middle levels, and arrive at complex ideas that integrate multiple interacting forces at the higher levels. We further validated these reasoning patterns with our student interviews. However, our flux conceptual framework was largely based on student responses to our ion flux assessment items. Therefore, to further validate our learning progression framework, we needed a greater diversity of flux assessment items that investigated student thinking more broadly (i.e., about bulk flow, water movement) across physiological systems.

Practical Outcome (Epistemic Commitment 2).

In design-based research, learning theories must “do real work” by improving student learning in real-world settings ( DBR Collective, 2003 ). Therefore, design-based researchers must reflect on whether or not the data they collected show evidence that the instructional tools improved student learning ( Cobb et al. , 2003 ; Sharma and McShane, 2008 ). We determined whether our flux-based instructional approach aided student learning by analyzing the kinds of answers students provided to our assessment questions. Specifically, we considered students who reasoned at level 4 or above as demonstrating productive flux reasoning. Because almost half of students were reasoning at level 4 or 5 on the post-unit assessment after experiencing the instructional tools in the neurophysiology unit (in Spring 2017), we concluded that our tools supported student learning in physiology. Additionally, we noticed that students used language in their explanations that directly tied to the Flux Reasoning Tool ( Figure 2 ), which instructed them to use arrows to indicate the magnitude and direction of gradient-driving forces. For example, in a posttest response to our ion flux item ( Figure 4 ), one student wrote:

Ion movement is a function of concentration and electrical gradients . Which arrow is stronger determines the movement of K+. We can make the electrical arrow bigger and pointing in by making the membrane potential more negative than Ek [i.e., potassium’s equilibrium potential]. We can make the concentration arrow bigger and pointing in by making a very strong concentration gradient pointing in.

Given that almost half of students reasoned at level 4 or above, and that students used language from the Flux Reasoning Tool, we concluded that using fundamental concepts was a productive instructional approach for improving student learning in physiology and that our instructional tools aided student learning. However, some students in the 2016–2017 academic year continued to apply flux ideas more narrowly than intended (i.e., for ion and simple diffusion cases, but not water flux or bulk flow). This suggested that students had developed nascent flux conceptual frameworks after experiencing the instructional tools but could use more support to realize the broad applicability of this principle. Also, although our cross-sectional interview approach demonstrated how students’ ideas, overall, could change after experiencing the instructional tools, it did not provide information about how a student developed flux reasoning.

Reflecting on practical outcomes also means interpreting any learning gains in the context of the learning ecology. This reflection allowed us to identify whether there were particular aspects of the instructional tools that were better at supporting learning than others ( DBR Collective, 2003 ). Indeed, this was critical for our understanding why 42% of students scored at level 3 and above on the pre-unit ion assessment in the Spring of 2017, while only 2% of students scored level 3 and above in Autumn of 2016. When we reviewed notes of the Spring 2017 implementation scheme, we saw that the pretest was due at the end of the first day of class after students had been exposed to ion flux ideas in class and in a reading/video assignment about ion flow, which may be one reason for the students’ high performance on the pretest. Consequently, we could not tell whether students’ initial high performance was due to their learning from the activities in the first day of class or for other reasons we did not measure. It also indicated we needed to close pretests before the first day of class for a more accurate measure of students’ incoming ideas and the effectiveness of the instructional tools employed at the beginning of the unit.

Design Principles (Epistemic Commitment 3).

Although design-based research is enacted in local contexts (i.e., a particular classroom), its purpose is to inform learning ecologies that have broad applications to improve learning and teaching ( Edelson, 2002 ; Cobb et al. , 2003 ). Therefore, design-based research should produce design principles that describe characteristics of learning environments that researchers and instructors can use to develop instructional tools specific to their local contexts (e.g., Edelson, 2002 ; Subramaniam et al. , 2015 ). Consequently, the design principles must balance specificity with adaptability so they can be used broadly to inform instruction ( Collins et al. , 2004 ; Barab, 2014 ).

From our first cycle of design-based research, we developed the following design principles: 1) Key scientific concepts should provide an overarching framework for course organization. This way, the individual components that make up a course, like instructional units, activities, practice problems, and assessments, all reinforce the centrality of the key concept. 2) Instructional tools should explicitly articulate the principle of interest, with specific guidance on how that principle is applied in context. This stresses the applied nature of the principle and that it is more than a fact to be memorized. 3) Instructional tools need to show specific instances of how the principle is applied in multiple contexts to combat students’ narrow application of the principle to a limited number of contexts.

Design-Based Research Cycle 2, Phase 1: Redesign and Refine the Experiment

The last “epistemic commitment” Sandoval (2014) articulated was that design-based research be an iterative process with an eye toward continually refining the instructional tools, based on evidence of student learning, to produce more robust learning environments. By viewing educational inquiry as formative research, design-based researchers recognize the difficulty in accounting for all variables that could impact student learning, or the implementation of the instructional tools, a priori ( Collins et al. , 2004 ). Robust instructional designs are the products of trial and error, which are strengthened by a systematic analysis of how they perform in real-world settings.

To continue to advance our work investigating student thinking using the principle of flux, we began a second cycle of design-based research that continued to address the learning problem of helping students reason with fundamental scientific concepts. In this cycle, we largely focused on broadening the number of physiological systems that had accompanying formative assessment questions (i.e., beyond ion flux), collecting student reasoning from a more diverse population of students (e.g., upper division, allied heath, community college), and refining and validating the flux learning progression with both written and interview data in a student through time. We developed a suite of constructed-response flux assessment questions that spanned neuromuscular, cardiovascular, respiratory, renal, and plant physiological contexts and asked students about several kinds of flux: ion movement, diffusion, water movement, and bulk flow (29 total questions; available at beyondmultiplechoice.org). This would provide us with rich qualitative data that we could use to refine the learning progression. We decided to administer written assessments and conduct interviews in a pretest/posttest manner at the beginning and end of each unit both as a way to increase our data about student reasoning and to provide students with additional practice using flux reasoning across contexts.

From this second round of designing instructional tools (i.e., broader range of assessment items), testing them in the classroom (i.e., administering the assessment items to diverse student populations), evaluating the tools (i.e., developing learning progression–aligned rubrics across phenomena from student data, tracking changes in the frequency distribution of students across levels through time), and reflecting on the tools’ success, we would develop a more thorough and robust characterization of how students use flux across systems that could better inform our creation of new instructional tools to support student learning.

HOW CAN DESIGN-BASED RESEARCH EXTEND AND ENRICH BER?

While design-based research has primarily been used in educational inquiry at the K–12 level (see Reiser et al. , 2001 ; Mohan et al. , 2009 ; Jin and Anderson, 2012 ), other science disciplines at undergraduate institutions have begun to employ this methodology to create robust instructional approaches (e.g., Szteinberg et al. , 2014 in chemistry; Hake, 2007 , and Sharma and McShane, 2008 , in physics; Kelly, 2014 , in engineering). Our own work, as well as that by Zagallo et al. (2016) , provides two examples of how design-based research could be implemented in BER. Below, we articulate some of the ways incorporating design-based research into BER could extend and enrich this field of educational inquiry.

Design-Based Research Connects Theory with Practice

One critique of BER is that it does not draw heavily enough on learning theories from other disciplines like cognitive psychology or the learning sciences to inform its research ( Coley and Tanner, 2012 ; Dolan, 2015 ; Peffer and Renken, 2016 ; Davidesco and Milne, 2019 ). For example, there has been considerable work in BER developing concept inventories as formative assessment tools that identify concepts students often struggle to learn (e.g., Marbach-Ad et al. , 2009 ; McFarland et al. , 2017 ; Summers et al. , 2018 ). However, much of this work is detached from a theoretical understanding of why students hold misconceptions in the first place, what the nature of their thinking is, and the learning mechanisms that would move students to a more productive understanding of domain ideas ( Alonzo, 2011 ). Using design-based research to understand the basis of students’ misconceptions would ground these practical learning problems in a theoretical understanding of the nature of student thinking (e.g., see Coley and Tanner, 2012 , 2015 ; Gouvea and Simon, 2018 ) and the kinds of instructional tools that would best support the learning process.

Design-Based Research Fosters Collaborations across Disciplines

Recently, there have been multiple calls across science, technology, engineering, and mathematics education fields to increase collaborations between BER and other disciplines so as to increase the robustness of science education research at the collegiate level ( Coley and Tanner, 2012 ; NRC, 2012 ; Talanquer, 2014 ; Dolan, 2015 ; Peffer and Renken, 2016 ; Mestre et al. , 2018 ; Davidesco and Milne, 2019 ). Engaging in design-based research provides both a mechanism and a motivation for fostering interdisciplinary collaborations, as it requires the design team to have theoretical knowledge of how students learn, domain knowledge of practical learning problems, and instructional knowledge for how to implement instructional tools in the classroom ( Edelson, 2002 ; Hoadley, 2004 ; Wang and Hannafin, 2005 ; Anderson and Shattuck, 2012 ). For example, in our current work, our research team consists of two discipline-based education learning scientists from an R1 institution, two physiology education researchers/instructors (one from an R1 institution the other from a community college), several physiology disciplinary experts/instructors, and a K–12 science education expert.

Design-based research collaborations have several distinct benefits for BER: first, learning or cognitive scientists could provide theoretical and methodological expertise that may be unfamiliar to biology education researchers with traditional science backgrounds ( Lo et al. , 2019 ). This would both improve the rigor of the research project and provide biology education researchers with the opportunity to explore ideas and methods from other disciplines. Second, collaborations between researchers and instructors could help increase the implementation of evidence-based teaching practices by instructors/faculty who are not education researchers and would benefit from support while shifting their instructional approaches ( Eddy et al. , 2015 ). This may be especially true for community college and primarily undergraduate institution faculty who often do not have access to the same kinds of resources that researchers and instructors at research-intensive institutions do ( Schinske et al. , 2017 ). Third, making instructors an integral part of a design-based research project ensures they are well versed in the theory and learning objectives underlying the instructional tools they are implementing in the classroom. This can improve the fidelity of implementation of the instructional tools, because the instructors understand the tools’ theoretical and practical purposes, which has been cited as one reason there have been mixed results on the impact of active learning across biology classes ( Andrews et al. , 2011 ; Borrego et al. , 2013 ; Lee et al. , 2018 ; Offerdahl et al. , 2018 ). It also gives instructors agency to make informed adjustments to the instructional tools during implementation that improve their practical applications while remaining true to the goals of the research ( Hoadley, 2004 ).

Design-Based Research Invites Using Mixed Methods to Analyze Data

The diverse nature of the data that are often collected in design-based research can require both qualitative and quantitative methodologies to produce a rich picture of how the instructional tools and their implementation influenced student learning ( Anderson and Shattuck, 2012 ). Using mixed methods may be less familiar to biology education researchers who were primarily trained in quantitative methods as biologists ( Lo et al. , 2019 ). However, according to Warfa (2016 , p. 2), “Integration of research findings from quantitative and qualitative inquiries in the same study or across studies maximizes the affordances of each approach and can provide better understanding of biology teaching and learning than either approach alone.” Although the number of BER studies using mixed methods has increased over the past decade ( Lo et al. , 2019 ), engaging in design-based research could further this trend through its collaborative nature of bringing social scientists together with biology education researchers to share research methodologies from different fields. By leveraging qualitative and quantitative methods, design-based researchers unpack “mechanism and process” by characterizing the nature of student thinking rather than “simply reporting that differences did or did not occur” ( Barab, 2014 , p. 158), which is important for continuing to advance our understanding of student learning in BER ( Dolan, 2015 ; Lo et al. , 2019 ).

CHALLENGES TO IMPLEMENTING DESIGN-BASED RESEARCH IN BER

As with any methodological approach, there can be challenges to implementing design-based research. Here, we highlight three that may be relevant to BER.

Collaborations Can Be Difficult to Maintain

While collaborations between researchers and instructors offer many affordances (as discussed earlier), the reality of connecting researchers across departments and institutions can be challenging. For example, Peffer and Renken (2016) noted that different traditions of scholarship can present barriers to collaboration where there is not mutual respect for the methods and ideas that are part and parcel to each discipline. Additionally, Schinske et al. (2017) identified several constraints that community college faculty face for engaging in BER, such as limited time or support (e.g., infrastructural, administrative, and peer support), which could also impact their ability to form the kinds of collaborations inherent in design-based research. Moreover, the iterative nature of design-based research requires these collaborations to persist for an extended period of time. Attending to these challenges is an important part of forming the design team and identifying the different roles researchers and instructors will play in the research.

Design-Based Research Experiments Are Resource Intensive

The focus of design-based research on studying learning ecologies to uncover mechanisms of learning requires that researchers collect multiple data streams through time, which often necessitates significant temporal and financial resources ( Collins et al., 2004 ; O’Donnell, 2004 ). Consequently, researchers must weigh both practical as well as methodological considerations when formulating their experimental design. For example, investigating learning mechanisms requires that researchers collect data at a frequency that will capture changes in student thinking ( Siegler, 2006 ). However, researchers may be constrained in the number of data-collection events they can anticipate depending on: the instructor’s ability to facilitate in-class collection events or solicit student participation in extracurricular activities (e.g., interviews); the cost of technological devices to record student conversations; the time and logistical considerations needed to schedule and conduct student interviews; the financial resources available to compensate student participants; the financial and temporal costs associated with analyzing large amounts of data.

Identifying learning mechanisms also requires in-depth analyses of qualitative data as students experience various instructional tools (e.g., microgenetic methods; Flynn et al. , 2006 ; Siegler, 2006 ). The high intensity of these in-depth analyses often limits the number of students who can be evaluated in this way, which must be balanced with the kinds of generalizations researchers wish to make about the effectiveness of the instructional tools ( O’Donnell, 2004 ). Because of the large variety of data streams that could be collected in a design-based research experiment—and the resources required to collect and analyze them—it is critical that the research team identify a priori how specific data streams, and the methods of their analysis, will provide the evidence necessary to address the theoretical and practical objectives of the research (see the following section on experimental rigor; Sandoval, 2014 ). These are critical management decisions because of the need for a transparent, systematic analysis of the data that others can scrutinize to evaluate the validity of the claims being made ( Cobb et al. , 2003 ).

Concerns with Experimental Rigor

The nature of design-based research, with its use of narrative to characterize versus control experimental environments, has drawn concerns about the rigor of this methodological approach. Some have challenged its ability to produce evidence-based warrants to support its claims of learning that can be replicated and critiqued by others ( Shavelson et al. , 2003 ; Hoadley, 2004 ). This is a valid concern that design-based researchers, and indeed all education researchers, must address to ensure their research meets established standards for education research ( NRC, 2002 ).

One way design-based researchers address this concern is by “specifying theoretically salient features of a learning environment design and mapping out how they are predicted to work together to produce desired outcomes” ( Sandoval, 2014 , p. 19). Through this process, researchers explicitly show before they begin the work how their theory of learning is embodied in the instructional tools to be tested, the specific data the tools will produce for analysis, and what outcomes will be taken as evidence for success. Moreover, by allowing instructional tools to be modified during the testing phase as needed, design-based researchers acknowledge that it is impossible to anticipate all aspects of the classroom environment that might impact the implementation of instructional tools, “as dozens (if not millions) of factors interact to produce the measureable outcomes related to learning” ( Hoadley, 2004 , p. 204; DBR Collective, 2003 ). Consequently, modifying instructional tools midstream to account for these unanticipated factors can ensure they retain their methodological alignment with the underlying theory and predicted learning outcomes so that inferences drawn from the design experiment accurately reflect what was being tested ( Edelson, 2002 ; Hoadley, 2004 ). Indeed, Barab (2014) states, “the messiness of real-world practice must be recognized, understood, and integrated as part of the theoretical claims if the claims are to have real-world explanatory value” (p. 153).

CONCLUSIONS

providing a methodology that integrates theories of learning with practical experiences in classrooms,

using a range of analytical approaches that allow for researchers to uncover the underlying mechanisms of student thinking and learning,

fostering interdisciplinary collaborations among researchers and instructors, and

characterizing learning ecologies that account for the complexity involved in student learning

By employing this methodology from the learning sciences, biology education researchers can enrich our current understanding of what is required to help biology students achieve their personal and professional aims during their college experience. It can also stimulate new ideas for biology education that can be discussed and debated in our research community as we continue to explore and refine how best to serve the students who pass through our classroom doors.

1 “Epistemic commitment” is defined as engaging in certain practices that generate knowledge in an agreed-upon way.

ACKNOWLEDGMENTS

We thank the UW Biology Education Research Group’s (BERG) feedback on drafts of this essay as well as Dr. L. Jescovich for last-minute analyses. This work was supported by a National Science Foundation award (NSF DUE 1661263/1660643). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF. All procedures were conducted in accordance with approval from the Institutional Review Board at the University of Washington (52146) and the New England Independent Review Board (120160152).

• Alonzo, A. C. ( 2011 ). Learning progressions that support formative assessment practices . Measurement , 9 (2/3), 124–129. Google Scholar
• Anderson, T., & Shattuck, J. ( 2012 ). Design-based research: A decade of progress in education research? Educational Researcher , 41 (1), 16–25. Google Scholar
• Andrews, T. M., Leonard, M. J., Colgrove, C. A., & Kalinowski, S. T. ( 2011 ). Active learning not associated with student learning in a random sample of college biology courses . CBE—Life Sciences Education , 10 (4), 394–405. Link ,  Google Scholar
• Badenhorst, E., Hartman, N., & Mamede, S. ( 2016 ). How biomedical misconceptions may arise and affect medical students’ learning: A review of theoretical perspectives and empirical evidence . Health Professions Education , 2 (1), 10–17. Google Scholar
• Barab, S. ( 2014 ). Design-based research: A methodological toolkit for engineering change . In The Cambridge handbook of the learning sciences (2nd ed., pp. 151–170). Cambridge University Press. https://doi.org/10.1017/CBO9781139519526.011 Google Scholar
• Barab, S., & Squire, K. ( 2004 ). Design-based research: Putting a stake in the ground . Journal of the Learning Sciences , 13 (1), 1–14. Google Scholar
• Bell, P. ( 2004 ). On the theoretical breadth of design-based research in education . Educational Psychologist , 39 (4), 243–253. Google Scholar
• Borrego, M., Cutler, S., Prince, M., Henderson, C., & Froyd, J. E. ( 2013 ). Fidelity of implementation of research-based instructional strategies (RBIS) in engineering science courses . Journal of Engineering Education , 102 (3), 394–425. Google Scholar
• Brown, A. L. ( 1992 ). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings . Journal of the Learning Sciences , 2 (2), 141–178. Google Scholar
• Chi, M. T. H., Roscoe, R. D., Slotta, J. D., Roy, M., & Chase, C. C. ( 2012 ). Misconceived causal explanations for emergent processes . Cognitive Science , 36 (1), 1–61. Medline ,  Google Scholar
• Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. ( 2003 ). Design experiments in educational research . Educational Researcher , 32 (1), 9–13. Google Scholar
• Coley, J. D., & Tanner, K. D. ( 2012 ). Common origins of diverse misconceptions: Cognitive principles and the development of biology thinking . CBE—Life Sciences Education , 11 (3), 209–215. Link ,  Google Scholar
• Coley, J. D., & Tanner, K. ( 2015 ). Relations between intuitive biological thinking and biological misconceptions in biology majors and nonmajors . CBE—Life Sciences Education , 14 (1). https://doi.org/10.1187/cbe.14-06-0094 Medline ,  Google Scholar
• Collins, A., Joseph, D., & Bielaczyc, K. ( 2004 ). Design research: Theoretical and methodological issues . Journal of the Learning Sciences , 13 (1), 15–42. Google Scholar
• Corcoran, T., Mosher, F. A., & Rogat, A. D. ( 2009 ). Learning progressions in science: An evidence-based approach to reform (CPRE Research Report No. RR-63) . Philadelphia, PA: Consortium for Policy Research in Education. Google Scholar
• Davidesco, I., & Milne, C. ( 2019 ). Implementing cognitive science and discipline-based education research in the undergraduate science classroom . CBE—Life Sciences Education , 18 (3), es4. Link ,  Google Scholar
• Design-Based Research Collective . ( 2003 ). Design-based research: An emerging paradigm for educational inquiry . Educational Researcher , 32 (1), 5–8. Google Scholar
• Dolan, E. L. ( 2015 ). Biology education research 2.0 . CBE—Life Sciences Education , 14 (4), ed1. Link ,  Google Scholar
• Duschl, R., Maeng, S., & Sezen, A. ( 2011 ). Learning progressions and teaching sequences: A review and analysis . Studies in Science Education , 47 (2), 123–182. Google Scholar
• Eddy, S. L., Converse, M., & Wenderoth, M. P. ( 2015 ). PORTAAL: A classroom observation tool assessing evidence-based teaching practices for active learning in large science, technology, engineering, and mathematics classes . CBE—Life Sciences Education , 14 (2), ar23. Link ,  Google Scholar
• Edelson, D. C. ( 2002 ). Design research: What we learn when we engage in design . Journal of the Learning Sciences , 11 (1), 105–121. Google Scholar
• Flynn, E., Pine, K., & Lewis, C. ( 2006 ). The microgenetic method—Time for change? The Psychologist , 19 (3), 152–155. Google Scholar
• Gouvea, J. S., & Simon, M. R. ( 2018 ). Challenging cognitive construals: A dynamic alternative to stable misconceptions . CBE—Life Sciences Education , 17 (2), ar34. Link ,  Google Scholar
• Gunckel, K. L., Mohan, L., Covitt, B. A., & Anderson, C. W. ( 2012 ). Addressing challenges in developing learning progressions for environmental science literacy . In Alonzo, A. C.Gotwals, A. W. (Eds.), Learning progressions in science: Current challenges and future directions (pp. 39–75). Rotterdam: SensePublishers. https://doi.org/10.1007/978-94-6091-824-7_4 Google Scholar
• Hake, R. R. ( 2007 ). Design-based research in physics education research: A review . In Kelly, A. E.Lesh, R. A.Baek, J. Y. (Eds.), Handbook of design research methods in mathematics, science, and technology education (p. 24). New York: Routledge. Google Scholar
• Hoadley, C. M. ( 2004 ). Methodological alignment in design-based research . Educational Psychologist , 39 (4), 203–212. Google Scholar
• Jackson, M., Tran, A., Wenderoth, M. P., & Doherty, J. H. ( 2018 ). Peer vs. self-grading of practice exams: Which is better? CBE—Life Sciences Education , 17 (3), es44. https://doi.org/10.1187/cbe.18-04-0052 Link ,  Google Scholar
• Jin, H., & Anderson, C. W. ( 2012 ). A learning progression for energy in socio-ecological systems . Journal of Research in Science Teaching , 49 (9), 1149–1180. Google Scholar
• Joseph, D. ( 2004 ). The practice of design-based research: Uncovering the interplay between design, research, and the real-world context . Educational Psychologist , 39 (4), 235–242. Google Scholar
• Kelly, A. E. ( 2014 ). Design-based research in engineering education . In Cambridge handbook of engineering education research (pp. 497–518). New York, NY: Cambridge University Press. https://doi.org/10.1017/CBO9781139013451.032 Google Scholar
• Lee, C. J., Toven-Lindsey, B., Shapiro, C., Soh, M., Mazrouee, S., Levis-Fitzgerald, M., & Sanders, E. R. ( 2018 ). Error-discovery learning boosts student engagement and performance, while reducing student attrition in a bioinformatics course . CBE—Life Sciences Education , 17 (3), ar40. https://doi.org/10.1187/cbe.17-04-0061 Link ,  Google Scholar
• Lo, S. M., Gardner, G. E., Reid, J., Napoleon-Fanis, V., Carroll, P., Smith, E., & Sato, B. K. ( 2019 ). Prevailing questions and methodologies in biology education research: A longitudinal analysis of research in CBE — life sciences education and at the society for the advancement of biology education research . CBE—Life Sciences Education , 18 (1), ar9. Link ,  Google Scholar
• Marbach-Ad, G., Briken, V., El-Sayed, N. M., Frauwirth, K., Fredericksen, B., Hutcheson, S., … & Smith, A. C. ( 2009 ). Assessing student understanding of host pathogen interactions using a concept inventory . Journal of Microbiology & Biology Education , 10 (1), 43–50. Medline ,  Google Scholar
• McFarland, J. L., Price, R. M., Wenderoth, M. P., Martinková, P., Cliff, W., Michael, J. , … & Wright, A. ( 2017 ). Development and validation of the homeostasis concept inventory . CBE—Life Sciences Education , 16 (2), ar35. Link ,  Google Scholar
• McKenney, S., & Reeves, T. C. ( 2013 ). Systematic review of design-based research progress: Is a little knowledge a dangerous thing? Educational Researcher , 42 (2), 97–100. Google Scholar
• Mestre, J. P., Cheville, A., & Herman, G. L. ( 2018 ). Promoting DBER-cognitive psychology collaborations in STEM education . Journal of Engineering Education , 107 (1), 5–10. Google Scholar
• Michael, J. A. ( 2007 ). What makes physiology hard for students to learn? Results of a faculty survey . AJP: Advances in Physiology Education , 31 (1), 34–40. Medline ,  Google Scholar
• Michael, J. A., Modell, H., McFarland, J., & Cliff, W. ( 2009 ). The “core principles” of physiology: What should students understand? Advances in Physiology Education , 33 (1), 10–16. Medline ,  Google Scholar
• Middleton, J., Gorard, S., Taylor, C., & Bannan-Ritland, B. ( 2008 ). The “compleat” design experiment: From soup to nuts . In Kelly, A. E.Lesh, R. A.Baek, J. Y. (Eds.), Handbook of design research methods in education: Innovations in science, technology, engineering, and mathematics learning and teaching (pp. 21–46). New York, NY: Routledge. Google Scholar
• Modell, H. I. ( 2000 ). How to help students understand physiology? Emphasize general models . Advances in Physiology Education , 23 (1), S101–S107. Medline ,  Google Scholar
• Mohan, L., Chen, J., & Anderson, C. W. ( 2009 ). Developing a multi-year learning progression for carbon cycling in socio-ecological systems . Journal of Research in Science Teaching , 46 (6), 675–698. Google Scholar
• National Academies of Sciences, Engineering, and Medicine . ( 2018 ). How People Learn II: Learners, Contexts, and Cultures . Washington, DC: National Academies Press. Retrieved June 24, 2019, from https://doi.org/10.17226/24783 Google Scholar
• National Research Council (NRC) . ( 2002 ). Scientific research in education . Washington, DC: National Academies Press. Retrieved January 31, 2019, from https://doi.org/10.17226/10236 Google Scholar
• NRC . ( 2007 ). Taking science to school: Learning and teaching science in grades K–8 . Washington, DC: National Academies Press. Retrieved March 22, 2019, from www.nap.edu/catalog/11625/taking-science-to-school-learning-and-teaching-science-in-grades . https://doi.org/10.17226/11625 Google Scholar
• NRC . ( 2012 ). Discipline-based education research: Understanding and improving learning in undergraduate science and engineering . Washington, DC: National Academies Press. Retrieved from www.nap.edu/catalog/13362/discipline-based-education-research-understanding-and-improving-learning-in-undergraduate . https://doi.org/10.17226/13362 Google Scholar
• NRC . ( 2018 ). How people learn II: Learners, contexts, and cultures . Washington, DC: National Academies Press. Retrieved from www.nap.edu/read/24783/chapter/7 . https://doi.org/10.17226/24783 Google Scholar
• O’Donnell, A. M. ( 2004 ). A commentary on design research . Educational Psychologist , 39 (4), 255–260. Google Scholar
• Offerdahl, E. G., McConnell, M., & Boyer, J. ( 2018 ). Can I have your recipe? Using a fidelity of implementation (FOI) framework to identify the key ingredients of formative assessment for learning . CBE—Life Sciences Education , 17 (4), es16. Link ,  Google Scholar
• Peffer, M., & Renken, M. ( 2016 ). Practical strategies for collaboration across discipline-based education research and the learning sciences . CBE—Life Sciences Education , 15 (4), es11. Link ,  Google Scholar
• Reiser, B. J., Smith, B. K., Tabak, I., Steinmuller, F., Sandoval, W. A., & Leone, A. J. ( 2001 ). BGuILE: Strategic and conceptual scaffolds for scientific inquiry in biology classrooms . In Carver, S. M.Klahr, D. (Eds.), Cognition and instruction: Twenty-five years of progress (pp. 263–305). Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Google Scholar
• Sandoval, W. ( 2014 ). Conjecture mapping: An approach to systematic educational design research . Journal of the Learning Sciences , 23 (1), 18–36. Google Scholar
• Sandoval, W. A., & Bell, P. ( 2004 ). Design-based research methods for studying learning in context: Introduction . Educational Psychologist , 39 (4), 199–201. Google Scholar
• Schinske, J. N., Balke, V. L., Bangera, M. G., Bonney, K. M., Brownell, S. E., Carter, R. S. , … & Corwin, L. A. ( 2017 ). Broadening participation in biology education research: Engaging community college students and faculty . CBE—Life Sciences Education , 16 (2), mr1. Link ,  Google Scholar
• Scott, E., Anderson, C. W., Mashood, K. K., Matz, R. L., Underwood, S. M., & Sawtelle, V. ( 2018 ). Developing an analytical framework to characterize student reasoning about complex processes . CBE—Life Sciences Education , 17 (3), ar49. https://doi.org/10.1187/cbe.17-10-0225 Link ,  Google Scholar
• Scott, E., Wenderoth, M. P., & Doherty, J. H. ( 2019 ). Learning progressions: An empirically grounded, learner-centered framework to guide biology instruction . CBE—Life Sciences Education , 18 (4), es5. https://doi.org/10.1187/cbe.19-03-0059 Link ,  Google Scholar
• Sharma, M. D., & McShane, K. ( 2008 ). A methodological framework for understanding and describing discipline-based scholarship of teaching in higher education through design-based research . Higher Education Research & Development , 27 (3), 257–270. Google Scholar
• Shavelson, R. J., Phillips, D. C., Towne, L., & Feuer, M. J. ( 2003 ). On the science of education design studies . Educational Researcher , 32 (1), 25–28. Google Scholar
• Siegler, R. S. ( 2006 ). Microgenetic analyses of learning . In Damon, W.Lerner, R. M. (Eds.), Handbook of child psychology (pp. 464–510). Hoboken, NJ: John Wiley & Sons, Inc. https://doi.org/10.1002/9780470147658.chpsy0211 Google Scholar
• Stringer, E. T. ( 2013 ). Action research . Thousand Oaks, CA: Sage Publications, Inc. Google Scholar
• Subramaniam, M., Jean, B. S., Taylor, N. G., Kodama, C., Follman, R., & Casciotti, D. ( 2015 ). Bit by bit: Using design-based research to improve the health literacy of adolescents . JMIR Research Protocols , 4 (2), e62. Medline ,  Google Scholar
• Summers, M. M., Couch, B. A., Knight, J. K., Brownell, S. E., Crowe, A. J., Semsar, K. , … & Batzli, J. ( 2018 ). EcoEvo-MAPS: An ecology and evolution assessment for introductory through advanced undergraduates . CBE—Life Sciences Education , 17 (2), ar18. Link ,  Google Scholar
• Szteinberg, G., Balicki, S., Banks, G., Clinchot, M., Cullipher, S., Huie, R. , … & Sevian, H. ( 2014 ). Collaborative professional development in chemistry education research: Bridging the gap between research and practice . Journal of Chemical Education , 91 (9), 1401–1408. Google Scholar
• Talanquer, V. ( 2014 ). DBER and STEM education reform: Are we up to the challenge? Journal of Research in Science Teaching , 51 (6), 809–819. Google Scholar
• Wang, F., & Hannafin, M. J. ( 2005 ). Design-based research and technology-enhanced learning environments . Educational Technology Research and Development , 53 (4), 5–23. Google Scholar
• Wang, J.-R. ( 2004 ). Development and validation of a Two-tier instrument to examine understanding of internal transport in plants and the human circulatory system . International Journal of Science and Mathematics Education , 2 (2), 131–157. Google Scholar
• Warfa, A.-R. M. ( 2016 ). Mixed-methods design in biology education research: Approach and uses . CBE—Life Sciences Education , 15 (4), rm5. Link ,  Google Scholar
• Windschitl, M., Thompson, J., Braaten, M., & Stroupe, D. ( 2012 ). Proposing a core set of instructional practices and tools for teachers of science . Science Education , 96 (5), 878–903. Google Scholar
• Zagallo, P., Meddleton, S., & Bolger, M. S. ( 2016 ). Teaching real data interpretation with models (TRIM): Analysis of student dialogue in a large-enrollment cell and developmental biology course . CBE—Life Sciences Education , 15 (2), ar17. Link ,  Google Scholar
• Codéveloppement d’un programme d’autogestion de la douleur chronique en ligne: un projet de recherche basé sur la conception et axé sur l’engagement des patients 12 March 2024 | Canadian Journal of Pain, Vol. 8, No. 1
• Enhancing undergraduates’ engagement in a learning community by including their voices in the technological and instructional design Computers & Education, Vol. 214
• Practice-Based Teacher Education Benefits Graduate Trainees and Their Students Through Inclusive and Active Teaching Methods 16 October 2023 | Journal for STEM Education Research, Vol. 7, No. 1
• Leveraging learning experience design: digital media approaches to influence motivational traits that support student learning behaviors in undergraduate online courses 11 October 2022 | Journal of Computing in Higher Education, Vol. 35, No. 3
• Investigating an Assessment Design that Prevents Students from Using ChatGPT as the Sole Basis to Pass Assessment at the Tertiary Level 30 November 2023 | E-Journal of Humanities, Arts and Social Sciences
• Spatial Variations in Aquatic Insect Community Structure in the Winam Gulf of Lake Victoria, Kenya International Journal of Ecology, Vol. 2023
• The Perceived Effectiveness of Various Forms of Feedback on the Acquisition of Technical Skills by Advanced Learners in Simulation-Based Health Professions Education Cureus, Vol. 44
• Occupational therapists' acceptance of 3D printing 22 August 2023 | South African Journal of Occupational Therapy, Vol. 53, No. 2
• An app by students for students – the DPaCK-model for a digital collaborative teamwork project to identify butterflies 4 August 2023 | Frontiers in Education, Vol. 8
• Applying DBR to design protocols for synchronous online Chinese learning: An activity theoretic perspective System, Vol. 116
• Defining the Nature of Augmented Feedback for Learning Intraosseous Access Skills in Simulation-Based Health Professions Education Cureus, Vol. 86
• Practice-based 21st-century teacher education: Design principles for adaptive expertise Teaching and Teacher Education, Vol. 128
• Undergraduate students’ neurophysiological reasoning: what we learn from the attractive distractors students select Advances in Physiology Education, Vol. 47, No. 2
• Oaks to arteries: the Physiology Core Concept of flow down gradients supports transfer of student reasoning Advances in Physiology Education, Vol. 47, No. 2
• Audrey Chen ,
• Kimberley A. Phillips ,
• Jennifer E. Schaefer , and
• Patrick M. Sonner
• Kyle Frantz,, Monitoring Editor
• Optimizing the Learner’s Role in Feedback: Development of a Feedback-Preparedness Online Application for Medical Students in the Clinical Setting Cureus, Vol. 42
• History, Status, and Development of AI-Based Learning Science 8 April 2023 | SN Computer Science, Vol. 4, No. 3
• An Analytical Dashboard of Collaborative Activities for the Knowledge Building 4 March 2023 | Technology, Knowledge and Learning, Vol. 29
• The Application of a Design-Based Research Framework for Simulation-Based Education Cureus, Vol. 22
• Erin Stanfield ,
• Corin D. Slown ,
• Quentin Sedlacek , and
• Suzanne E. Worcester
• James Hewlett, Monitoring Editor
• The effect of the e-mentoring-based education program on professional development of preschool teachers 3 July 2021 | Education and Information Technologies, Vol. 27, No. 1
• Education Sciences, Vol. 12, No. 8
• Training Digital Competences of Educators in Continuing Education: A Three-Level Approach 27 October 2022
• Design-based research as a framework for developing and deploying augmented reality applications and scenarios for intercultural exchange 13 December 2021
• Repetition Is Important to Students and Their Understanding during Laboratory Courses That Include Research Journal of Microbiology & Biology Education, Vol. 22, No. 2
• Another look at the core concepts of physiology: revisions and resources Advances in Physiology Education, Vol. 44, No. 4

Submitted: 18 November 2019 Revised: 3 March 2020 Accepted: 25 March 2020

© 2020 E. E. Scott et al. CBE—Life Sciences Education © 2020 The American Society for Cell Biology. This article is distributed by The American Society for Cell Biology under license from the author(s). It is available to the public under an Attribution–Noncommercial–Share Alike 3.0 Unported Creative Commons License (http://creativecommons.org/licenses/by-nc-sa/3.0).

This paper is in the following e-collection/theme issue:

Published on 26.3.2024 in Vol 26 (2024)

Designing and Implementation of a Digitalized Intersectoral Discharge Management System and Its Effect on Readmissions: Mixed Methods Approach

Authors of this article:

Original Paper

• Christoph Strumann 1 , PhD   ; 
• Lisa Pfau 1 , MD   ; 
• Laila Wahle 2 , MBA   ; 
• Raphael Schreiber 1 , MD   ; 
• Jost Steinhäuser 1 , Prof Dr Med, MD  

1 Institute of Family Medicine, University Medical Centre Schleswig-Holstein, Campus Lübeck, Lübeck, Germany

2 Lacanja GmbH Health Innovation Port, Hamburg, Germany

Corresponding Author:

Christoph Strumann, PhD

Institute of Family Medicine

University Medical Centre Schleswig-Holstein, Campus Lübeck

Ratzeburger Allee 160

Lübeck, 23538

Phone: 49 451 3101 8005

Email: [email protected]

Background: Digital transformation offers new opportunities to improve the exchange of information between different health care providers, including inpatient, outpatient and care facilities. As information is especially at risk of being lost when a patient is discharged from a hospital, digital transformation offers great opportunities to improve intersectoral discharge management. However, most strategies for improvement have focused on structures within the hospital.

Objective: This study aims to evaluate the implementation of a digitalized discharge management system, the project “Optimizing instersectoral discharge management” (SEKMA, derived from the German Sektorübergreifende Optimierung des Entlassmanagements), and its impact on the readmission rate.

Methods: A mixed methods design was used to evaluate the implementation of a digitalized discharge management system and its impact on the readmission rate. After the implementation, the congruence between the planned (logic model) and the actual intervention was evaluated using a fidelity analysis. Finally, bivariate and multivariate analyses were used to evaluate the effectiveness of the implementation on the readmission rate. For this purpose, a difference-in-difference approach was adopted based on routine data of hospital admissions between April 2019 and August 2019 and between April 2022 and August 2022. The department of vascular surgery served as the intervention group, in which the optimized discharge management was implemented in April 2022. The departments of internal medicine and cardiology formed the control group.

Results: Overall, 26 interviews were conducted, and we explored 21 determinants, which can be categorized into 3 groups: “optimization potential,” “barriers,” and “enablers.” On the basis of these results, 19 strategies were developed to address the determinants, including a lack of networking among health care providers, digital information transmission, and user-unfriendliness. On the basis of these strategies, which were prioritized by 11 hospital physicians, a logic model was formulated. Of the 19 strategies, 7 (37%; eg, electronic discharge letter, providing mobile devices to the hospital’s social service, and generating individual medication plans in the format of the national medication plan) have been implemented in SEKMA. A survey on the fidelity of the application of the implemented strategies showed that 3 of these strategies were not yet widely applied. No significant effect of SEKMA on readmissions was observed in the routine data of 14,854 hospital admissions ( P =.20).

Conclusions: This study demonstrates the potential of optimizing intersectoral collaboration for patient care. Although a significant effect of SEKMA on readmissions has not yet been observed, creating a digital ecosystem that connects different health care providers seems to be a promising approach to ensure secure and fast networking of the sectors. The described intersectoral optimization of discharge management provides a structured template for the implementation of a similar local digital care networking infrastructure in other care regions in Germany and other countries with a similarly fragmented health care system.

Introduction

Digital patient process systems offer several advantages over analog systems. On the one hand, this can lead to more systematic, targeted use of resources, and on the other hand, easier communication and transmission of data can enable better coordination of the various cooperating partners [ 1 ]. Patient records are becoming increasingly digitalized, with some countries being prototypes in this area, such as Latvia, Denmark, and Spain [ 2 ].

In Germany, there have been several governmental attempts to shape different elements of health care digitalization. A recent example is the Hospital Future Act (Krankenhauszukunftsgesetz) from 2020. It was designed to support digitalization in hospitals by promoting the technical equipment of hospitals through state-funded investments. The investments are expected to improve process organization, documentation, and communication (internal, sectoral, and intersectoral) [ 3 ]. The results suggest that the Hospital Future Act, together with the COVID-19 pandemic, led to an increase in the digital maturity of hospitals and, thus, reduced the digitalization backlog [ 4 ]. Another approach to promote health care digitalization is the introduction of an electronic health record (EHR) within a secure telematics infrastructure. The EHR should not only simplify rapid communication within and across different health care institutions but also enable further eHealth applications, for example, electronic prescriptions [ 5 ]. However, the introduction of EHRs as well as other reforms promoting health care digitalization have been accompanied with strong resistance underpinned by arguments of data protection and security as well as by technical problems. Especially in the outpatient sector, the latter has resulted in a perceived disproportionate administrative effort without adequate financial compensation for the care providers such as private practices [ 6 ]. As a result, Germany lags behind other industrialized countries in the digitalization of the health care system [ 7 , 8 ].

An EHR could make treatment pathways more transparent and improve communication between different health care providers, including inpatient, outpatient and care facilities [ 9 ]. The exchange of information is particularly susceptible if a patient is discharged from hospital. With regard to the strongly pronounced sectoral separation in Germany [ 10 , 11 ], information loss is particularly high between inpatient and outpatient care. Moreover, owing to the accelerated tendency toward shortening the length of stay of patients in the inpatient sector as a result of the introduction of the diagnosis-related group–based reimbursement system [ 12 ], hospitals no longer provide care and treatment until full recovery [ 13 ]. Instead, parts of the treatment and recovery process are moved to the posthospital setting [ 14 ]. Similar developments have been observed after introducing the diagnosis-related group–based reimbursement system in other countries, for example, the United States [ 15 - 17 ]. Shortened length of stay and ineffectively designed transitions are associated with adverse events, higher risks of readmission, and higher costs [ 18 - 21 ]. Up to 1 (18%) in 5 patients are readmitted to the hospital within 30 days of discharge [ 22 , 23 ]. Individualized discharge management can reduce the number of readmissions of older patients with a health problem [ 24 ], leading to potential cost savings for the health care system [ 25 ]. To date, many strategies to improve discharge management have focused on structures within the hospital. However, to ensure a holistic and continuous treatment, the cooperation between different health care providers from the inpatient and outpatient sectors as well as care facilities should also be considered.

As there is still no EHR accessible to all caregivers in Germany, experience with digitalized health information systems has been gathered only in model projects, which are intended to provide insights into possible barriers and enablers for a successful implementation [ 5 , 26 - 30 ].

This study aims to explore the determinants of a digitalized discharge management system, to implement such a system within 1 area, and to evaluate its impact on the readmission rate.

The evaluation was done within the project “Optimizing intersectoral discharge management” (SEKMA, derived from the German Sektorübergreifende Optimierung des Entlassmanagements).

Study Design

A mixed methods design was chosen to evaluate SEKMA. Owing to the complexity of the intervention, the evaluation was based on the framework of developing tailored interventions [ 31 ]. This approach allows a detailed description and analysis of the components of the intervention that contributed to its effectiveness or ineffectiveness. For this purpose, this framework distinguishes between a development and an application phase. In the first step, barriers and enabling factors for a successful implementation of a digitalized discharge management system such as SEKMA were explored using qualitative research methods, that is, interviews. Second, strategies were developed for addressing these determinants. Third, these strategies were prioritized using a (quantitative) questionnaire, and a logic model was formulated to describe the logical linkages among the resources and activities needed to achieve the results. After the implementation (application phase), the congruence between the planned intervention (logic model) and the implemented intervention was evaluated. In this step, the fidelity of the use of the different strategies in the routine was examined [ 32 ]. Finally, the effectiveness of the implementation on the readmission rate (outcome) was evaluated based on routine data of hospital admissions.

The digitalized discharge management system was implemented at a medium-sized hospital (approximately 350 beds) in the northern German federal state Schleswig-Holstein in the Metropolitan area of Hamburg, the second-largest city in Germany. Before the intervention, the internal and external exchange of information was typically performed by phone, fax, and email. As the network between the various caregivers was rather weak, communication occurred only on request, tying up resources and causing delays in the transfer of information.

SEKMA aimed to develop and implement a digitalized, intersectoral discharge management system that considers the patient’s entire treatment pathway, from hospital admission to possible admission to a care facility, and the follow-up treatment by general practitioners (GPs). All information relevant to ongoing (postinpatient) treatment and care should be available quickly and easily to all care providers involved. This includes providers from the inpatient and outpatient sectors as well as care facilities. For this purpose, an ecosystem of hospital and postinpatient care facilities has been implemented within a digital infrastructure based on a standardized and harmonized IT system for data exchange [ 33 ]. The workflow of the digitalized, intersectoral discharge management can be described as follows:

• The hospital coordinates and organizes follow-up care in a timely manner based on the patient’s agreement with the hospital’s discharge management.
• A discharge plan for medication, follow-up care, and rehabilitation is created and all professionals in the hospital are involved. This includes admission staff, medical service, nursing service, social service, and the patient information system.
• In cooperation with the nursing staff and social service, the patient is informed and advised about care options and structures that correspond to their illness. The contents are prepared digitally.
• The patient is discharged from the hospital and transitions to outpatient, rehabilitative, or nursing care. All documents necessary for discharge and further treatment are available digitally and can be transmitted directly to the relevant sectors.
• If a patient contacts a primary care physician for outpatient follow-up treatment, the patient’s digital discharge documentation is already in the system of the private practice.
• In case of a query or deterioration of health status, the primary care physician can contact the hospital and previously treating physicians directly.
• If there is a readmission, the hospital can digitally access documentation on posttreatment care and procedures, as well as the medical history, at any time and continue treatment directly. The same applies to nursing and rehabilitation facilities.

The information transfer across the distinct health care provider is organized via KIM (Kommunikation im Medizinwesen) embedded in the telematics infrastructure. All organizations involved in the project have a KIM connection. Using KIM, participants can transmit documents in a secure and encrypted manner [ 34 ]. Overall, all communication processes have been digitalized compared with before the intervention. Since April 2022, optimized discharge management has been implemented in the department of vascular surgery.

Individual Interviews

Enabling factors and barriers leading toward successful digital discharge management were identified through individual interviews with physicians, medical assistants, social workers and nurses at the hospital, GPs, and staff from nursing homes and care services. This was performed using the COREQ (Consolidated Criteria for Reporting Qualitative Research) guidelines for qualitative studies ( Multimedia Appendix 1 provides details of the COREQ guidelines [ 35 ]). Originally, a combination of interviews and focus groups was planned. Owing to the COVID-19 pandemic, focus groups had to be abandoned.

The hospital, along with collaborating partners such as physician networks and nursing homes, conducted participant recruitment for interviews through face-to-face interactions, telephone calls, and emails. Previously developed partially standardized interview guidelines were used and pilot tested ( Multimedia Appendix 2 ). The interviews were conducted by telephone by a medical student (LP) between April 30, 2020, and October 9, 2020, at the workplace of the interviewees. A theoretical saturation effect in the statements made during the interviews resulted in the final number of interviewees.

The individual interviews were conducted in a protected setting and subsequently pseudonymized, thus providing the opportunity to explore the personal opinions of the interviewees beyond any possible social group pressures. The interviews were recorded using a digital dictaphone and were transcribed orthographically. The material was subsequently analyzed using structured content analysis according to Mayring [ 36 ]. The development of the categories was initially based on the questions (deductive) listed in the partially standardized interview guideline ( Multimedia Appendix 2 ). In addition, categories were extracted from the text (inductive). Five persons were involved in the development of the category scheme (LP: medical student [female researcher], JS: GP and experienced health service researcher including qualitative research [male researcher], CS: health economist with some experience in qualitative research [male researcher], a legal project advisor [female researcher], and a physiotherapist [female researcher]; all of them except LP were employed at the Institute of Family Medicine at the University of Lübeck at the time of the analysis). After individual coding, a coding scheme was discussed in a consensus meeting. The final coding scheme was applied to the interview material.

Development and Evaluation of Strategies

On the basis of the described processes for treating the patients, the optimization potential, and the determinants from the evaluated individual interviews as well as the workshop with clinicians and physicians in private practice, strategies for the implementation of optimized discharge management were developed. These strategies were developed in such a way that they addressed the determinants identified and were, thus, conducive to a successful implementation.

During a project meeting on February 3, 2022, employees of the Institute of Family Medicine at the University of Lübeck and the chief and senior physicians of the involved hospital discussed these results. Subsequently, the hospital’s chief or senior physicians were invited to evaluate each identified strategy according to its relevance and feasibility using a 6-point Likert scale (very high, rather high, high, rather low, low, and very low) to avoid the central tendency bias.

The resulting list of the ranked strategies formed the logic model. This model was finally compared with the list of strategies implemented in the project.

Routine Data Analysis

The focus of the evaluation of the optimized discharge management was the reduction of (unnecessary) readmissions. With the help of the evaluation of the routine admission data of the involved hospital, the effect of optimized discharge management on rehospitalization was analyzed.

Routine Data and Study Design

The hospital extracted routine data from its internal patient information system. The extracted data were provided by the hospital in an anonymized form. For each inpatient case, the data consisted of information on the date of admission and discharge, the reason for admission and discharge, diagnoses and conducted medical procedures, demographic information of the patients, and the department or departments where the patients had been treated.

Within the framework of a longitudinal study design, a pre- and postcomparison was performed. The intervention group was the department of vascular surgery, in which the optimized discharge management was implemented since April 2022. A case was assigned to the intervention group if the patient was admitted to or discharged from the department of vascular surgery. The outbreak of COVID-19 during the sample period might have affected the readmissions of the entire hospital. To minimize the risk of bias owing to the pandemic on the intervention effect, in addition to the pre-post comparison of the department of vascular surgery, a control group comparison was applied to enrich the empirical strategy. To ensure that the patients in the intervention group were as similar as possible to those in the control group, the departments of internal medicine (medical clinic) and cardiology formed the control group.

Statistical Analysis

The effect of the implementation was estimated using the difference-in-difference (DiD) approach. The sample covers the period from 2019 to August 2022. To counteract the possible COVID-19 pandemic bias, patients admitted between January 2020 and March 2022 were not considered in the analysis. To avoid any seasonal influences on the results, we restricted the preintervention period such that it covered exactly the period after the implementation, that is, from April to August. Therefore, the baseline period (T 0 ) consisted of April 1, 2019, to August 31, 2019, whereas the intervention period (T 1 ) started from April 1, 2022.

In addition to the bivariate analysis, a multivariate logistic regression model was applied. By including control variables, differences between patients from the intervention and control group were minimized. In the first step, risk factors for rehospitalization were determined by estimating separate bivariate logistic regression models. The identified risk factors served as control variables in the multivariate DiD regression analysis. A P value <.05 was considered statistically significant. Statistical analyses were performed with Stata (version 15; StataCorp LLC).

Ethical Considerations

The study was approved by the ethics committee of the University of Lübeck before recruitment commenced on December 11, 2019 (approval number 19-387). This study was conducted in accordance with the Declaration of Helsinki.

All participants provided verbal and written informed consent for their participation in the interviews and surveys. The participants were informed that they could withdraw their consent at any time. No identifiable information was recorded to ensure the confidentiality of the participants. No compensation was paid for participation.

For the analysis of routine hospital data, only anonymized data were transferred to the evaluating institution. Owing to the anonymization of the data, no additional informed consent was required to perform the routine data analysis in accordance with German law, ethical standards, and the Declaration of Helsinki. No data requiring informed consent will be presented in the routine data analysis. The ethics committee of the University of Lübeck waived the requirement for informed consent owing to the retrospective nature of this study.

A total of 26 interviews were conducted. These consisted of 14 employees of the hospital (3 doctors, 4 nurses, 4 social workers, and 3 administrative staff), 9 employees from nursing homes or mobile nursing services, and 3 GPs. The average age of the participants was 42.4 (SD 8.9; range 25-65) years, and the proportion of female participants was 54% (14/26). The average interview duration was 33 minutes and 11 seconds. An overview of the characteristics of the interview participants is provided in Table S1 in Multimedia Appendix 3 .

A total of 21 determinants were explored with various subcategories for the introduction of successful digitalized discharge management. These could be divided into 3 categories: “optimization potential,” “barriers,” and “enablers.” The aspects mentioned for optimizing the discharge process covered all areas from admission to follow-up and included inter- and intrasectoral transmission of information ( Textbox 1 ).

Category and subcategories

• Preliminary discharge letter before discharge
• Final discharge letter at the time of discharge
• Digital transmission (mail, chat, and video call)
• Platform for information exchange
• Standardized information
• Increased readiness to communicate
• Information exchange at admission
• Consent to discharge management
• Awareness of the existence of discharge management in the hospital
• Timely completion of the discharge process
• Continuous preparation for (unplanned) discharge
• Improvement of patient communication
• Faster approvals by health insurances
• Discharge in the morning of the working day
• Material transfer, issuing of prescriptions and incapacity certificate
• Nursing services accompany discharge from hospital
• Increase in the availability of patient transport
• Visits to general practitioner after discharge
• More aftercare places
• Training on discharge
• Digital checklist
• Standardized processes
• Clarified responsibilities
• Knowledge of the performance and processes at other facilities
• Evaluation of criticism or review
• Supervision
• Ethics committee

In the German health care system, the discharge letter is at the center of information transmission between the inpatient and outpatient sectors. Participants saw a need for improvement in the early, or at least timely, delivery of this letter. In the best case, information would already be transmitted during the hospital stay to the follow-up service providers such as private practices or care facilities:

To have all the information and data, everything before the patient arrives here. That would be the absolute dream. [...] You can just admit the person better[...] if you just have preliminary information. [P03]

Digital transmission of data was also perceived as beneficial; the participants could imagine using conventional media such as email or video calls as well as via a platform provided specifically for this purpose:

If you could even find some other common platform where information can be exchanged. [P01]

Furthermore, the potential for optimization was seen in the standardization of the information. The information to be communicated should be transmitted through a central entity, and at the same time, selected contacts who can be reached on demand and who can provide information about the patients would be beneficial:

Yes, standard, standard, standard. So, that you try to agree on what information I need and then it has to appear—in a structured form, so in principle already like my patient information. [P10]

Some participants also noted that, in principle, a greater willingness to communicate between the individual players would improve the transmission of information.

Participants noted that for a seamless discharge, information about the patient should already be available at the time of admission to the hospital:

Discharge or discharge planning and a good discharge process starts at admission. [...] The important thing is not to think about discharge on the day of discharge, but already on admission. [P25]

Improved patient communication was also considered important by interviewees:

And that is certainly a wish that I would have that the patients in the hospital are also informed about what they actually have, what has happened and what the next steps are. [P01]

An optimal discharge should ideally take place in the morning on a working day, and the handing over of medication and required materials should be regulated. This is considered to be the case by nursing homes and outpatient care services as well as by hospital staff:

From 9 or 10 a.m onwards, the number of patients in the emergency room increases and drops again from 8 p.m onwards. And during this peak time, there are few beds available in the hospital. Afterwards, however, when we are closed, the hospital finally loses cases and at night we have more free capacity again. And that is a mismatch between demand and capacity which can be improved. [P20]

Textbox 2 shows the barriers and enabling factors for intersectoral collaboration in the context of optimizing discharge management. In addition to the technical aspects and subjective reasons, there were concerns about data protection and fear that a change in the discharge process would require more time:

Time pressure is always an issue, both in the hospital and in outpatient care. We just often don’t have the time for some processes that we would all consider useful. [P01]

• Data transmission security
• Legal uncertainties
• Leaving known structures and processes
• Lack of electronic data processing experience
• Higher time consumption
• Lack of personnel
• Limitation of one’s own competence
• Unclear communication processes
• No perceived benefit
• Low appreciation for discharge management
• No priority of discharge management
• No consequences for noncompliance
• User-unfriendly system
• Electronic data processing errors
• Interface problems
• Outdated technical equipment
• Lack of education or communication
• Lack of networking among health care providers
• Clear responsibilities, instructions, contact persons, or responsibilities
• Surveillance
• Introduction or training of new processes
• No overload and enough time
• Regular exchange for networking
• Time saving
• Workload reduction
• Improved exchange of information
• Feedback loops
• Priority in the management
• Communicating the advantages
• Involvement of employees

In contrast, a possible reduction in workload owing to digitalized processes was seen as conducive:

Digitalization must not be an end in itself, in my opinion, but it must really mean an advantage for the processes, increase safety, increase communication, but it must not be a question of just because it is digital, that it is better in every case and is then associated with the fact that medical or nursing working time is lost or additionally created. [P23]

For the changeover to be successful, the communication of the advantages associated with optimized discharge management was emphasized above all as part of change management.

On the basis of the surveyed processes, the optimization potential, and the determinants from the evaluated individual interviews as well as the workshop with clinicians and physicians in private practice, 19 strategies for the implementation of optimized discharge management were developed. To rank these strategies, chief physicians of the hospital were invited to rate their relevance and feasibility.

A total of 11 physicians participated in the survey to evaluate the strategies ( Table 1 ). The strategies of always sending the discharge letter to the GP, equipping the hospital’s social service with mobile devices (eg, laptops and tablets), generating individual medication plans in the format of the national medication plan, and exclusively using the federal medication plan received the highest ratings. In contrast, the introduction of a chat function used exclusively by physicians for direct exchange between hospital and office-based physicians received the lowest rating.

a Mean over participants (6=very high, 5=rather high, 4=high, 3=rather low, 2=low, and 1=very low).

b GP: general practitioner.

c Not part of the intervention but planned for the future by the hospital.

On the basis of these ratings, the hospital staff discussed which of these strategies were already being implemented or planned for implementation in the near future. Of the 19 strategies, 6 (31%) were assessed as already implemented, 7 (37%) were assessed as planned, and 6 (31%) were assessed as not feasible to implement in the project. The 7 strategies rated highest in the development of the logic model (planned implementation) have been implemented or will be implemented in the near future as part of SEKMA.

To summarize, by April 2022, at the department of vascular surgery, (1) discharge letters were continuously updated digitally, (2) they were always sent, (3) they were sent electronically to the GP (via the infrastructure of KIM), (4) the hospital social service was equipped with mobile devices, (5) individual medication plans were in the format of the national medication plan, (6) the discharge management consent process at admission was standardized, and (7) a hotline for direct communication between hospital physicians and primary care physicians was implemented. The information transfer via the discharge letter was oriented by the standard of medical information objects (MIOs) eArztbrief. The development of this standard was initiated in 2022 by the National Association of Statutory Health Insurance Physicians and the German Hospital Association. It defines a standard for the electronic hospital discharge letter within the EHR ensuring the transition of relevant information from inpatient to subsequent care in a structured and secure manner [ 37 ]. The MIO eArztbrief was not yet ready during the project; however, the current status of the MIO was incorporated into the letter as much as possible.

Fidelity Analysis

After the implementation of the optimized discharge management into the routine in the department of vascular surgery as well as at the external partners in April 2022, the stakeholders participating in the project were asked in a fidelity analysis in September 2022 to what extent the identified strategies were implemented in practice. The survey showed that many of these strategies were not yet widely applied.

A total of 14 individuals responded to the survey (Table S2 in Multimedia Appendix 2 ). Of the 14 individuals, 11 (79%) were employed at the hospital and 1 (7%) each at an outpatient nursing service, nursing home, and private practice. Employees from social services and medical assistants did not participate in this survey. Of those surveyed, >30% (4/13) stated that they were satisfied with the implementation of the change in discharge management.

There are differences in the fidelity of use among the strategies implemented (Table S3 in Multimedia Appendix 3 ). Although sending or receiving an electronic discharge letter was always or sometimes used in their routine by only a quarter of respondents, approximately 85% (11/13) of the respondents indicated that medication plans from the hospital were always in the format of the federal medication plan at discharge.

Readmissions

In total, 12,407 patients were admitted to the hospital as inpatients during the study period (from April 2019 to August 2019 and from April 2022 to August 2022), corresponding to 14,854 cases treated. The internal medicine department (medical clinic) treated most of the cases (4175/14,854, 28.11%). Cases treated in the interventional group (vascular surgery) accounted for 5.11% (759/14,854) of all inpatient cases. Overall, 8.73% (994/11,386) of the patients were readmitted after 30 days. In terms of treated cases, the readmission rate was 9.07% (1222/13,477). The rates increased to 17.1% (1542/9016) for patients and 18.85% (1975/10,478) for cases when considering a longer time horizon for the readmission (90 days). Readmission rates were generally higher in the intervention group (80/705, 11.3%) at 30 days and 28.8% (161/560 at 90 days) than in the hospital as a whole and the control group. Table S4 in Multimedia Appendix 3 provides the number of admitted patients and cases treated as well as the readmissions after 30, 60, and 90 days for the total hospital cases and the departments involved.

Risk Factors

Risk factors for readmission were identified to take the differences between patients from different departments into account for the evaluation of the project’s implementation effect.

Older patients, as well as cases with a length of stay of >6 days, had a significantly higher risk of readmission. Similarly, discharge time influenced the readmission risk: patients discharged during the night (9 PM to 5 AM) had a higher risk of readmission. Similarly, there were significant differences in readmissions between cases with different ICD-10 ( International Statistical Classification of Diseases , Tenth Revision ) chapters of principal and secondary diagnoses (Table S5 in Multimedia Appendix 3 ).

Intervention Effect

Table 2 shows the implementation effects on the readmission rate after 30, 60, and 90 days (DiD) of the bivariate analysis. In the intervention group, the 30-day readmission rate increased by 2.33 percentage points from 10.4% (45/431) to 12.8% (35/274) after SEKMA was implemented. For the 60- and 90-day readmission rate, the increase was even higher (60 days: 2.25 and 90 days: 3.94). These increases have been smaller in the control group. Therefore, a reduction effect of the intervention on the readmission rate (ie, a negative DiD estimate) cannot be observed. Concentrating the analysis on patients aged ≥65 years revealed similar results (Table S6 in Multimedia Appendix 3 ). As a robustness check, the preintervention period was extended to include admissions between 2011 and 2019. These results confirm the previous findings.

a Only admissions between April 2019 and August 2019 and between April 2022 and August 2022 were considered.

b DiD: difference-in-difference, Δ: Difference between the readmission rates of the intervention and the control group at T 0 and T 1 , respectively.

c Intervention period (T 1 ): from April 1, 2022.

d N/A: not applicable.

e Baseline period (T 0 ): April 1, 2019, to August 31, 2019.

The results of the multivariate logistic regression model ( Table 3 ) confirm the results of the bivariate analysis that there were higher readmission rates in the intervention group and that there was no significant effect of the optimized discharge management on readmissions in the available data. This result was also confirmed for patients aged >65 years (Table S7 in Multimedia Appendix 3 ). Furthermore, the insignificance of the effect of the implementation of SEKMA on readmission rates was also confirmed in a pre-post comparison estimated by a multivariate logistic regression based on vascular surgery cases only (Table S8 in Multimedia Appendix 3 ). Finally, the estimated effects remained very similar if the preintervention period began in 2011 and ended at the end of 2019.

b In addition to the variables listed here, the International Statistical Classification of Diseases, Tenth Revision chapters of the principal and secondary diagnoses were also included as control variables.

d DiD: difference-in-difference.

This study aims to explore the barriers and enablers of a digitalized discharge management system, to implement such a system using a logic model developed from these determinants, and to evaluate its impact on the readmission rate.

Determinants and Implementation Strategies

The importance of the transmission of information for improved discharge management is also highlighted in the high rating of the strategies regarding the discharge letter, that is, developing an electronic discharge letter, continuously entering information into the letter, and always sending it to the GP. The discharge letter is the standard communication tool between inpatient and ambulatory care and found to be a source for deficits in information transfer [ 38 ]. In particular, delay and incompleteness of medication-related information endanger patients’ safety [ 39 , 40 ], leading to an increased risk of hospital readmission [ 41 ]. As shown for a sample of 20 Dutch hospitals, discharge letters vary in quality depending on patient and admission characteristics [ 42 ]. A standardized discharge letter can reduce transcription time and improve medical communication between physicians [ 43 ]. In addition, GPs prefer that discharge letters be written in a clear, concise, and understandable manner [ 44 ]. An electronic discharge letter generated from a computer-based document not only avoids transcription errors and lacks standardization but also ensures timely delivery [ 45 ]. In Germany, the discharge letter played a central role in approaches to creating a standard for intersectoral information exchange. For example, the VHitG (derived from the German “Verband der Hersteller von IT-Lösungen im Gesundheitswesen”) initiative “Intersectoral Communication” developed an implementation to facilitate the exchange of discharge letters between sectors, which is integrated into the existing IT system [ 46 ]. Another example is the recent approach by the National Association of Statutory Health Insurance Physicians and the German Hospital Association to create a standard for the electronic hospital discharge letter within the EHR [ 37 ].

To improve the standardization of the transmitted medication information, the use of the format of the nationwide medication plan was considered an important strategy in this study. In Germany, several projects have shown that physicians, pharmacists, and patients realize the benefits and accept the nationwide medication plan [ 47 - 49 ]. It can serve for the health care providers as a promising tool to improve the interdisciplinary and multiprofessional collaboration, especially as a digital solution that can realize its full potential [ 50 ]. Similar results have been reported in other countries [ 51 , 52 ]. In this study, participants suggested transferring medication-related information electronically and always in the format of the national medication plan. In the participating hospital, this strategy has been implemented during the project. For older patients in particular, shared medication records have the potential to reduce hospital readmissions [ 51 ].

Concerns about technical and temporal integrability were identified as an important barrier to the implementation of optimized discharge management. This includes an expected higher time consumption for the introduction of digitalized processes, a general fear of contact (owing to leaving known structures and a lack of electronic data processing experience), and further technical aspects (as a user-unfriendly system, electronic data processing errors, and interface problems). Similar barriers were identified in related eHealth projects [ 53 - 57 ]. Although the digitalization of processes was expected, in general, to be associated with time advantages, many of those involved associate the introduction with additional work effort. To overcome these concerns, successful implementation requires streamlining, simplifying, and redesigning the existing health care practices as a first step [ 58 ]. The strategy of introducing a physician-only hotline and a chat function for direct communication between the hospital and GPs could be seen as a simplification of communication instead of relying solely on the legally required discharge letter.

Effect on Readmissions

A possible explanation for the low level of fidelity as well as the insignificant effect of SEKMA on readmissions could be the relatively short application period of half a year (from April 2022 to September 2022). Complex implementations such as those elaborated in SEKMA may require a longer time before they are applied in daily routines. Another reason for the insignificant effect on readmissions could be the rather good baseline level of the outcome in national comparison. Although other studies in Germany showed readmission rates, for example, of 18.1% (30 days) to 35.4% (90 days) for older patients (aged >65 years) [ 22 ], these rates were substantially lower for the patients in this study, that is, 11.8% (30 days) to 23.6% (90 days).

Limitations

Our study had several limitations. First, the restrictions that existed owing to the COVID-19 pandemic might have affected the effectiveness of the implementation. All stakeholders involved in SEKMA faced a high workload owing to the pandemic as well as the requirements and measures resulting from the pandemic. However, the study results show that even under the special circumstances of the pandemic, it was possible to develop and implement an intersectoral optimization of discharge management. The infrastructure for the intersectoral care of patients created by the project has great potential to increase the quality of care, even if this could not yet be demonstrated with regard to readmissions. Future research should analyze the routine hospital data over the next 5 years.

Although the study included all relevant health care providers and considered the entire patient care pathway, the number of respondents from some professions may be rather small. For example, only 3 GPs were interviewed. However, the theoretical saturation effect in the statements made during the interviews suggests that this number is sufficient to identify the optimization potential as well as determinants.

Conclusions

Creating a digital ecosystem that connects different health care providers seems to be a promising approach to ensure secure and fast networking of the sectors and to promote rapid information exchange between the sectors. The described intersectoral optimization of discharge management provides a structured template for the implementation of a similar local digital care networking infrastructure in other care regions in Germany and other countries with a similarly fragmented health care system.

Acknowledgments

This study was financially supported by the Ministry of Justice and Health (Ministerium für Justiz und Gesundheit), Schleswig-Holstein. This study was conducted independently.

Data Availability

The data sets generated during and analyzed during this study are not publicly available due to the votum of the Ethics Committee of the University of Lübeck.

Authors' Contributions

CS contributed to conceptualization, formal analysis, investigation, methodology, and validation; prepared the original draft; and reviewed and edited the manuscript. LP participated in methodology, conducted and analyzed the interviews, and reviewed and edited the draft. LW participated in conceptualizing the digital discharge system and reviewed and edited the draft. RS was involved in conceptualization and reviewing and editing the draft. JS contributed to conceptualization, investigation, methodology, and validation and reviewed and edited the draft. All authors have read and approved the final manuscript.

Conflicts of Interest

LW was a hospital manager with a focus on digitalization at the hospital under study during the time of the project, Sektorübergreifende Optimierung des Entlassmanagements (SEKMA). LW is the founder of the company Lacanja GmbH Health Innovation Port, Hamburg, Germany, and is a member of several committees, including the expert group of the Gematik IOP (Interop) Council. All other authors declare no other conflicts of interest.

COREQ (Consolidated Criteria for Reporting Qualitative Research) checklist.

Interview guide.

Supplemental tables with results of supplemental analyses.

• Imison C, Castle-Clarke S, Watson R, Edwards N. Delivering the benefits of digital health care. Nuffield Trust. 2016. URL: https:/​/www.​nuffieldtrust.org.uk/​sites/​default/​files/​2017-01/​delivering-the-benefits-of-digital-technology-web-final.​pdf [accessed 2024-03-11]
• Bonomi S. The electronic health record: a comparison of some European countries. In: Ricciardi F, Harfouche A, editors. Information and Communication Technologies in Organizations and Society. Cham, Switzerland. Springer; 2016.
• Jorzig A. Digitalisierung von krankenhäusern und krankenhauszukunftsgesetz. Gynäkologie. Oct 11, 2022;55:880-884. [ CrossRef ]
• Burmann A, Fischer B, Brinkkötter N, Meister S. Managing directors' perspectives on digital maturity in German hospitals-a multi-point online-based survey study. Int J Environ Res Public Health. Aug 06, 2022;19(15):9709. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Graf A, Fehring L, Henningsen M, Zinner M. Going digital in Germany: an exploration of physicians' attitudes towards the introduction of electronic prescriptions - a mixed methods approach. Int J Med Inform. Jun 2023;174:105063. [ CrossRef ] [ Medline ]
• Nohl-Deryk P, Brinkmann JK, Gerlach FM, Schreyögg J, Achelrod D. [barriers to digitalisation of healthcare in Germany: a survey of experts]. Gesundheitswesen. Nov 2018;80(11):939-945. [ CrossRef ] [ Medline ]
• Caumanns J. [For discussion: the state of digitization of the German healthcare system]. Z Evid Fortbild Qual Gesundhwes. Jun 2019;143:22-29. [ CrossRef ] [ Medline ]
• Lupiáñez-Villanueva F, Devaux A, Valverde-Albacete J, Folkvord F, Fauli C, Altenhofer M, et al. Benchmarking deployment of eHealth among general practitioners. Publications Office of the European Union. 2018. URL: https://www.rand.org/pubs/external_publications/EP67863.html [accessed 2024-03-11]
• Melby L, Brattheim BJ, Hellesø R. Patients in transition--improving hospital-home care collaboration through electronic messaging: providers' perspectives. J Clin Nurs. Dec 2015;24(23-24):3389-3399. [ CrossRef ] [ Medline ]
• Blümel M, Spranger A, Achstetter K, Maresso A, Busse R. Germany: health system review. Health Syst Transit. Dec 2020;22(6):1-272. [ FREE Full text ] [ Medline ]
• Büyükdurmus T, Kopetsch T, Schmitz H, Tauchmann H. On the interdependence of ambulatory and hospital care in the German health system. Health Econ Rev. Dec 2017;7(1):2. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Koné I, Maria Zimmermann B, Nordström K, Simone Elger B, Wangmo T. A scoping review of empirical evidence on the impacts of the DRG introduction in Germany and Switzerland. Int J Health Plann Manage. Jan 2019;34(1):56-70. [ CrossRef ] [ Medline ]
• von Eiff W, Schüring S, Greitemann B, Karoff M. [REDIA--impacts of DRG introduction in the acute sector on medical rehabilitation]. Rehabilitation (Stuttg). Aug 2011;50(4):214-221. [ CrossRef ] [ Medline ]
• Braun B. Auswirkungen der DRGs auf Versorgungsqualität und Arbeitsbedingungen im Krankenhaus. In: Manzei A, Schmiede R, editors. 20 Jahre Wettbewerb im Gesundheitswesen. Wiesbaden, Germany. Springer VS; 2014.
• Sager MA, Easterling DV, Kindig DA, Anderson OW. Changes in the location of death after passage of medicare's prospective payment system. A national study. N Engl J Med. Feb 16, 1989;320(7):433-439. [ CrossRef ] [ Medline ]
• Rogers WH, Draper D, Kahn KL. Quality of care before and after implementation of the DRG-based prospective payment system: a summary of effects. JAMA. Oct 17, 1990;264(15):1989-1994. [ CrossRef ]
• Kosecoff J, Kahn KL, Rogers WH. Prospective payment system and impairment at discharge: the `quicker-and-sicker' story revisited. JAMA. Oct 17, 1990;264(15):1980-1983. [ CrossRef ]
• Freund T, Mahler C, Erler A, Gensichen J, Ose D, Szecsenyi J, et al. Identification of patients likely to benefit from care management programs. Am J Manag Care. May 2011;17(5):345-352. [ FREE Full text ] [ Medline ]
• Shulan M, Gao K, Moore CD. Predicting 30-day all-cause hospital readmissions. Health Care Manag Sci. Jun 2013;16(2):167-175. [ CrossRef ] [ Medline ]
• Forster AJ, Murff HJ, Peterson JF, Gandhi TK, Bates DW. Adverse drug events occurring following hospital discharge. J Gen Intern Med. Apr 2005;20(4):317-323. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Solet DJ, Norvell JM, Rutan GH, Frankel RM. Lost in translation: challenges and opportunities in physician-to-physician communication during patient handoffs. Acad Med. Dec 2005;80(12):1094-1099. [ CrossRef ] [ Medline ]
• Ruff C, Gerharz A, Groll A, Stoll F, Wirbka L, Haefeli WE, et al. Disease-dependent variations in the timing and causes of readmissions in Germany: a claims data analysis for six different conditions. PLoS One. Apr 26, 2021;16(4):e0250298. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zabawa C, Cottenet J, Zeller M, Mercier G, Rodwin VG, Cottin Y, et al. Thirty-day rehospitalizations among elderly patients with acute myocardial infarction: impact of postdischarge ambulatory care. Medicine (Baltimore). Jun 2018;97(24):e11085. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Gonçalves-Bradley DC, Lannin NA, Clemson L, Cameron ID, Shepperd S. Discharge planning from hospital. Cochrane Database Syst Rev. Feb 24, 2022;2(2):CD000313. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kripalani S, Chen G, Ciampa P, Theobald C, Cao A, McBride M, et al. A transition care coordinator model reduces hospital readmissions and costs. Contemp Clin Trials. Jun 2019;81:55-61. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Schönemann-Gieck P, Evers A. Klinikentlassungen älterer Patienten mit sozialem Interventionsbedarf: Möglichkeiten und Grenzen Kommunalen Handelns am Beispiel des Wiesbadener Gesundheitsnetzes "GeReNet.Wi". Pflege & Gesellschaft. 2018. URL: https://tinyurl.com/yc56yfr2 [accessed 2024-03-11]
• Held LA, Wewetzer L, Steinhäuser J. Determinants of the implementation of an artificial intelligence-supported device for the screening of diabetic retinopathy in primary care - a qualitative study. Health Informatics J. 2022;28(3):14604582221112816. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Dünnebeil S, Sunyaev A, Blohm I, Leimeister JM, Krcmar H. Determinants of physicians' technology acceptance for e-health in ambulatory care. Int J Med Inform. Nov 2012;81(11):746-760. [ CrossRef ] [ Medline ]
• Hennemann S, Beutel ME, Zwerenz R. Drivers and barriers to acceptance of web-based aftercare of patients in inpatient routine care: a cross-sectional survey. J Med Internet Res. Dec 23, 2016;18(12):e337. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Poss-Doering R, Kunz A, Pohlmann S, Hofmann H, Kiel M, Winkler EC, et al. Utilizing a prototype patient-controlled electronic health record in Germany: qualitative analysis of user-reported perceptions and perspectives. JMIR Form Res. Aug 03, 2018;2(2):e10411. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Jäger C, Szecsenyi J, Freund T, Reichel JK, Kuhlmey C, Wensing M, et al. [Developing a tailored intervention: implementing recommendations for polypharmacy in multimorbid patients (PomP)]. Z Evid Fortbild Qual Gesundhwes. 2014;108(5-6):270-277. [ CrossRef ] [ Medline ]
• Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S. A conceptual framework for implementation fidelity. Implement Sci. Nov 30, 2007;2:40. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sektor­übergreifende optimierung des entlass­managements. SEKMA. URL: https://www.sekma.de/ [accessed 2023-02-07]
• KIM - Der standard für sicheren E-Mail und datenaustausch im gesundheitswesen. Gematik. URL: https://fachportal.gematik.de/fileadmin/Fachportal/Veranstaltungen/gematik_KIM_DMEA_2021.pdf [accessed 2023-11-01]
• Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. Dec 2007;19(6):349-357. [ CrossRef ] [ Medline ]
• Mayring P. Qualitative inhaltsanalyse. In: Mey G, Mruck K, editors. Handbuch Qualitative Forschung in der Psychologie. Wiesbaden, Germany. VS Verlag für Sozialwissenschaften; 2010.
• Informationsobjekt "Krankenhaus-Entlassbrief": letter of intent bekräftigt zusammenarbeit zwischen KBV und DKG. Deutsche Krankenhausgesellschaft. Apr 15, 2021. URL: https://www.dkgev.de/dkg/presse/details/gemeinsame-pressemitteilung/ [accessed 2023-10-30]
• Kripalani S, LeFevre F, Phillips CO, Williams MV, Basaviah P, Baker DW. Deficits in communication and information transfer between hospital-based and primary care physicians: implications for patient safety and continuity of care. JAMA. Feb 28, 2007;297(8):831-841. [ CrossRef ] [ Medline ]
• Freyer J, Kasprick L, Sultzer R, Schiek S, Bertsche T. A dual intervention in geriatric patients to prevent drug-related problems and improve discharge management. Int J Clin Pharm. Oct 2018;40(5):1189-1198. [ CrossRef ] [ Medline ]
• Schwarz CM, Hoffmann M, Schwarz P, Kamolz LP, Brunner G, Sendlhofer G. A systematic literature review and narrative synthesis on the risks of medical discharge letters for patients' safety. BMC Health Serv Res. Mar 12, 2019;19(1):158. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Pesko MF, Gerber LM, Peng TR, Press MJ. Home health care: nurse-physician communication, patient severity, and hospital readmission. Health Serv Res. Apr 2018;53(2):1008-1024. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Langelaan M, Baines RJ, de Bruijne MC, Wagner C. Association of admission and patient characteristics with quality of discharge letters: posthoc analysis of a retrospective study. BMC Health Serv Res. Mar 21, 2017;17(1):225. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Doyle MA, Malcolm JC, Liu D, Maranger J, Ooi TC, Keely E. Using a structured discharge letter template to improve communication during the transition from a specialized outpatient diabetes clinic to a primary care physician. Can J Diabetes. Dec 2015;39(6):457-466. [ CrossRef ] [ Medline ]
• Weetman K, Spencer R, Dale J, Scott E, Schnurr S. What makes a "successful" or "unsuccessful" discharge letter? Hospital clinician and general practitioner assessments of the quality of discharge letters. BMC Health Serv Res. Apr 15, 2021;21(1):349. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Schiele F, Lemesle G, Angoulvant D, Krempf M, Kownator S, Cheggour S, et al. Proposal for a standardized discharge letter after hospital stay for acute myocardial infarction. Eur Heart J Acute Cardiovasc Care. Oct 2020;9(7):788-801. [ CrossRef ] [ Medline ]
• Klar R, Pelikan E. Telemedicine in Germany. In: Kramme R, Hoffmann KP, Pozos RS, editors. Springer Handbook of Medical Technology. Berlin, Heidelberg. Springer; 2011.
• Mueller MA, Opitz R, Grandt D, Lehr T. The federal standard medication plan in practice: an observational cross-sectional study on prevalence and quality. Res Social Adm Pharm. Oct 2020;16(10):1370-1378. [ CrossRef ] [ Medline ]
• Ulmer I, Mildner C, Krämer I. PS-029 Feasibility of utilisation and patient satisfaction with a nationwide standardised electronic medication plan. Eur J Hospital Pharmacy. 2017;24:A239-A240. [ CrossRef ]
• Botermann L, Krueger K, Eickhoff C, Kloft C, Schulz M. Patients' handling of a standardized medication plan: a pilot study and method development. Patient Prefer Adherence. Apr 22, 2016;10:621-630. [ CrossRef ] [ Medline ]
• Dormann H, Maas R, Eickhoff C, Müller U, Schulz M, Brell D, et al. [Standardized national medication plan : the pilot projects MetropolMediplan 2016, model region Erfurt, and PRIMA]. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. Sep 25, 2018;61(9):1093-1102. [ CrossRef ] [ Medline ]
• Payen A, Godard-Sebillotte C, Sourial N, Soula J, Verloop D, Defebvre MM, et al. The impact of including a medication review in an integrated care pathway: a pilot study. Br J Clin Pharmacol. Mar 2023;89(3):1036-1045. [ CrossRef ] [ Medline ]
• Bugnon B, Geissbuhler A, Bischoff T, Bonnabry P, von Plessen C. Improving primary care medication processes by using shared electronic medication plans in Switzerland: lessons learned from a participatory action research study. JMIR Form Res. Jan 07, 2021;5(1):e22319. [ CrossRef ] [ Medline ]
• Eden KB, Totten AM, Kassakian SZ, Gorman PN, McDonagh MS, Devine B, et al. Barriers and facilitators to exchanging health information: a systematic review. Int J Med Inform. Apr 2016;88:44-51. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Pfeuffer N, Beyer A, Penndorf P, Leiz M, Radicke F, Hoffmann W, et al. Evaluation of a health information exchange system for geriatric health care in rural areas: development and technical acceptance study. JMIR Hum Factors. Sep 15, 2022;9(3):e34568. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Anderson JG. Social, ethical and legal barriers to e-health. Int J Med Inform. 2007;76(5-6):480-483. [ CrossRef ] [ Medline ]
• Sligo J, Gauld R, Roberts V, Villa L. A literature review for large-scale health information system project planning, implementation and evaluation. Int J Med Inform. Jan 2017;97:86-97. [ CrossRef ] [ Medline ]
• Jimma BL, Enyew DB. Barriers to the acceptance of electronic medical records from the perspective of physicians and nurses:a scoping review. Inform Med Unlocked. 2022;31:100991. [ CrossRef ]
• Pohlmann S, Kunz A, Ose D, Winkler EC, Brandner A, Poss-Doering R, et al. Digitalizing health services by implementing a personal electronic health record in Germany: qualitative analysis of fundamental prerequisites from the perspective of selected experts. J Med Internet Res. Jan 29, 2020;22(1):e15102. [ FREE Full text ] [ CrossRef ] [ Medline ]

Abbreviations

Edited by T Leung; submitted 09.03.23; peer-reviewed by P Nohl-Deryk, S Meister; comments to author 21.04.23; revised version received 13.06.23; accepted 31.01.24; published 26.03.24.

©Christoph Strumann, Lisa Pfau, Laila Wahle, Raphael Schreiber, Jost Steinhäuser. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 26.03.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 26 March 2024

A theoretical model for roller-shape design of three-roller continuous and synchronous calibration process of ovality and straightness for large thin-walled pipes

• Xueying Huang 1 , 2 &
• JinPing Gu 1  

Scientific Reports volume  14 , Article number:  7094 ( 2024 ) Cite this article

Metrics details

• Engineering
• Ocean sciences

Large thin-walled pipes are particularly suitable for oil and gas transport and auxiliary pipes in cold areas or deep sea-beds. At present, the rounding and straightening processes are completed independently, and the theoretical model for roller-shape design is analyzed only in a single direction. To solve this problem, a theoretical model for roller-shape design of three-roller continuous and synchronous adjusting straightness and ovality process for large thin-walled pipes is established. The established model is verified by numerical simulation and experimental research using 304 stainless steel pipes. The results show that the three roller-shape design schemes, including three-section, four-section and five-section, proposed based on the theoretical model, can obtain qualified formed pipes. Based on the model, the residual ovality, residual straightness and maximum residual stress of the three roller-shape schemes are discussed. The residual straightness can reach within 0.2%, and the residual ovality can reach within 1%. It verifies the applicability of the model and the feasibility of the process. The model can provide a theoretical basis for presetting the process parameters and optimizing the roller-shape.

Introduction

Large thin-walled pipes are widely used in oil, natural gas, aerospace and power industries. The most prominent of which is the construction of oil and gas pipelines. Natural gas has become the main energy in the world. Large thin-walled pipes are widely used in oil and gas transportation in cold zones or deep water. API Spec 5L (Line pipe Specification developed by the American Petroleum Institute’s Fifth Committee) has strict requirements for the straightness and ovality of finished welded pipes 1 . Affected by factors such as welding thermal stress, material properties and forming equipment, the straightness and ovality of the formed welded pipe cannot meet the industry standard. It needs to be straightened and rounded again. Expanding the pipe diameter, increasing the pressure and improving the steel grade are the development trends to further improve the transportation capacity of pipelines. The accuracy of the dimension of large thin-walled pipes is required to be higher and higher. Currently, the existing straightening and rounding processes are done independently. As a result, it is not easy to realize automated and intelligent production. On the other hand, a single roller-shape cannot solve the flattening problem of large thin-walled pipes. It is difficult to adjust the straightness and ovality to the optimal level at the same time. It seriously affects the on-site welding and pipeline safety. Therefore, the study of the roller-shape theoretical model to realize the cooperative control of straightness and ovality has become a key technology for the production of large thin-walled pipes. It is also a bottleneck problem restricting the construction of oil and gas pipelines.

The existing roller-shape design studies are only applicable to the straightening process. Roller-shape designs that combine both straightening and rounding have not yet appeared. Most of the research on roller-shape designs by researchers focuses on the engineering application. Its application scope is limited. Moreover, there is a lack of theoretical guidance on roller-shape design. It is difficult to carry out systematic research.

Equal curvature split die or equal curvature roller are adopted to realize the rounding process during the rounding process. The rounding process mainly includes: whole-diameter rounding, over-bending rounding and roller rounding. The production of large thin-walled pipes is usually rounded by adjusting a whole-diameter 2 , 3 . However, the method will change the circumference of pipes. When the size of circumference is qualified, the method is not applicable. Based on the above situation, Zhao et al., proposed the over-bending rounding process. A pair of equal curvature upper and lower flap dies with small curvature were used to apply pressure to pipes along the long axis. It made the long axis shorter and the short axis longer, and produced elastoplastic deformation to achieve the rounding 4 . This process is not suitable for the overall rounding of pipes. The roller rounding is a process in which the position of the pipe changes by the rotational motion of a roller of equal curvature 5 . As for the roller rounding, Yu et al. 6 , 7 , 8 proposed a three-roller rounding process. The upper roller is placed inside the pipe, which requires a cylinder turning device during the feeding process. However, when the length of the pipe is relatively long, the stiffness of the upper roller is not enough. In view of this, Huang et al. 9 , 10 proposed a rounding process in which three rollers are placed on the outside of pipes. This process can realize the overall and continuous rounding of pipes. It effectively reduces the residual stress and improve the quality of pipes.

To obtain high-quality, high-precision pipes, researchers have carried out a lot of research in the design of straightening rollers. The straightening process mainly includes pressure straightening and oblique roller straightening. Song et al. 11 , 12 proposed two straightening control strategies for large longitudinal welded pipes based on the three-point bending principle. The spacing between the fulcrums is a little big. So, there is a big deflection after deformation. Oblique roller straightening is used for pipes straightening. That is, based on three-step bending, the single-point bending straightening is converted into uninterrupted compression bending straightening. For the oblique roller straightening, Ma et al. 13 proposed a variable curvature roller-shape design, which was verified by theoretical analysis methods. Wang et al. 14 established a model of a ten-roller straightener by using hyperbolic roller shape. Yi et al. 15 , 16 set up a multi-roller wave straightening model and analyzed the influence of roller parameters on the straightening process. Ma et al. 17 built a segmented straightening roller and proved the effectiveness of the method. In summary, it can be seen that the design of the straightening roller plays a decisive role in the straightening process. Having a proper roller is the key to increasing straightening productivity and producing high-quality products. Based on this, Huang et al. proposed a “segmented” roller design for the first time. The proportion of the roller area, the curvature design, the elastic area ratio, and the feasibility of the process were studied 18 .

In this paper, a theoretical model of roller-shape design is established. This model is for the characteristics of large thin-walled pipes. It is suitable for continuous and synchronous calibration process of ovality and straightness with rollers. This model provides a theoretical basis for the setting of process parameters and the optimization of roller shapes. Based on the traditional assumptions, this paper establishes the curvature equations of the roller. Based on the obtained curvature equations, the springback curvature equations of the circumferential and axial directions of the pipe are established. According to the theoretical model and the design idea of “segmented”, three roller-shape schemes of “Three-section”, “Four-Section” and “Five-Section” are proposed. Numerical simulation analysis is carried out on the above schemes. The effects of the three roller-design schemes on the residual straightness, residual ovality and maximum residual stress are studied. The theoretical model is verified by numerical simulation. Additionally, through physical experiments, the feasibility of the “Five-Section” roller-shape scheme is carried out.

Process introduction

As shown in Fig.  1 , the main working parts of this process are three parallel rollers, including a convex roller (upper roller) and two concave rollers (lower roller). Two concave rollers are driven by servo motors to rotate simultaneously, while the pipe and convex rollers are driven to rotate under friction. At the same time, the pipe is continuously pushed along the slideway through the push plate to achieve the calibration process.

Schematic diagram of process.

As shown in Fig.  2 , the three rollers move synchronously towards the center of the pipe with the same radial reduction. The radial reduction of each roller is recorded as H .

where \(H\) is the radial reduction; \({R}_{1}\) is the radius of roller; \(R\) is the radius of pipe; \({H}_{j}\) is the distance from the center of pipe to the center of roller after loading.

Diagram of loading parameters.

Theoretical model of roller-shape design

Basic assumptions.

The pipe is continuous, homogeneous, and isotropic. The linear simple kinematic hardening (LSKH) constitutive model 19 is adopted, as shown in Fig.  3 .

where \({\sigma }_{s}\) is yield stress; \(D\) is plastic modulus; \(E\) is elastic modulus; \(\sigma\) is the stress; \(\varepsilon\) is the strain; \(\sigma_{0}\) is the intercept stress.

LSKH constitutive model.

Any cross-section of the pipe is always perpendicular to the geometric central axis, remains a plane during the deformation. There is no tilt or distortion between the two adjacent cross-sections 20 . So, the shear stress and shear strain are negligible.

The deformation of pipe follows the principle of volume invariance.

According to the theory of thin-walled shells, the change of wall thickness is ignored, namely \({\varepsilon }_{r}=0\) .

Because the movement of neutral layer is small for thin-walled pipe in the deformation process, it can be considered that the strain neutral layer, stress neutral layer and geometric central layer of the pipe are always coincided.

Roller meshing curve equation

The schematic diagram of roller shape is shown in Fig.  4 . A is the loading section, that is, the inlet end. To make the pipe enter the gap between three rollers and achieve radial reduction, it is designed to be truncated cone shape. D is the unloading section, that is, the outlet end. Its shape is also designed to be truncated cone to ensure that the pipe can be smoothly unloaded after calibration. B1 and B2 are the ovality calibration sections, where the shape is cylindrical. C is the synchronous calibrating straightness and ovality section.

Schematic diagram of the “Five-section” roller shape (On the right side of this figure is a one-half convex roller). A, Loading section; B (B1 and B2), ovality calibration section; C, ovality and straightness calibration section; D, unloading section.

Establish the Cartesian coordinate system OXYZ . Let the roller and the pipe mesh at point M , and point M is ( x, y, z ) at the OXYZ coordinates. Since the pipe rotates between three rollers, any point on the surface of the pipe alternately undergoes variable curvature. The point M is always in line with the change in curvature on the roller.

Let the total length of the roller be L g . Taking the “Five-section” roller as an example, the proportional parts of the loading section, the ovality calibration section, the ovality and straightness calibration section, the supplementary ovality calibration section and the unloading section of the roller are n 1 : n 2 : n 3 : n 4 : n 5 respectively.

Loading section (section A)

According to the shape diagram of the roller, it can be seen that,

where, \(l = \frac{{L_{\text{g}} }}{{{\text{n}}_{1} + {\text{n}}_{2} + {\text{n}}_{3} + {\text{n}}_{4} + {\text{n}}_{5} }}\) is the length of each part; \({r}_{n}\) is the radius at the intersection of sections B and A; \({r}_{1}\) is the minimum radius of section A.

In the OXZ plane of the Fig.  4 , set the curve equation of the section A as

where, b is the coefficient of the primary equation. Based on the geometric characteristics of section A, the boundary conditions can be obtained,

Substituting the boundary conditions into Eq. ( 5 ),

So, the curve equation for section A is,

Ovality calibration section (section B)

In the OXZ plane of the Fig.  4 , the section B is a straight segment with a curvature of k  = 0. The curve equation for the section B is:

Based on the geometric characteristics of section B, the boundary conditions can be obtained,

By substituting the boundary conditions into Eq. ( 9 ), the curve equation for section B can be obtained as

Ovality and straightness calibration section

In the OXZ plane of the Fig.  4 , the curve equation of the section C is hyperbolic 18 .

where, a and b are the coefficients of the hyperbolic equation. Based on the geometric characteristics of section A, the boundary conditions can be obtained,

where, \({r}_{m}\) is the radius of the center position of the section C.

Substitute the boundary conditions into Eq. ( 12 ),

Therefore, the curve equation for the section C is,

By substituting the above curve equation with coordinates, it can be obtained that

Therefore, the curve equation shown can be expressed as,

The curvature equation for section C can be expressed as,

Unloading section

where, \(H_{{{\text{max}}}}\) is the maximum amount of radial reduction.

According to the shape of the roller,

where, \(r_{2}\) is the minimum radius of the section D.

In the OXZ plane of the Fig.  4 , let the curve equation for the section D be,

Based on the geometric characteristics of section D, the boundary conditions can be obtained,

Substitute the boundary conditions into Eq. ( 21 ),

Therefore, the curve equation for the section D is,

In summary, the roller curve equation is,

The equation for the curvature of the roller is,

Curvature after springback

It is assumed that the pipe fits perfectly with the three rollers during the process. Then, the curvature at any point on the surface of the roller coincides with the curvature of the pipe after loading.

Since the deformation of each section in the process is continuous, the initial curvature is the curvature after the last bending and springback. According to Ref. 21 , the recursive equation for reciprocating bending can be expressed as

where, n is the number of reciprocating bends; \(K_{pn}\) is the springback curvature after reciprocating bending n times; \(K_{1}\) is the curvature after the first bending; \(K_{0}\) is the initial curvature; \(M_{1}\) is the loading moment of the first bending; \(K_{n}\) is the curvature after reciprocating bending n times; \(K_{{p\left( {n - 1} \right)}}\) is the springback curvature after reciprocating bending n -1 times; \(M_{n}\) is the loading moment of the n th bending; I is the moment of inertia of the pipe cross-section, \(I = \frac{\pi }{4}\left( {R_{1}^{4} - R_{2}^{4} } \right)\) , \(R_{1}\) is the inner diameter of the pipe, \(R_{2}\) is the outer diameter of the pipe.

Based on small curvature plane bending springback theory, the springbck curvature of the pipe is deduced after reciprocating bending n times along the axial direction by using the mathematical induction method.

• Straightness

When the first bending is reversed, the uniform equation for curvature after springback is

where, \(K_{2}\) is the curvature after the second bending; \(K_{n - 1}\) is the curvature after reciprocating bending n -1 times; \(K_{n - 2}\) is the curvature after reciprocating bending n -2 times.

When the first bending is positive, the uniform equation for curvature after springback is

Equations ( 28 ) and ( 29 ) can be expressed uniformly as

Equation ( 30 ) proves that multiple reciprocating bending can annihilate the difference in the axial initial curvature of the pipe, so that the curvature can be unified to the same direction and value. The uniform curvature is related to the material properties, the inner and outer diameters of the pipe, and the bending curvature. From the above analysis, it can be seen that the straightening deformation process belongs to a small deformation. That is, when the loading curvature \({K}_{n}\) tends to 0, the axial curvature after reciprocating bending and springback is also close to 0. The straightening process is achieved.

Equations ( 31 ) and ( 32 ) can be expressed uniformly as

Equation ( 33 ) proves that multiple reciprocating bending can annihilate the difference in the circumferential initial curvature of the pipe. As a result, the curvature is unified to the same direction and value. The uniform curvature is related to material properties, wall thickness, and bending curvature. The rounding deformation process belongs to a small deformation. That is, when the loading curvature gradually tends to a certain value, the circumferential curvature of the pipe after reciprocating bending gradually tends to be consistent. The rounding process is achieved.

• Roller-shape design

According to the “segmented” design idea and theoretical model, three roller-shape schemes are proposed. That is, “Three-Section” (loading section—ovality and straightness calibration section—unloading section), “Four-Section” (loading section—ovality calibration section—ovality and straightness calibration section—unloading section or loading section—ovality and straightness calibration section—ovality calibration section—unloading section), and “Five-Section” (loading section- ovality calibration section—ovality and straightness calibration section—ovality calibration section—unloading section), as shown in Fig.  5 .

Roller-shape design scheme.

Finite element model

The four finite element models for the four roller-shape designs are built using the ABAQUS 6.10 software package. Take the “Four-Section” roller-shape design as an example, as shown in Fig.  6 . The geometric dimension of the pipe and roller and the mechanical properties of the pipe are shown in Tables 1 , 2 and 3 . According to the characteristics of the process, the pipe and the three rollers are modeled. Take the “Four-Section” roller-shape design as an example, an 8-node hexahedral linear uncoordinated mode element (C3D8R) is used to discretize the pipe. The total number of nodes is 31,020 and the number of units is 24,640. The three rollers are modeled as discrete rigid bodies. The contact between the pipe and the roller is set to pure master–slave contact and motion contact conditions, and the friction coefficient is 0.2. The speed of the pipe in the axial direction is 10 mm/s. The rotational speed of the two concave rollers is 6.28 rad/s.

A finite element model of the “Four-Section” roller-shaped design.

Results and discussion

Taking 304 pipes as the research object, the three-roller continuous and synchronous calibration process of ovality and straightness is analyzed. The equivalent stress distribution of the pipe during the process, as shown in Fig.  7 . The equivalent stress on the inner and outer surfaces gradually increases and exceeds the yield stress of the material (294 MPa) during the propulsion of the pipe. Additionally, the circumferential and axial directions of the pipe continuously undergo reciprocating bending. The circumferential and axial curvature of the pipe are unified.

Equivalent stress distribution during this process.

Residual ovality

The results of residual ovality obtained by the different roller-shape designs are shown in Fig.  8 . The residual ovality decreases gradually with the increase of the radial reduction H . When H reaches 1.5 mm, the “Five-Section” and “Four-Section” roller-shape schemes are closer to the theoretical values than the “Three-Section” scheme. The residual ovality can reach less than 0.4%, which meets the industrial requirements of large thin-walled pipes. The above results show that the straight section of the roller (section B) is more conducive to rounding than the variable curvature section (section C). The larger the proportion of section B, the smaller the residual ovality.

Results of residual ovality obtained by the different roller-shape designs. (The number of reciprocating bends n tends to be 140).

Residual straightness

The results of residual straightness obtained by the different roller-shape designs are shown in Fig.  9 . The residual straightness decreases gradually with the increase of the radial reduction H . The “Three-Section” roller-shape scheme is closer to the theoretical values than the “Four-Section” and “Five-Section” schemes. The residual straightness of three schemes can reach within 0.2%, which meets the industrial requirements of large thin-walled pipes. The above results show that the larger the proportion of section C, the smaller the residual straightness.

Results of residual straightness obtained by the different roller-shape designs. (The number of reciprocating bends n tends to be 140).

Maximum residual stress

Residual stress directly affects the service performance of large thin-walled pipes. Figure  10 is the equivalent stress distribution contour diagram of the straightened pipe of the “Four-Section” scheme when the H is 1.0 mm. The pipe is not unstable and does not undergo serious distortion. The main reason is the inconsistency of the deformation of the inner and outer of the pipe. The residual stresses are mainly concentrated at the outlet end of the pipe. The whole pipe is distributed in a ring shape, which gradually increases along the z -axis. By analyzing the distribution of the equivalent stress, the contact between the roller and the pipe can be inferred. It provides a reference for the correction of roller-shape design and the optimization of process parameters.

Equivalent stress distribution contour diagram of the straightened pipe. (“Four-Section” scheme; H is 1.0 mm).

The results of residual stress obtained by the different roller-shape designs are shown in Fig.  11 . The residual stress increases gradually with the increase of the radial reduction H . When the same H is applied, the residual stresses of the three schemes do not differ by more than 10 MPa. The maximum residual stress shall not exceed 235 MPa.

Results of residual stress obtained by the different roller-shape designs. (The number of reciprocating bends n tends to be 140).

Experimental validations

The experimental device for pipe calibration is shown in Fig.  12 . The device can realize both continuous rounding and continuous straightening of large thin-walled pipes. The geometric dimensions and mechanical properties of experimental pipes are shown in Table 4 . The geometric dimension of experimental rollers is shown in Table 5 .

Experimental device for pipe calibration. 1, Screw. 2, Lead screw drive. 3, Servo motor. 4, Support assembly. 5, Push plate. 6, Control cabinet. 7, Pipe. 8, Upper roller. 9, Frame. 10, Lower roller. 11, Slider. 12, Pedestal.

As shown in Fig.  12 , the roller is connected with the slider fixed on the frame via bearing. The slider can slide vertically along the frame surface via a screw to adjust the radial reduction of the three rollers. The support assembly keeps the balance of the pipe. The servo motor drives two lower rollers to rotate synchronously, which drives the pipe and the upper roller to start turning. At the same time, the push plate drives the pipe to move along the slideway. So far, the calibration process of the pipe is realized.

The 3000iTM series portable coordinate measuring instrument produced by CimCore was used to obtain the coordinates of the points around the outer diameter of the pipe before and after deforming. The coordinates of these points were processed to obtain the straightness and ovality of the pipe.

As shown in Table 6 , the ovality and straightness of the pipe are compared. It is found that the residual ovality can reach less than 1% and the residual straightness can reach less than 0.2%, which meets the industrial standard. The above proves the feasibility of the process. In the future, we will continue to use the simulation results to verify the calibration effect of the roller-shape scheme. Forming effect of 304 stainless steel pipes are shown in Fig.  13 .

Forming effect of 304 stainless steel pipes.

Conclusions

A theoretical model for roller-shape design of three-roller continuous and synchronous adjusting straightness and ovality process for large thin-walled pipes is proposed. The model can provide a theoretical basis for presetting the process parameters and optimizing the roller-shape.

The residual ovality and residual straightness of pipes decrease with the increase of the radial reduction H . The larger the proportion of section B, the smaller the residual ovality. The larger the proportion of section C, the smaller the residual straightness. The maximum residual stress increases gradually with the increase of the radial reduction H .

The residual ovality can reach less than 1% and the residual straightness can reach less than 0.2%, which meets the industrial standard.

Ethical approval

The authors declare that this manuscript was not submitted to more than one journal for simultaneous consideration. Also, the submitted work is original and not have been published elsewhere in any form or language.

Consent to participate and publish

The authors declare that they participated in this paper willingly and the authors declare to consent to the publication of this paper.

ANSI/API SPEC 5L-2012, Specification for Line Pipe (FORTY-FIFTH EDITION; ERTA 1: April 2015) [S]. Standard number: ANSI/API SPEC 5L-2012 Release date: 2012-12-01 Issued By: Foreign-Foreign-American National Standards Institute US-ANSI

Ren, Q., Zou, T. X., Li, D. Y., Tang, D. & Peng, Y. H. Numerical study on the X80 UOE pipe forming process. J. Mater. Process. Technol. 215 , 264–277 (2015). https://doi.org/10.1016/j.jmatprotec.2014.08.013

Article   CAS   Google Scholar  

Yin, J., Zhao, J., Sun, H. L. & Zhan, P. P. Molded precise reducer rounding of large pipes. Opt. Precis. Eng. 19 (9), 2072–2078 (2011) https://doi.org/10.3788/OPE.20111909.2072

Article   Google Scholar  

Zhao, J., Zhan, P. P., Ma, R. & Zhai, R. X. Prediction and control of springback in setting round process for pipe-end of large pipe. Int. J. Press. Vessels Pip. 116 , 56–64 (2014) https://doi.org/10.1016/j.ijpvp.2014.01.006

Guo, L. P. & Li, Z. W. Commonly used three-roller bending machine structure and transmission mode. China Metalform. Equip. Manufact. Technol. 45 (6), 21–23 (2010) https://doi.org/10.3969/j.issn.1672-0121.2010.06.005

ADS   Google Scholar  

Yu, G. C., Zhao, J., Xing, J. J., Zhao, F. P. & Li, S. L. Research on symmetrical three-roller rounding process. J. Mech. Eng. 53 (14), 136–143 (2017) https://doi.org/10.3901/JME.2017.14.136

Yu, G. C., Zhai, R. X., Zhao, J. & Ma, R. Theoretical analysis and numerical simulation on the process mechanism of two-roller straightening. Int. J. Adv. Manufact. Technol. 94 (9), 4011–4021 (2018) https://doi.org/10.1007/s00170-017-1120-5

Zhao, J., Yu, G. C. & Wang, H. R. A roller-type continuous and synchronous calibrating straightness and roundness equipment and processing method for pipes. Hebei: CN201611012103.5 (2017).

Huang, X. Y. et al. Numerical simulation and experimental investigations on a three-roller setting round process for thin-walled pipes. Int. J. Adv. Manufact. Technol. 107 , 355–369 (2020).

Huang, X. Y. et al. Three-roller continuous setting round process for longitudinally submerged arc welding pipes. Trans. Nonferr. Metals Soc. China 31 , 1411–1426 (2021).

Zhao, J. & Song, X. K. Control strategy of multi-point bending one-off straightening process for lsaw pipes. Int. J. Adv. Manufact. Technol. 72 (9–12), 1615–1624 (2014).

Zhao, J., Song, X. K., Cao, H. Q. & Liu, J. The new process principle of multi-point bending and one-time straightening of large longitudinal welded pipes. J. Mech. Eng. 50 (2), 92–97 (2014).

Ma, L. D., Huang, Q. X., Chen, S. & Ma, H. X. Optimization of varying curvature profile of two-roll bar straightener. Iron Steel 53 (9), 45–52 (2018).

Google Scholar  

Wang, C. & Hu, Z. X. TA9 tube ten roll straightening process simulation. Modern Manufact. Technol. Equip. 5 , 62–63 (2018).

CAS   Google Scholar  

Yi, G. D., Gu, Y., Qiu, L. M. & Wang, C. Multi-roll levelling method for longitudinal waves in metal sheet based on a coupled curvature integration model. Int. J. Adv. Manufact. Technol. 106 , 2721–2734 (2020).

Yi, G. D., Gu, Y., Qiu, L. M. & Wang, C. Multi-roll levelling for wave defects of metal sheets based on the beam-membrane method. Int. J. Adv. Manufact. Technol. 105 , 4783–4795 (2019).

Ma, Z. Y., Ma, L. F., Wang, R. J., Huang, Q. X. & Ma, L. D. Study on control strategies of two-roll straightening for bar high precision straightening. J. Mech. Eng. 53 (20), 77–88 (2017).

Huang, X. Y. et al. Roller design and numerical simulation of three-roller continuous and synchronous adjusting straightness and roundness process on LSAW pipes. J. Mech. Eng. 57 (10), 148–159 (2021).

Yu, G. C., Zhao, J., Ma, R. & Zhai, R. X. Uniform curvature theorem by reciprocating bending and its experimental verification. Chin. J. Mech. Eng. 52 , 57–63 (2016).

Zhang, Z. Q., Yan, Y. H., Yang, H. L. & Wang, L. Modeling and analysis of the limit bending-radius for straightening a thin-walled tube with equal curvature. J. Mech. Strength 37 , 651–656 (2015).

Zhao, J., Yin, J., Ma, R. & Ma, L. X. Springback equation of small curvature plane bending. Sci. China Technol. Sci. 54 , 2386–2396 (2011).

Article   ADS   Google Scholar  

Download references

Acknowledgements

The authors would like to thank the General Project of Department of Education of Zhejiang Province for their financial support.

This work was funded by General Project of Department of Education of Zhejiang Province, (Grant no. Y202352463).

Author information

Authors and affiliations.

School of Digital Technology and Engineering, Ningbo University of Finance and Economics, Ningbo City, 315175, People’s Republic of China

Xueying Huang & JinPing Gu

Ministry Education, Key Lab Mech Theory & Equipment Design, Tianjin University, Tianjin, 300350, People’s Republic of China

Xueying Huang

You can also search for this author in PubMed   Google Scholar

Contributions

Xueying Huang: conceptualization, methodology, validation, formal analysis, investigation, data curation, writing—original draft, writing—review & editing, software, visualization. Jinping Gu: conceptualization, methodology, formal analysis, supervision, writing—review & editing.

Corresponding author

Correspondence to Xueying Huang .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Huang, X., Gu, J. A theoretical model for roller-shape design of three-roller continuous and synchronous calibration process of ovality and straightness for large thin-walled pipes. Sci Rep 14 , 7094 (2024). https://doi.org/10.1038/s41598-024-57753-0

Download citation

Received : 14 February 2024

Accepted : 21 March 2024

Published : 26 March 2024

DOI : https://doi.org/10.1038/s41598-024-57753-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Large thin-walled pipes
• Calibration process of ovality and straightness
• Theoretical model

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Design & Make with Autodesk

• Architecture, Engineering, Construction & Operations
• Product Design & Manufacturing
• Media & Entertainment
• Emerging Tech
• Point of View

Generative Design

Quickly generate high-performing design alternatives—many that you’d never think of on your own—from a single idea. With generative design, there is no single solution; instead, there are multiple great solutions. You choose the design that best fits your needs.

What is generative design?

Generative design is a design exploration process. Designers or engineers input design goals into the generative design software, along with parameters such as performance or spatial requirements, materials, manufacturing methods, and cost constraints. The software explores all the possible permutations of a solution, quickly generating design alternatives. It tests and learns from each iteration what works and what doesn’t.

Generative design by industry

Generative Design for Product Design and Manufacturing

Helps you explore more CAD-ready design options that are optimized for cost, material, and production method.

Image courtesy of Briggs Automotive Company

Generative Design for Architecture, Engineering & Construction

Helps you explore solutions by rapidly testing, analyzing, and evaluating iterations for building design challenges.

Benefits of generative design

Explore a wider range of design options

In the time you can create one idea, a computer can generate thousands, along with the data to prove which designs perform best.

Make impossible designs possible

Generative design lets you create optimized complex shapes and internal lattices. Some of these forms are impossible to make with traditional manufacturing methods. Instead, they're built using new additive manufacturing methods.

Optimize for materials and manufacturing methods

Set goals and parameters, and the software will create high-performing design options based on those constraints. The software resolves conflicting design constraints so you can focus on innovating.

From aerospace to architecture and construction

Generative design is out of the lab and being used in the field. Learn how the world's most innovative companies are using this technology today.

From light-weighting components to parts consolidation, Autodesk generative design is being used by companies shaping the future of the automotive industry.

With design and manufacturing goals around weight reduction, environmental impact, and safety, the aerospace industry is adopting generative design to explore new design options and improve performance.

Consumer goods

The ability to simultaneously generate multiple CAD-ready, process-aware solutions to a design problem has a positive impact on innovation and productivity.

Architecture and construction

As demonstrated with the layout and design of the Autodesk Toronto office, generative design technology can help you explore solutions for goals and constraints outside of manufacturing.

Industrial machinery

Manufacturers of specialty tools and equipment are proving that Autodesk generative design can help them break innovation barriers.

Building products

Industrial building products suppliers are proving complex assemblies can be simplified with the help of generative design and additive manufacturing.

Generative design customer stories

Driving mobility innovation

The Hyundai Motor Group is pushing the boundaries of vehicle development and reimagining how vehicles might traverse the world with the help of generative design.

Discovering new impact with generative design

Edera Safety uses Fusion 360 to design, prototype, and begin manufacturing an innovative flight mask to help keep service workers safer and healthier during the COVID pandemic.

Mainstreaming sustainability

MG AEC used generative design for proof-of-concept that high-performance building design can, and in the face of climate crisis,  must , become mainstream.

Related manufacturing technologies

Autodesk has software to cover all your design for manufacturing needs. Learn about some of the other innovative technology we produce.

Internal lattice structures

Create lightweight parts with performance characteristics specific to your application.

Topology optimization

Generate forms that are optimized for stiffness and weight, based on the loads and constraints of the part.

Trabecular structures

Precisely scale and distribute tiny pores through solid materials, creating surface roughness to mimic bone in medical implants to help patients heal.

Complex part simulation

Apply thermomechanical modeling to large geometries designed for additive manufacturing.

Privacy | Do not sell or share my personal information | Cookie preferences | Report noncompliance | Terms of use | Legal  |  © 2024 Autodesk Inc. All rights reserved

Automatic welding-robot programming based on product-process-resource models

• ORIGINAL ARTICLE
• Open access
• Published: 25 March 2024

Cite this article

You have full access to this open access article

• Ioan-Matei Sarivan   ORCID: orcid.org/0000-0002-1469-9639 1 ,
• Ole Madsen 1 &
• Brian Vejrum Wæhrens 1  

This paper describes a novel end-to-end approach for automatic welding-robot programming based on a product-process-resource (PPR) model, for one-of-a-kind manufacturing systems. Traditionally, the information needed to program a welding robot is processed and transferred along the manufacturing organisation’s value chain by using several stand-alone digital systems which require extensive human input and high skill to operate. A PPR model is proposed through this research as a platform for storing and processing the necessary information along the value chain seamlessly. Unlike existing approaches which make use of complex algorithms to automatically identify the weldment seams, the approach suggested in this research makes use of information already digitalised by design engineers under the form of ISO 2553:2019 compliant weldment annotations. Hence, the PPR model contains the weldment annotations; it enables the automatic programming of welding robots and reduces human input down to a few minutes only. The applicability in manufacturing of the theoretical concept is demonstrated through technical implementations tested in the laboratory and on the value chain of an engineering-to-order (ETO) industrial partner involved in the metal fabrication industry. The experiments were conducted by creating several products using the proposed artefact. Experiments show that automatic programming of welding robots can be achieved using PPR models. The conducted experiments showed a reduction of about 80% in human input measured in terms of time, when using the proposed solution. The reduction of the human input can free up skilled labour resource which ETO SMEs can reallocate to other tasks.

Avoid common mistakes on your manuscript.

1 Introduction

In this paper, the theoretical concept of a product-process-resource (PPR) model for automatic welding-robot programming is proposed and then its potential to be adopted by practitioners is tested through practical experiments in the laboratory and at a partner manufacturing company. This concept lays the basis of organising the PPR [ 1 ] data required to program a welding robot in a centralised manner which facilitates the fast and automatic programming of welding robots proving their high utilisation rate in high-mix low-volume production environments to be feasible. Traditionally, the computer-aided design (CAD) environment is disconnected from the offline programming (OLP) environment for welding robots. This discontinuity within the digital value chain of the company requires redundant human input to re-digitalise data which is often already present in CAD models but lost upon transfer in the OLP environment. The proposed PPR model enables the seamless integration of the multiple operations necessary to be conducted before obtaining a functional robot-welding program, by making use of the CAD system’s compliance with the ISO 2553 standard. Thus, an end-to-end digital pipeline can be put in place to automatically program welding robots without the need of highly skilled labour, resulting in lower production cost and dramatically faster lead time, without affecting the flexibility associated with a one-of-a-kind manufacturing system.

Enterprises located in high-cost environments, e.g. Denmark, face the challenge of an ever-increasing competition from low-cost environments [ 2 ]. Furthermore, the Danish metal fabrication industry is facing a shortage of skilled welders [ 3 ]. This shortage compels enterprises in high-cost environments to rely on automation, which is highly challenging in one-of-a-kind manufacturing setups. With every product being unique, companies with one-of-a-kind manufacturing systems must invest heavily in engineering resources to create new product designs and to reprogram the welding robots every time a new order is received. Similarly with the situation of skilled welders, robotics engineers are expensive too. Therefore, labour cost is one of main challenges which ETO manufacturing enterprises are facing day by day. By conducting the research presented in this paper, it is desired to investigate a novel and effective tool which ETO manufacturing enterprises can use to reduce the manual inputs needed from robotics engineers to program welding robots. The time which is normally used by the robotics engineers to program the robots can then be reallocated to other tasks, thus reducing the cost of programming a welding robot with manual inputs.

Programming of welding robots is a process involving advanced competencies and time resources. Bejlegaard et al. and Tolio et al. show however in their studies that digital technologies tailored to fit the needs of the ETO small-medium-enterprises (SME) can reduce many of the trade-offs which concern these companies, such as flexibility versus efficiency [ 4 , 5 ]. The artefact which is described in Section 4 of this paper is meant to drastically reduce the time and the competencies level required to reprogram welding robots, while at the same time facilitating lower production cost, faster lead time and increased utilisation of welding robots when compared with traditional, manual programming methods. It is showed through the practical experiments which were setup to demonstrate the potential of the proposed solution that a reduction of 80% in human input, measured by time, can be achieved when using the proposed solution.

2 Theoretical background and related work

To program welding robots, a thorough understanding of the product, the manufacturing process and the manufacturing system’s capabilities are required. In ETO value chains, engineering resources are used to either create new products based on customer’s requirements or adapt existing product designs to satisfy the customer’s demand. A literature study was conducted on existing literature in the domain of product-process-resource models and how product-process-resource data can be structured in such manner that it allows fast redesign based on the customer’s requirements and automatic programming of welding robots. The findings in the fields of PPR models and methods for automatic programming of welding robots are presented in the following.

2.1 PPR models

Integrating the customer’s requirements into the product design is a matter of high importance when it comes to ETO supply chains and this can put extraordinary strain on the delivery dependability of products, especially for highly individualised products, where the customer is involved in the product design and engineering process [ 6 ]. The scalability and the personalisation of the product’s architecture in the most efficient manner possible is a topic investigated extensively in the literature [ 6 , 7 , 8 ]. Tan et al. propose an open product architecture design, which makes use of several optimisation criteria to come up with the best solution to satisfy both the customer’s requirements and the manufacturing company’s targets for production cost. By using genetic algorithm solvers, the optimal attribute selection for the product is provided [ 8 ]. Customers can interact with the product architecture through product configuration systems, which facilitates customer-centric product development. By using the open architecture product platform conceptualised by Zheng et al., customers can either choose a configure-to-order approach on their order or an engineer-to-order approach to have access to the full suite of engineering services provided by the company [ 6 ]. A similar approach is investigated by Bejlegaard et al. where the case company provides emerging customers access to a focused product portfolio, in a made-to-order regime, while frequent customers are given access to innovative products in an ETO regime [ 4 ].

To program welding robots, information is required concerning the product design, the process and the resources needed to perform the weldments [ 9 ]. The systematic overview on product-process-resource information is also known as the “PPR perspective” and it is formalised in existing literature through PPR models [ 10 , 11 , 12 ]. PPR models include in their scope details about the resource capability, or skill required to conduct the manufacturing processes necessary to product an item. Brecher et al. and Ferrer et al. use PPR models to particularly formalise the process required and the skills needed to sustain assembly operations. The information required to set up the assembly process and the resource dimensions is extracted from the CAD through means of ontology-tailored meta models or semantic web rule language [ 13 , 14 ]. A similar approach is taken by Ahmed et al. by using a PPR model in their study case for automotive engine assembly lines. The PPR model is queried for information in two stages, first to determine the capabilities of the available resources given the product requirements, and then to align the resource capabilities with the proper assembly process parameters [ 1 ]. As Ahmed et al. point out, the PPR model has the potential to increase engineering productivity and bring together information and knowledge under a single knowledge base. The same view on the PPR concept in shared by Winkler et al. who use PPR to trace the product requirements along the value chain and facilitate the propagation of information across the product-process and resource domains which form an engineering asset network [ 15 ]. The implementation of PPR view actively contributes at the increase of the adaptability of the production systems and allows for faster new product insertion with reduced human intervention [ 16 , 17 ].

While the work mentioned above showcases the benefits of using the PPR perspective on assembly lines, no substantial literature and empirical research was found on how PPR models can be used for welding applications, specifically for automatic programming of welding robots in one-of-a-kind manufacturing systems. Therefore, the approach presented in this paper aims towards defining a PPR model, using the PPR perspective to enable automatic programming of welding robots and fast redesign of products, all within an end-to-end integrated information flow. The environment in which product data can be manipulated by the product engineers is the CAD software tool. A deep dive into existing methods on how automatic programming of welding robots can be obtained with the data existing in CAD models is given in the following.

2.2 CAD-based automatic programming of welding robots

Welding is one of the most exploited applications for industrial robotics [ 18 ] and the programming of welding robots is a well-researched field that has been in the attention of both researchers and practitioners for several decades. Current research in this field is focused on the reduction of time, skills and overall costs involved in programming and deploying a welding-robot program. This is especially important for one-of-a-kind production setups where welding robots must be reprogrammed every time a new customer order is received. There are three major approaches towards automatic programming of welding robots [ 19 ]: CAD-based programming, sensor-based programming and hybrid programming.

As mentioned in the previous subsection, the main environment where product data is manipulated is the CAD model. Therefore, the focus of the manuscript is placed on CAD-based programming methods only. The computer-aided design (CAD) system lays at the core of the engineering-to-order (ETO) supply chains facilitating the creation and the transformation of the product’s design by design engineers [ 20 ]. Methods for automatic programming of welding robots using CAD models are at the core of intelligent weldment systems (IWS) [ 21 ]. The CAD model is needed within offline programming systems (OLP) for collision detection and avoidance algorithms [ 22 , 23 , 24 , 25 , 26 ] which allow automatic robot programming (ARP) [ 27 , 28 ]. However, this is only possible if information about the locations and the geometry of the weldments exists. Traditionally, this information is manually inserted by the robotics engineers using various human machine interfaces (HMI) either readily available in the OLP system or custom made to make the programming process as intuitive as possible and independent of programming languages [ 29 ]. The information that is manually inserted in the OLP is gathered from the technical drafts which are automatically generated by modern CAD systems under the form of 2D technical drawings annotated with the positions of the weldments as stipulated by ISO Standard 2553:2019.

Recent developments in the field of ARP for welding applications have been focused on the automation of weldment-related information transfer from the CAD environment directly to the OLP environment. Feature recognition algorithms are used to process the CAD model stored in STEP (Standard for the Exchange of Product Data ISO 10303) files [ 30 ]. Xuan and Ngoc make use of the hierarchical structure of the STEP format to access the set of faces and edges which make up the geometry of the product designed in CAD [ 31 ]. The relationships between the faces and the edges are evaluated and the coordinates of the weldments are determined. Kuss et al. exploit a similar approach augmented by rules, which delimit the recognition procedure to fillet weldments only [ 32 ]. By using the surface normal of the extracted faces, Kuss et al. can determine the angle between the surfaces which are to be welded. Fang et al. also make use of data extracted from the CAD model to discretise the weldment by using methods available in the SolidWorks CAD system [ 33 ].

Modern CAD systems allow product design engineers to model the geometry of the product, perform engineering analysis, review the design and finally create the technical drafts of the product. The technical draft contains 2D drawings annotated with the positions of the weldments [ 34 ]. Weldments are considered rigid engineering connections in CAD assemblies and by using modern CAD tools; weldment information can be overlaid on CAD models as directed by ISO Standard 2553:2019 [ 35 , 36 , 37 , 38 , 39 ]. This is also supported to some extent in the second edition of STEP AP242 and an implementation for extracting information about the weldments directly from the CAD model without feature recognition algorithms is given by Mohammed et al. [ 40 ]. Mohammed et al. show that process information like current, voltage and torch speed can be additionally overlaid on the STEP file format by intervening directly on the extensible markup language (XML) structure of the STEP file.

To obtain reliable information for ARP purposes, Tran et al. use a composed implementation using information obtained both from STEP files with ISO Standard 2553:2019 annotations, and feature recognition tools [ 41 ]. The annotations present in the STEP files are created by Tran et al. using the tools available in the Siemens NX CAD system [ 39 ]. The weldment-related information obtained by Tran et al. can be further used to automatically generate a robot program with collision-free movements between and across the weldments [ 42 ].

Larkin et al. classify the methods for extracting weldment information from CAD models using the CAD tools available, as either automatic, using the feature recognition algorithms, or semi-automatic, by using manual inputs to create the necessary annotations on the CAD model which define weldments [ 43 ]. It is to be noted, however, that weldment annotations fall under the ISO Standard 2553:2019, which stipulates the proper drafting of products with rigid engineering connection held together by weldments. And as mentioned before, there are several CAD systems commercially available which are compliant with this standard. Therefore, the proper annotation of weldments falls under the responsibility of the product design engineer. Moreover, as shown in the related work presented above, information encapsulated in both the product and process domain can be stored in modern CAD file formats. While research similar with the one made by Tran et al. [ 41 , 42 ] and by Mohammed et al. [ 40 ] offer examples on how data can be extracted directly from CAD models to program welding robots, a research gap is identified in the lack of a general approach for extracting relevant information from CAD systems compliant with the ISO 2553:2019 standard. The approach presented in this paper and the proposed theoretical concept addresses CAD software compliant with the ISO 2553:2019 standard independent of the supplier.

It is concluded from the literature study made that the ETO case company may benefit from a technical solution which allows fast adjustment of product designs without the need for an iterative process, ideally where the customer can have more control over the product’s design, but without the assistance of the engineers. This is highlighted in Table  1 by directly comparing the methods mentioned above with the method proposed in this research. The main comparison dimension used is the amount of human input necessary to obtain a new robot program after having modified the CAD model to obtain a new product variant. Currently, when a company is using CAD software and OLP software, the data exchange between these two environments is made manually and is disconnected. The digital integration of CAD and OLP is expected that the information flow will become streamlined, and reduce the required human input needed every time a new product variant is created.

3 Product-process-resource model

By linking the findings in the literature survey on the PPR perspective and automatic programming of welding robots based on CAD models, a PPR model for automatic programming of welding robots is hereby proposed. The product design engineers use CAD tools to document information about the product beyond the geometry of the product. The information can be overlaid on CAD models using modern CAD tools, can be used to set up the production processes and ensure that the process can be conducted using the available resources.

3.1 Theoretical foundations

Ahmad et al. offer a decomposition of the PPR model [ 1 ] used for automatic assembly applications as follows:

The product domain contains all the available information about the product’s characteristics, the product’s family and variations. The product domain is often fully encompassed in the CAD model of the product. The product can be an assembly of multiple components which share various engineering connections or liaisons (fixed, cylindrical, prismatic or rigid).

The process domain encapsulates the knowledge and information necessary to set up the operations required to manufacture the product. As described by Ahmad et al. [ 1 ], a production operation is realised through several processes which are sustained using resources. Some processes require certain skills and resources to be conducted.

The resource domain concerns the set of physical and non-physical entities involved in supporting the production process. The physical resources may be manufacturing tools and various consumables, while non-physical resources can be the tacit or documented knowledge regarding how the production process can be set up.

The product model concept proposed is shown in the diagram displayed in Fig.  1 . The Unified Modelling Language (UML) was selected to describe the PPR model proposed. UML provides a general and flexible modelling language using specific graphical elements which are widely understood in the software development community [ 44 ]. UML is fit for the development of a theoretical concept that has its foundation in the functionalities of existing CAD software tools. The general nature of the language allows for applicability across the whole domain of CAD software compliant with the ISO 2553:2019 standard. The flexible nature of UML allows the presented model to be adapted further, as the CAD tools will evolve in the future, or even modified to be compatible with custom-made or open-source CAD software which are not compliant with ISO 2553:2019 but have available tools for weldment annotations.

UML diagram describing the PPR model for automatic programming of welding robots. The UML version 2.0 is used as described by Eriksson et al. [ 44 ] and Scott [ 47 ]

The major suppliers of CAD systems are extending the capabilities of the tools they supply, to encapsulate more product data [ 1 ]. The implementation of rule-based design methods enables the digitalisation of the product knowledge [ 45 ] and parametric CAD modelling which enables the reusability, the adaptability and the configurability of a CAD model [ 46 ], making the product engineering stage faster and more cost-effective. These capabilities are made use of to build parametric rules and product configurations in the proposed PPR model. The changes in the product domain directly influence the way the manufacturing process is set up and the way the available resources can sustain the manufacturing process [ 1 ].

The classification of input parameters for programming of welding robot given by Lauridsen [ 9 ] is used to formalise the information contained and processed by the proposed PPR model:

Design parameters : the product parameters which are related to the workpiece to be welded. These parameters include the geometry of the workpiece, the thickness of the plates joined through welding and the material specification. These parameters are varied by varying the design of the product, as requested by the customer.

Motion parameters : the process parameters which are specific to the manipulator of the welding torch (welding robot). The motion parameters include the position and the orientation of the torch relative to the workpiece or welding seam which influence the length of the exposed wire at the tip of the welding torch (stickout), the velocity of the torch and the weaving of the torch. Depending on the design parameters, the motion parameters may include certain offsets such as multi-pass offsets which allow an incremental increase of the distance between the torch and the work piece as the weld bead increases in thickness.

Arc parameters : the process parameters which are specific to the welding apparatus and are independent of the overall geometrical design of the product but are used to fulfil the quality requirements of the weldment, e.g. weldment throat size. These parameters are the arc current and voltage. Modern welding apparatus allows for automatic control of wire feed rate and shielding gas flow depending on the preset material specification and joining plate thicknesses. Other parameters which are important before the welding process can commence and which directly influences the arc parameters are the shielding gas and the filler wire material.

3.2 Proposed PPR model for automatic programming of welding robots

In the proposed PPR model, represented in Fig.  1 , the PPR perspective is made apparent by using different colours to represent the classes which constitute the product, process and resource-related elements. The classes represented with dark and light orange are related to the product component of the model. The green-coloured class diagrams are related to the process component of the model, and the purple-coloured class diagram is related to the resource part of the model. The main contribution brought to the ARP field is the process-related component of the model.

The product element of the proposed PPR model has its roots in the generic representation of a product in a CAD system. As observed in Fig.  1 , a CAD component class is composed of one or more Surface instances. A Surface class can be composed of one or more Edge instances. An Edge class is constituted of several tessellation points which can be accessed through the GetTessellationPoints() accessor. Depending on the shape of the Edge object, it can have two (straight edge) or more tessellation points (curved edge). The Surface class contains the normal vector associated with a surface, or several normal vectors, if the surface is curved. The coordinates of the tessellation points and the components of the normal vectors are expressed in relation to the origin frame of the CADComponent instance. In the context of this paper, the CADComponent is a metal plate that must be welded; the attributes the CADComponent class has are the shape parameters, the material (metallic alloy) of the component, the thickness of the plate and the transformation associated with the origin of the product when it composes a CADAssembly.

A CAD model can be driven by several ParametricRule objects instantiated by the product design engineers. These rules compose a ProductModelConfiguration object. This facilitates the accessible configuration of a product model by inserting the design parameters through the interface realised by the ConfigurationWizard . A ParametricRule contains a mathematical expression which drives various elements inside the model. For example, depending on the thickness of the plates, the throat size of the weldment can be automatically adjusted based on a mathematical rule set by the design engineer, based on specifications. Another example can be a ParametricRule object attached to a joining Surface object and a base Surface object that controls the length of the two surfaces in the same direction can directly drive the length of the intersection Edge and therefore the length of the Weldment object attached to that Edge . This simplifies the design process from having to modify two parameters to having to modify only one. The marker “ < < Drives > > ” suggests that the entity towards which the arrow is pointing it is directly controlled by the entity from which the arrow stems from. The marker “1…*” suggests that a ProductModelConfiguration object is composed of at least one ParametricRule and there is no upper limit on how many ParametricRule objects can exist.

The interface towards the CADAssembly object is realised by the ProductModel class. The ProductModel interface can be implemented by using the API (application programming interface) available with the CAD system. The interface clearly specifies the methods which must be made available through the API, for the CAD system to correctly adhere to the proposed PPR model [ 44 ]. Through the interface, accessors are made available for the design parameters of the weldment. The CADAssembly object is composed of at least two CADComponent objects. The “0…*” marker suggests that a CADComponent object can also exist independently of a CADAssembly , although this case is irrelevant in this research.

The process component of the product model is the main contribution of this work as part of the proposed PPR model for automatic programming of welding robots. The process model is constituted of a WeldmentExtractionTool class, which acts as a bridge between the CAD environment and the OLP environment, by using the available interfaces implemented for these systems. The WeldmentExtractionTool directly connects to the ProductModel interface and makes use of the available accessors to extract the design parameters of the weldments. The WeldmentExtractionTool uses a ProcessParameters database, which contains the digitalised tacit knowledge of the engineers under the form of a lookup table. For example, depending on the designed throat size of the weldment and the plate thickness on which the weldment is attached, a certain set of process parameters (motion parameters and arc parameters) are used for the welding process. Some of the motion parameters contained in the database which are dependent on the design parameters are the welding angle α and the travel angle β , which are represented in Fig.  2 , the torch travel velocity, and the stickout, which is the distance between the welding torch and the welding seam. The arc parameters can be the voltage and the current for the weldment. Some process parameters, like the wire feed and the gas flow, may either be present in this database or be controlled by the welding apparatus.

Graphical representation of a CADAssembly object compose of three CADComponent objects. The CADWeldment objects are attached to the Edge objects between the joining surface of the two components on top and the base surface of the base surface on the bottom. The welding angle α and the travel angle β are also represented in this figure

The WeldingProgram class is instantiated by the WeldmentExtractionTool class upon call of the ComputeTorchPositionParameters() method. The WeldingProgram is composed of one or more WeldmentOperation objects which share the same process setup parameters, e.g. the materials of the consumables, the welding process type (MIG, TIG, etc.) and the origin frame in which the geometry of the weldments is expressed. These parameters remain constant across the whole welding process. A WeldmentOperation object is discretised in one or more DiscreteWeldmentOperation objects, as described by Sarivan et al. [ 48 ]. Each discrete weldment operation which composes a WeldmentOperation object shares the same process parameters, e.g. the torch travel speed, the wire stickout and so on (the attributes of the WeldmentOperation class). Every DiscreteWeldmentOperation object is instantiated with its own torch position parameters.

After the WeldingProgram is instantiated, collision-free trajectories must be generated for the welding torch in order for the welding process to be successfully supported. This is made possible through the OfflineProgrammingSystem interface. The interface is made available by using the OLP’s API.

The resource component in the proposed PPR model is represented through the OfflineProgrammingSystem interface which offers access to the capabilities of the offline programming system which has available in it a digital twin of the robot-welding cell. These capabilities are the motion solver of the robot, which makes available the possible configurations in which a robotic manipulator can reach a point, and the collision detection functionality. By using these capabilities, it is ensured that the available resources on the manufacturing system can successfully support the welding process. The algorithms used to setup the welding process and the collision-free trajectories for the welding program are given in the next section.

3.3 Automatic programming based on product model

To facilitate the automatic programming of a welding robot based on a PPR model, the following prerequisites are necessary:

The creation of a CAD model with weldments annotations as stipulated by ISO 2553:2019 standard by the product design engineers (represented by the ProductModel class in Fig.  1 ).

The creation of a configuration through parametric rules which will be used to easily modify the CAD model without skilled intervention (represented by the ProductModelConfiguration class, in Fig.  1 ).

The digitalisation of the process parameters (motion and arc parameters) under the form of lookup tables depending on the design parameters of the product which will be organised in the ProcessParameters database.

With the prerequisites in place, the following steps must be followed to automatically program a welding robot using the proposed PPR model:

The design parameters are input to the ConfigurationWizard interface by the sales managers or the design engineers. A new instance of the CADAssembly class is created by using the ParametricRule objects which compose the configuration created by the product design engineers.

The composing elements of the CADAssembly instance are driven by the rules, including the weldment information, constituting as such the Design Parameters which are used to program the welding robot.

The geometry of the weldments is extracted from the CADAssembly instance through the ProductModel interface.

The geometry of the weldments is discretised and for each discrete state, motion and arc parameters are computed through the WeldmentExtractionTool class.

Collision-free trajectories for the robot are automatically computed using the functions available in the OLP and accessed through the OfflineProgrammingSystem interface.

The program is saved and ready to be validated through simulations and then deployed on the welding robot.

Algorithms are provided for steps 3–5 of the automatic welding-robot programming process. Each algorithm is associated with the methods available in the WeldmentExtractionTool class. A description of how the weldment geometry can be extracted from the product model is provided in Algorithm 1.

Implementation of "DetermineWeldmentsGeometry"

Once the ProductModel instance is created, it is possible to query the list of weldments using the GetWeldments() method. The method is accessed by first creating a handler towards the CAD system. For each weldment, the Edge to which the weldment is attached is extracted, together with the joining and base surfaces. The number of Edge instances attached to a weldment must be the same as the number of joining Surface instances. The tessellation points defining the curve underlaying the Edge instance are extracted. The tessellation points are important for the discretisation of the weldment into prismatic quasi-stationary elements [ 48 ] and are part of the weldment definition. If the underlaying curve is curved and not straight, for each tessellation point, the coordinates are saved and the normal vector components of each joining Surface and base Surface is saved. The normal vector components are saved in order to determine the position and the orientation of the torch relative to the weldment seam. In case the underlaying curve of the Edge instance is straight, there is only one set of normal vectors of each Surface ( joining and base ). It is to be mentioned that there can exist only one base Surface instance, while it is allowed to have multiple joining Surface instances for any single Weldment instance.

To compute the motion parameters for the welding process, namely the torch position and orientation for each discrete weldment point afferent to each extracted tessellation point, the ComputeTorchPositionParameters(ProcessParameters) method is called. The method makes use of the ProcessParameters database to properly set the orientation of the welding torch relative to the weldment bead. The steps used by the method are presented in Algorithm 2. Horn’s Absolute Orientation Method [ 48 , 49 ] is used in Algorithm 2 when the ComputeAbsoluteOrientation method is called to change the reference frame in which the weldments’ geometry is expressed.

Implementation of "ComputeTorchPositionParameters" interface

The weldments geometry extracted in step 3 is used as input for Algorithm 2. For each weldment definition, the coordinates of each tessellation point and the afferent normal vectors are extracted from each column of the matrix M. This happens as described in step 4 and each discrete state is associated with a tessellation point. The first three entries of the column represent the position of the tessellation point, the next three entries represent the normal vector’s components of the base surface, the next three components represent the normal vector’s components of the joining surface and the last entry is the dimension of the weldment’s throat in millimetres. The dimension of the weldment’s throat is used as lookup parameter to extract the associated sickout S (distance between welding torch and weldment seam), welding angle of the torch α (angle between torch and the base surface), and travel angle β (found between the torch and the weld seam) using the lookup table in the Process Parameters database. The α and β angles are also represented in Fig.  2 . By using the coordinates of each tessellation point and its afferent normal vectors, discrete weldment data is generated by using Horn’s Absolute Orientation method [ 48 , 49 ]. For each torch position determined using Horn’s method, the S , α and β parameters are set and the necessary transformation is computed from the origin frame of the CAD model to the position of the weldment torch afferent to the respective discrete weldment point. The result of Algorithm 2 can be observed in Fig.  3 .

Visualisation of the discretised weldment points which compose a welding operation and a welding program, resulted from Algorithm 2. On the left-hand side, an overview is provided. On the right-hand side, the torch positioned in one of the discrete weldment points can be observed. Images were generated using RoboDK

With the torch position parameters computed, Algorithm 3 is used to generate collision-free trajectories for the welding robot. This is associated with step 5 in the proposed automatic programming of welding robots’ procedure. Algorithm 3 takes as input all the torch position parameters computed in step 4. A handler towards the offline programming software is created in order to access the kinematics of the welding robot and test the reachability of the target point and to access the collision detection functionality of the OLP [ 22 , 23 , 24 , 25 , 26 ]. For each discrete point which composes each weldment, the end effector (the welding torch’s tip) of the virtual welding robot is commanded to reach the target point. While collisions are detected, the torch is twisted around the axis of the welding wire both ways until a configuration is reached which is free of collisions. Algorithm 3 is offered as alternative in case the collision detection and automatic collision solver algorithms are not available in the OLP of choice; however, many of the modern OLPs do have such methods available [ 27 , 28 ].

Implementation of CreateCollisionFreeTrajection"method

3.4 Reflections on the proposed system

This section has provided a thorough description of the proposed PPR model concept for automatic programming of welding robots which facilitates automatic programming of welding robots without the intervention of robotics engineers in defining the weldment paths for the welding robots. The concept was developed based on findings and formalisations from literature, as presented in Subsection 2.1 to achieve a PPR integration addressed to welding applications. The PPR model is composed of a collection of data contained in a CAD system compliant with the ISO 2553:2019 standard and process data that can be digitalised in the ProcessParameters database specific to the company where the system is implemented. The proposed PPR model summarised in Fig.  1 extends the scope of extant research, which is applied on a specific CAD system, as shown in Subsection 2.2 [ 41 , 42 ]. Algorithms were provided on how the data is processed to achieve a program for welding robots in an automatic manner. The proposed system is meant to drastically reduce manual input required to program weldment locations in the OLP.

4 Technical implementation

In order to prove the feasibility of the proposed PPR model, a technical implementation was put together using tools commercially available which are widely used across industry. The elements of the technical implementation are put together as showcased in Fig.  4 . For the creation of the product family design to be configured, the SolidWorks CAD software was selected due to availability for the researcher and its compliance with ISO 2553:2019 standard. The CAD assembly created in SolidWorks serves as part of the Product component of the PPR model and it can be accessed through the ProductModel interface, as presented in Fig.  1 . The interface is created as showed in Fig.  4 by using the SolidWorks API. The ProductModelConfiguration is created using the DriveWorks add-on for SolidWorks. DriveWorks was selected as it allows the creation of complex ParametricRule instances which can drive both the CAD models and the ISO 2553:2019 standard compliant annotations of the weldments. The ProductModelConfiguration is created by the design engineers. Upon instantiation of the Product Model through the graphical user interface implemented in DriveWorks, the CADWeldment information can be extracted using the Weldment Extraction Tool interface annotated with the robot arm in the diagram presented in Fig.  4 , and by using the ProcessParameters database populated by the welding and robotics engineers in a Microsoft Excel document which serves as a lookup table. The Weldment Extraction Tool uses the SolidWorks API in order to extract the necessary information to automatically program the welding robot. The result of the extraction is the.STEP file which can be imported in the OLP and files containing information about the weldments and process parameters. By using Algorithm 3, automatic programming of the welding robot is achieved in the RoboDK OLP software. A special tool called Oqton is used to facilitate the automatic programming of welding robots for the DTPS software supplied by Valk Welding (technology supplier) [ 28 ].

Diagram showcasing the elements used in the technical implementation of the PPR model. The red arrow indicates manually input information, while the black arrows indicate automatic flow of information along the value chain for programming of welding robots

There are two approaches towards the automatic programming of the welding robot:

Using the RoboDK software, which makes use of the Algorithm 3 described in Section 4 , by connecting to the RoboDK API and importing the CAD model in STEP file format and the Welding Program containing weldment information and process parameters.

By creating special CSR files which are programs that can be imported in the Desktop Programming Software developed by Panasonic and supplied by Valk Welding. The programming of the welding robot is fully automized using the Oqton add-on supplied by Valk Welding [ 28 ].

5 Experiments and results

Experiments were conducted with the technical implementations in the welding laboratory at Aalborg University and on the production line at the case company to test the hypothesis that welding robots can be programmed automatically based on the PPR model with minimum human input necessary. Therefore, two experimental pipelines and two experimental setups are used with the available hardware and software in the two environments. The technical implementation using the RoboDK OLP is used in the laboratory at Aalborg University and the technical implementation using the DTPS OLP is used on the production line at the case company.

5.1 Product design for experiments

An experimental product was designed in collaboration with the case company which will cover in its design the wide range of geometrical elements encountered in the company’s real products (observed in Fig.  5 and Fig.  3 ). The geometrical elements are circle arches of various dimensions with various orientations. The weldments which are tested are fillet weldments. Weldment annotations compliant with the ISO 2553:2019 standard are added using the “Weld Bead” tool available in the SolidWorks CAD software and are visible as black and grey bead-like elements overlaid on the CAD model [ 37 ]. The CAD model in Fig.  5 together with the overlaid annotations constitutes a CADAssembly instance, accessible through the ProductModel interface as explained in the architecture presented in Subsection 3.2 .

Test item designed in collaboration with the case company in SolidWorks. The weldments annotations compliant with the ISO 2553:2019 standard, are visible and marked with black and grey

By using the DriveWorks add-on available in SolidWorks, a ProductModelConfiguration was put together by the design engineers which will drive the Product component of the PPR model. The ProductModelConfiguration for the experimental case receives a total of 8 parameters driving the dimensions of various features of the item showed in Table  2 .

The design parameters serve as input for a Configuration Wizard implemented in DriveWorks and are displayed in Fig.  6 . Each design parameter drives one or more parametric rules attached to various elements of the Product . The process of creating a new product variant takes only a few minutes to complete. For testing the technical implementation presented in the previous section, three products are created using the Product Model Configurator and then fabricated by the case company for running the experiments both on their production line an in the controlled environment at Aalborg University. The design input for each of the test products is shown in Table  2 .

Product model configurator wizard implemented in DriveWorks, using eight design parameters for controlling the model

The case company was provided with the technical drawings, and the components of the item were fabricated, and tack welded together, being ready to be completely welded by the welding robots. The material used for all the items used in experiments is SSAB 335MC. In order to weld this material, based on the recommendations received from the case company, G3SI1 compliant filler wire with 1.2 mm diameter is used, together with a shielding gas mixture composed of 82% Argon and 18% Carbon Dioxide.

The arc and motion parameters which populate the ProcessParameters database were also agreed upon together with the case company given the plate thickness used in the items to be welded. The parameters which are used in the experiments are displayed in Table  3 . The parameters for fillet weldments between plates of 10 and 12 mm are given; however, the full database contains parameters for plates between 8 and 26 mm. The scope of the experiments presented in this paper does not span beyond fillet weldments of plates with 10 mm thickness.

5.2 Experiment in controlled environment at Aalborg University

For the experiments in the controlled environment, at Aalborg University, the technical implementation with the RoboDK OLP is used. A digital model of the robot cell was built using the RoboDK OLP. The digital model of the welding-robot cell can be observed in Fig.  7 together with the physical cell. The experimental setup is composed of a Universal Robots UR10 collaborative manipulator, a Migatronic Sigma Select 400 CW welding apparatus and a Siegmund Professional 750 welding table. The welding table has a measurement mesh engraved which allows for precise placement of the work piece on its surface.

The robot-welding cell available in the laboratory at Aalborg University on the left-hand side together with the digital twin built using RoboDK on the right-hand side

After the geometry and the design parameters of the weldment are extracted using Algorithm 1, the motion parameters for the welding torch can be computed by using the Weldment Extraction Tool interface in which Algorithm 2 is implemented. Each position of the weldment torch relative to the weldment seams can be visualised in the OLP as presented in Fig.  3 . The motion parameters are imported in the offline programming tool as displayed in Fig.  3 using the application programming interface of RoboDK. Together with the orientation of the torch, the torch velocity is imported too alongside the arc parameters which will be used to obtain the weldment.

With all the necessary data imported in the OLP, the robot program is automatically generated using Algorithm 3 and then ready to be transferred on the robot. A visualisation of the robot programs in RoboDK is provided in Fig.  8 for all three items created using the design parameters in Table  2 .

Visualisation of the welding programs which will be transferred on the robot for each item 1, 2 and 3 (from left to right)

Video of the experiment is available on YouTube at https://www.youtube.com/watch?v=fVslC2x6YVg . The items which were welded using the automatically generated welding program are displayed in Fig.  9 .

Resulted weldments following the automatic programming of the welding robot used in laboratory. From left to right: Item 1, Item 2 and Item 3, based on the design parameters displayed in Table  2

The results of the experiment show that automatic programming of welding robots using PPR models is feasible. After the insertion of the design parameters in the Configuration Wizard , no other input was necessary from the human operator and no further intervention was made on the CAD model, on the design parameters of the product and on the process parameters required for the welding process. The employed Algorithm 3 uses the capabilities of the offline programming system to ensure that the available resources (the Universal Robots UR10 robot) are able to successfully execute the weldment before the program is sent to the robot, thus proving the end-to-end integration of product and production in a seamless chain of operations.

The resulted weldments were inspected by FORCE Technology, and it was concluded that the weldments can be classified in the class D of the ISO 5817 standard. FORCE Technology is a Danish-approved technological service institute tasked to get technologies into use at Danish companies.

5.3 Experiment in the production line

In collaboration with Sjørring Maskinfabrik A/S, experiments were conducted on a production line in order to check the feasibility of the concept in a real-life scenario. Sjørring Maskinfabrik is an ETO SME located in Denmark, which is involved in the metal fabrication industry. The products produced by the company are in the category of metallic structures used in the construction, automotive and transportation industry.

For the experiments in the production line at Sjørring Maskinfabrik, the technical implementation using the Desktop Programming System (DTPS-Panasonic) provided by Valk Welding was used. The complete robotic setup includes a Panasonic TL 1800 industrial manipulator together with welding apparatus, prismatic mobile base and a two axis-positioner for the workpiece supplied by Valk Welding. The setup together with its digital twin can be observed in Fig.  10 .

Welding cell at the case company equipped with a Panasonic TL 1800 welding robot, with prismatic mobile base and 2 axis-positioner for the work piece, supplied by Valk Welding. The physical setup is displayed on the left-hand side together with its digital twin in DTPS on the right-hand side

After the geometry and the design parameters of the weldment are extracted, each position of the weldment torch relative to the weldment seams can be visualised in the OLP as presented in Fig.  11 . The parameters which dictate the motion of the torch along the welding seams is imported in DTPS by importing the file generated using the Weldment Extraction Tool containing the motion parameters for the welding torch and the design parameters of the product, i.e. the thickness of the plates. The DTPS software uses its own database of arc parameters which was built by the welding engineers at the case company. The motion between weldments and the collision-free trajectories are automatically generated using the Oqton add-on supplied by Valk Welding instead of Algorithm 3 [ 28 ]. The Oqton software also facilitates the collision-free programming of the external axis and the mobile robot base, ensuring control over welding puddle of molten metal.

The motion parameters of the weldment are imported and can be clearly observed in the image in the left-hand side of the figure. On the right-hand side, both the obtained robot program and the imported CAD model of the work piece can be observed

The results of the experiment show that automatic programming of welding robots using the PPR model is feasible using the hardware and the software available at the case company along their value chain. After the insertion of the design parameters in the Configuration Wizard , the only input required from operators is the import of the weldment information in the DTPS and Oqton software, which is rudimentary and can be managed with general computer skills. The Oqton software can automatically generate trajectories for both the Panasonic industrial manipulator and the external axis using the information extracted from the product model, thus shortening the programming time of the welding robot drastically.

The quality of the weldment was inspected by the welding engineers at the case company, and they were deemed satisfactory and compliant with the ISO 5817 class D standard. The resulted program is validated through simulations and then sent to the robot. The resulted weldments produced by the welding robot can be observed in Fig.  12 . Video of the experiment is available at https://www.youtube.com/watch?v=hX0ZQHrMYWI

Resulted weldment following the automatic programming of the welding robot used at the case company. From left to right: Item 1, Item 2 and Item 3, based on the design parameters displayed in Table  2

5.4 Quantification of results

The experiments above prove the feasibility of the proposed end-to-end digital integration for automatic programming of welding robots using PPR models. To prove the impact of the method, a further scenario was set up by using the traditional method of programming welding robots without having the CAD system and the OLP system integrated. To execute this scenario, Item 3 from Table  2 was selected to be used. First, a robotics engineer will create a robot program using manual input only, both to operate the CAD system to obtain the new product variant and to operate the OLP system to obtain the robot program. The results are compared to those obtained by using the automatic programming of the robot using the PPR model as observed in Table  4 .

The results in Table  4 indicate a drastic reduction of the human input necessary to program the welding robot of about 80% in terms of time spent by the robotics engineer.

6 Discussion and conclusion

In this paper, a PPR model for automatic programming of welding robots is proposed to enable the end-to-end digital integration of product and production. By making use of the PPR perspective over the product design and manufacturing preparation, all the necessary data for programming the welding robot, design parameters and process parameters, are processed automatically using the Product Configuration composed of Parametric Rule entities, which accepts values for design parameters through a configuration wizard. The necessary data for setting up the process parameters for the welding process are seamlessly extracted from the CAD model which is compliant with ISO 2553:2019 standard, allowing automatic programming of welding robots in one-of-a-kind production environments.

6.1 Academic implications

The knowledge base in the CAD-based programming field is extended by proposing a general framework for programming of welding robots based on data extracted from CAD systems compliant with the ISO 2553:2019 standard, adding to the work conducted by Tran et al. [ 41 , 42 ] and Mohammed et al. [ 40 ]. Unlike feature recognition algorithms, the proposed method makes use of the annotations which the design engineers must add to CAD models and therefore, further input from robotics engineers is not necessary to program the locations of the weldments and motion parameters of the welding robot. The PPR perspective is used to structure the data required to program the welding robot in a rigorous manner defined using the unified modelling language (UML). It is showed through the proposed PPR model, that the PPR perspective is highly relevant to applications beyond assembly operations, like welding. By using the PPR perspective, an engineering framework is obtained which bring together information from the product’s design generated through the customer’s demands, knowledge regarding the process necessary to produce the item and information about the resources required and resources’ capabilities to support the welding process.

6.2 Implications for practitioners

The work presented in this paper is motivated by the context of ETO SMEs, which operate in high-cost environments. Moreover, ETO SMEs are particularly challenged as they need to cope with long lead times for their orders, low utilisation of welding robots and highly skilled engineers required to reprogram the welding robots. The gap between theory and practice in the field of ETO supply chain planning, described by Bhalla et al. [ 50 ], is addressed in this work through practical problem-solving by proposing the presented PPR model meant to reduce the lead time and the cost of value adding operations along the ETO value chain. This work is proposed as direct response to Bhalla et al.’s call towards problem-solving research in the field.

While trying to cope with the shortage of welders, the case company of this research tries to rely more on automation. However, as presented and highlighted in the quantification of results, even though welding robots are used, they do require a considerable input from robotics engineers and design engineers to handle the high variation of products associated with an ETO, and to generate new robot programs. In other words, the problem of welders shortage is translated into a production cost–related problem. As suggested in Table  4 , by using the proposed PPR model to program robots, there are several advantages to be considered:

By reducing the necessary human input to generate a CAD model for a new product variant and to generate a new associated robot program, the cost associated with that human worker’s time is reduced and the human worker can be reallocated to other tasks.

Robot utilisation rate: The execution of the robot program while welding the work piece has a duration of about 5 min. In the scenario where the robot is currently welding a smaller work piece, the downtime until the new robot program is ready can be up to 25 min for Item 3, as suggested in Section 5.4 .

Lead time: With the programming time longer than the actual execution of the welding program, the duration of the programming process directly affects the lead time of the product manufacturing. By drastically decreasing the time necessary to program the welding robots, the lead time of the product is reduced too.

By implementing the PPR model, an end-to-end seamless integration of the sales operations, product engineering operations, and production preparation operations has been achieved. The obtained information flow is streamlined, and the time needed to design a new product variant and program a welding robot is drastically reduced.

6.3 Research limitations

The research that was conducted and presented in this paper was subject to several limitations which must be considered. The focus of the research was limited to robotic welding applications only, specifically MIG welding. Although the implications of this research may be valid for other applications too (e.g. glue dispensing, TIG welding), they were not included in the scope of this research, and they were limited by the equipment available to perform the research. The workpiece which was used to conduct the experiments has only fillet weldments. Other types of weldments (e.g. groove weldments) were not tested in the experiments.

Further limitations were imposed by the systems available for the researcher to design the experiments for the technical realisation of the PPR model concept. Access was limited to the SolidWorks CAD software together with its DriveWorks add-on, and to the RoboDK OLP software and the OLP software provided by ValkWelding. Access for experiments on the production line of case company was limited based on the work-shop activity and had to be synchronised based on robot downtime (e.g. when the utilisation rate is 0).

6.4 Further research and development

In order to reliably ensure the end-to-end integration of operations across the value chain in an ETO enterprise and to address the limitation this research was constrained by, the following further research and development paths are identified by the authors:

This paper presents a novel approach towards CAD-based programming by extracting all the relevant information to program a welding robot directly from the CAD system if it is compliant with the ISO 2553:2019 standard. It was observed during experiments that it is highly important to ensure that after the welding-robot program is generated, the physical tack-welded product is true to the CAD model. Therefore, the PPR model in combination with sensor-based programming is found to be highly interesting to investigate further in order to compensate for imperfections which might occur in the physical product.

The relevant work documented in the literature about the PPR perspective covers assembly operations mainly. This work shows that the PPR perspective can be adapted and used to create models for automatic programming of welding robots. Further research is encouraged to investigate the possibility of creating the same kind of models for other production operations, e.g. cutting and bending.

The generalisability of the proposed PPR model is to be investigated further in other environments than those presented in this paper and by using a wider palette of products.

The research in this paper was limited to MIG welding process only and fillet weldments; however. there are several other welding processes which involve robotic manipulators: gas tungsten arc welding, laser welding. Moreover, glue dispensing applications involve similar process parameters: end-effector travel speed, glue dispensing rate. It is found highly interesting to further investigate how the PPR model can be adapted to cover these applications too.

6.5 Conclusion

This paper has brought in the reader’s attention the current context of high-cost environments where ETO manufacturing enterprises face several challenges: shortage of welders, high production cost and low utilisation of welding robots due to high variety in their product offering. Furthermore, the gap found in the literature regarding CAD-based programming methods and PPR perspective applications in the manufacturing industry is found apparent. The proposed PPR model streamlines the information flow across multiple operations conducted in ETO enterprises allowing for faster lead time. The generalisability of the model is enabled by addressing CAD systems compliant with the ISO 2553:2019 standard. By using the PPR perspective, a comprehensive overview on all the data required to program a welding robot is obtained and the architecture of the proposed model is adapted accordingly. It is concluded that the proposed PPR model is technically feasible, and it holds significant potential in reducing production costs, drastically reduces the time needed to create new product variants, drastically reduces the time needed to program welding robots and increases utilisation rate of welding robots.

Ahmad M, Ferrer BR, Ahmad B, Vera D, Martinez Lastra JL, Harrison R (2018) Knowledge-based PPR modelling for assembly automation. CIRP J Manuf Sci Technol.  https://doi.org/10.1016/j.cirpj.2018.01.001

McKinsey and Company (2016) Danish manufacturing - winning in the next decade. https://www.mckinsey.com/featured-insights/europe/danish-manufacturing-winning-in-the-next-decade . Accessed Mar 2024

Sebastian Wittrock (2023) Researchers at SDU receives 13 million DKK grant to develop a robot that will be crucial to the green transition. https://www.sdu.dk/en/om_sdu/fakulteterne/teknik/nyt_fra_det_tekniske_fakultet/sdu-forskere-faar-13-millioner-til-ny-robot

Bejlegaard M, Sarivan I, Waehrens BV (2021) The influence of digital technologies on supply chain coordination strategies. J Glob Oper Strateg Sourc. https://doi.org/10.1108/JGOSS-11-2019-0063

Article   Google Scholar  

Tolio T, Sacco M, Terkaj W, Urgo M (2013) Virtual factory: an integrated framework for manufacturing systems design and analysis. Procedia CIRP. https://doi.org/10.1016/j.procir.2013.05.005

Zheng P, Xu X, Yu S, Liu C (2017) Personalized product configuration framework in an adaptable open architecture product platform. J Manuf Syst. https://doi.org/10.1016/j.jmsy.2017.03.010

Berry C, Wang H, Hu SJ (2013) Product architecting for personalization. J Manuf Syst. https://doi.org/10.1016/j.jmsy.2013.04.012

Tan C, Chung H, Barton K, Jack HuS, Freiheit T (2020) Incorporating customer personalization preferences in open product architecture design. J Manuf Syst. https://doi.org/10.1016/j.jmsy.2020.05.006

Lauridsen JK (1991) Computer aided off-line programming of multipass TIG-Welding. Dissertation or Thesis, Institute of Production, Aalborg University. https://vbn.aau.dk/en/publications/computer-aided-off-line-programming-of-multipass-tig-welding

Meixner K, Decker J, Marcher H, Luder A, Biffl S (2020) Towards a domain-specific language for product-process-resource constraints. ETFA. https://doi.org/10.1109/ETFA46521.2020.9212063

Meixner K, Rinker F, Marcher H, Decker J, Biffl S (2021) A domain-specific language for product-process-resource modeling. ETFA. https://doi.org/10.1109/ETFA45728.2021.9613674

Agyapong-Kodua K, Haraszkó C, Németh I (2014) Recipe-based integrated semantic product, process, resource (PPR) digital modelling methodology. Procedia CIRP. https://doi.org/10.1016/j.procir.2014.03.118

Brecher C, Kusmenko E, Lindt A, Rumpe B, Storms S, Wein S, von Wenckstern M, Wortmann A (2018) Multi-level modeling framework for machine as a service applications based on product process resource models. Proceedings of the 2nd International Symposium on computer science and intelligent control. https://doi.org/10.1145/3284557.3284714

Ferrer BR, Ahmad B, Lobov A, Vera DA, Martinez Lastra JL, Harrison R (2015) An approach for knowledge-driven product, process and resource mappings for assembly automation. CoASE. https://doi.org/10.1109/CoASE.2015.7294245

Winkler D, Novak P, Meixner K, Vyskocil J, Rinker F, Biffl S (2021) Product-process-resource asset networks as foundation for improving CPPS engineering. ETFA. https://doi.org/10.1109/ETFA45728.2021.9613253

Schleipen M, Drath R (2009) Three-view-concept for modeling process or manufacturing plants with AutomationML. ETFA. https://doi.org/10.1109/ETFA.2009.5347260

Pfrommer J, Schleipen M, Beyerer J (2013) PPRS: production skills and their relation to product, process, and resource. ETFA. https://doi.org/10.1109/ETFA.2013.6648114

Bergur Thormundsson (2023) Total number of industrial robots installed globally from 2019 to 2021, by application. https://www.statista.com/statistics/1383931/industrial-robot-installation-by-field-worlwide/ . Accessed Mar 2024

Zych A (2021) Programming of welding robots in shipbuilding. Procedia CIRP. https://doi.org/10.1016/j.procir.2021.03.107

Kristianto Y, Helo P, Jiao RJ (2015) A system level product configurator for engineer-to-order supply chains. Comput Ind. https://doi.org/10.1016/j.compind.2015.04.004

Wang B, Li Y, Freiheit T (2022) Towards intelligent welding systems from a HCPS perspective: a technology framework and implementation roadmap. J Manuf Syst. https://doi.org/10.1016/j.jmsy.2022.09.012

FANUC ROBOGUIDE - Intelligent 3D simulation. https://www.fanuc.eu/dk/en/robots/accessories/roboguide . Accessed Mar 2024

KUKA From offline programming to virtual commissioning. https://www.kuka.com/en-de/products/robot-systems/software/planning-project-engineering-service-safety/kuka_sim . Accessed Mar 2024

ABB Detecting Collision. https://developercenter.robotstudio.com/api/robotstudio/articles/How-To/Miscellaneous/CollisionDetectionExample.html . Accessed Mar 2024

ValkWelding (2011) Panasonic, the best tool for the arc welding robot industry. ValkMailing. https://valkwelding.com/media/site/3556cf32b1-1682495626/valk-mailing-2011-1-en.pdf . Accessed Mar 2024

RoboDK Collision Detection. https://robodk.com/doc/en/Collision-Avoidance.html . Accessed Mar 2024

Pan Z, Polden J, Larkin N, Van Duin S, Norrish J (2012) Recent progress on programming methods for industrial robots. Robot Comput Integr Manuf. https://doi.org/10.1016/j.rcim.2011.08.004

Valk Welding (2023) Oqton teams with Valk welding for automatic robotic arc welding programming. MetalForming. https://www.metalformingmagazine.com/article/?/pressroom-automation/robotics/oqton-teams-with-valk-welding-for-automatic-robotic-arc-welding-programming . Accessed Mar 2024

Ferreira LA, Figueira YL, Iglesias IF, Souto MÁ (2017) Offline CAD-based robot programming and welding parametrization of a flexible and adaptive robotic cell using enriched CAD/CAM system for shipbuilding. Procedia Manuf. https://doi.org/10.1016/j.promfg.2017.07.228

Kiani MA, Saeed HA (2019) Automatic spot welding feature recognition from STEP data. RAEE. https://doi.org/10.1109/RAEE.2019.8886989

Xuan LanPhung, Ngoc Linh Tao (2021) Automatic extraction and welding feature recognition from STEP data. Advances in engineering research and application. Springer International Publishing AG, Switzerland, pp 210–215

Chapter   Google Scholar  

Kuss A, Dietz T, Ksensow K, Verl A (2017) Manufacturing task description for robotic welding and automatic feature recognition on product CAD models. Procedia CIRP. https://doi.org/10.1016/j.procir.2017.01.045

Fang H, Ong S, Nee A (2017) Robot path planning optimization for welding complex joints. Int J Adv Manuf Technol. https://doi.org/10.1007/s00170-016-9684-z

Groover MP (2016) Automation, production systems and computer-integrated manufacturing, Fourth Global Edition, Pearson Education Limited

AutoDesk Weldments in Fusion. https://help.autodesk.com/view/fusion360/ENU/?guid=DWG-REF-WELDING . Accessed Mar 2024

PTC Welding Design in Creo. In: https://support.ptc.com/help/creo/creo_pma/r10.0/usascii/?_gl=1*3ljo5r*_ga*MTc5NzAzNzY2My4xNjg4MTM4NDIy*_ga_1QBT6P6HR1*MTY4ODEzODQyMS4xLjEuMTY4ODEzODUyMS4wLjAuMA..*_ga_CBN5QVB9VJ*MTY4ODEzODQyMS4xLjEuMTY4ODEzODUyMS4wLjAuMA..#page/welding/welding.html . Accessed Mar 2024

Dassault systemes cosmetic weldments solidworks. https://help.solidworks.com/2024/english/SolidWorks/sldworks/t_Creating_Weld_Beads.htm . Accessed Mar 2024

Siemens Weldments in SolidEdge. https://community.sw.siemens.com/s/article/frames-and-weldments-in-solid-edge . Accessed Mar 2024

Siemens Weldment Assistant—Siemens NX. https://docs.plm.automation.siemens.com/tdoc/nx/1899/nx_help/#uid:best_practices_bp_weldments_assistant . Accessed Mar 2024

Mohammed SK, Arbo MH, Tingelstad L (2021) Constraint identification from STEP AP242 files for automated robotic welding. ICMIMT. https://doi.org/10.1109/ICMIMT52186.2021.9476209

Tran TA, Lobov A, Kaasa TH, Bjelland M, Midling OT (2021) CAD integrated automatic recognition of weld paths. Int J Adv Manuf Technol. https://doi.org/10.1007/s00170-021-07186-0

Tran TA, Njåstad EB, Midling OT, Bjelland M, Lobov A (2023) Generation of rule-adhering robot programs for aluminium welding automatically from CAD. Int J Adv Manuf Technol. https://doi.org/10.1007/s00170-023-10996-z

Chen S, Zhang Y, Feng Z (2017) Automated programming for robotic welding. Transactions on intelligent welding manufacturing. Springer Singapore Pte. Limited, Singapore, pp 48–59

Google Scholar  

Eriksson H, Fado D, Lyons B, Penker M (2003) UML 2 Toolkit. Wiley, Indianapolis, Ind

Chavali SR, Sen C, Mocko GM, Summers JD (2008) Using rule based design in engineer to order industry: an SME case study. Comput Aided Des Appl. https://doi.org/10.3722/cadaps.2008.178-193

Camba JD, Contero M, Company P (2016) Parametric CAD modeling: an analysis of strategies for design reusability. Comput Aided Des. https://doi.org/10.1016/j.cad.2016.01.003

Scott K (2004) Fast Track UML 2.0. Apress, Berkeley, CA. https://doi.org/10.1007/978-1-4302-0720-7

I Sarivan JS Larsen O Madsen BV Wæhrens 2023 Elementary welding operations for automatic robot programming https://doi.org/10.1007/978-3-031-27933-1_9

BKP Horn HM Hilden ShahriarNegahdaripour 1988 Closed-form solution of absolute orientation using orthonormal matrices J Opt Soc Am A Opt Image Sci https://doi.org/10.1364/JOSAA.5.001127

Bhalla S, Alfnes E, Hvolby H (2023) Tools and practices for tactical delivery date setting in engineer-to-order environments: a systematic literature review. Int J Prod Res. https://doi.org/10.1080/00207543.2022.2057256

Download references

Acknowledgements

Special thanks to Sjørring Maskinfabrik A/S for allowing unrestricted access to their value chain, their knowledge base and their robot-welding cell for running experiments. We wish to acknowledge the contributions of Klaus Kalstrup, CEO; Morten Hvass, Project Manager; John Yde Hove, Project Manager; Andreas Kjær, Head of Supply Chain; Per Holst Menander, Development Engineer; Mads Fredsøe, Development Engineer, Jens Holm, Head of Production and finally, the work-shop workers who have conducted the fabrication of the experiment items.

The authors express gratitude towards Valk Welding who supplied DTPS licenses and offered attentive assistance in the development of the technical artefact for integration with DTPS. We wish to acknowledge the contributions of James de Villiers, Development Engineer and Adriaan Broere, CTO.

We express our gratitude towards FORCE Technology, who helped with the objective classification of the weldments obtained in the lab environment. We acknowledge the contributions of Michel Honoré, Senior Team Lead and Per Madsen, Robotics and 3D Print Operator.

We express our gratitude towards DriveWorks and RoboDK for supplying educational licenses which made possible the technical implementation which supported the tests conducted for validating the proposed theory.

Open access funding provided by Aalborg University The authors declare that this work was funded by Manufacturing Academy of Denmark (MADE). MADE is a Danish national association of universities, research and technology organisations, and private companies which work together towards bringing the Danish manufacturing industry at the highest global competitive level possible.

Author information

Authors and affiliations.

Department of Materials and Production, Aalborg University, Fibigerstræde 16, 9220, Aalborg Øst, Denmark

Ioan-Matei Sarivan, Ole Madsen & Brian Vejrum Wæhrens

You can also search for this author in PubMed   Google Scholar

Contributions

All authors have contributed to the research hereby presented and its documentation in this paper. Ioan-Matei Sarivan has conducted the conceptualisation of the PPR model and the technical implementation of the model in collaboration with external parties (Sjørring Maskinfabrik, FORCE Technology and Valk Welding), and wrote the submitted version of the manuscript. Professor Brian Vejrum Wæhrens has contributed in ensuring the research process’s balance between relevance and rigour by making and made sure the novelty of this work is properly documented. Professor Ole Madsen has contributed to framing the literature study in the field of CAD-based programming of welding robots and PPR models, while ensuring that the proposed PPR concept is properly grounded in the existing knowledge base. All authors have read and approved the manuscript that is hereby submitted for The International Journal of Advanced Manufacturing Technology.

Corresponding author

Correspondence to Ioan-Matei Sarivan .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Sarivan, IM., Madsen, O. & Wæhrens, B.V. Automatic welding-robot programming based on product-process-resource models. Int J Adv Manuf Technol (2024). https://doi.org/10.1007/s00170-024-13409-x

Download citation

Received : 11 September 2023

Accepted : 09 March 2024

Published : 25 March 2024

DOI : https://doi.org/10.1007/s00170-024-13409-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Welding robots
• Automatic robot programming
• Product model
• End-to-end integration
• Find a journal
• Publish with us
• Track your research