design research paper

• 2-Year: 1.806
• 5-Year: 2.269
• h5-index: 23
• h5-median: 37

Information

• For Readers
• For Authors
• For Librarians
• For Microsoft Word

Vol 17, No 3 (2023)

The International Journal of Design is a peer-reviewed, open-access journal devoted to publishing research papers in all fields of design, including industrial design, visual communication design, interface design, animation and game design, architectural design, urban design, and other design related fields. It aims to provide an international forum for the exchange of ideas and findings from researchers across different cultures and encourages research on the impact of cultural factors on design theory and practice. It also seeks to promote the transfer of knowledge between professionals in academia and industry by emphasizing research in which results are of interest or applicable to design practices.

  Call for Papers:

Full paper due: 31 may 2024.

Vol 17, No 2 (2023)

Vol 17, No 1 (2023)

Vol 16, No 3 (2022)

Vol 16, No 2 (2022)

Vol 16, No 1 (2022)

Vol 15, No 3 (2021)

Vol 15, No 2 (2021)

Vol 15, No 1 (2021)

Vol 14, No 3 (2020)

Vol 14, No 2 (2020)

Vol 14, No 1 (2020)

Vol 13, No 3 (2019)

Vol 13, No 2 (2019)

Vol 13, No 1 (2019)

Vol 12, No 3 (2018)

Vol 12, No 2 (2018)

Vol 12, No 1 (2018)

Vol 11, No 3 (2017)

Vol 11, No 2 (2017)

Vol 11, No 1 (2017)

Vol 10, No 3 (2016)

Vol 10, No 2 (2016)

Vol 10, No 1 (2016)

Vol 9, No 3 (2015)

Vol 9, No 2 (2015)

Vol 9, No 1 (2015)

Vol 8, No 3 (2014)

Vol 8, No 2 (2014)

Vol 8, No 1 (2014)

Vol 7, No 3 (2013)

Vol 7, No 2 (2013)

Vol 7, No 1 (2013)

Vol 6, No 3 (2012)

Vol 6, No 2 (2012)

Vol 6, No 1 (2012)

Vol 5, No 3 (2011)

Vol 5, No 2 (2011)

Vol 5, No 1 (2011)

Vol 4, No 3 (2010)

Vol 4, No 2 (2010)

Vol 4, No 1 (2010)

Vol 3, No 3 (2009)

Vol 3, No 2 (2009)

Vol 3, No 1 (2009)

Vol 2, No 3 (2008)

Vol 2, No 2 (2008)

Vol 2, No 1 (2008)

Vol 1, No 3 (2007)

Vol 1, No 2 (2007)

Vol 1, No 1 (2007)

Evaluating the effectiveness of functional decomposition in early-stage design: development and application of problem space exploration metrics

• Jinjuan She
• Elise Belanger
• Caroline Bartels

Mapping the landscape of product models in embodiment design

• Lukas Paehler
• Sven Matthiesen

Module partition for complex products based on stable overlapping community detection and overlapping component allocation

• Pengcheng Zhong
• Jianrong Tan

Optimize or satisfice in engineering design?

• Janet K. Allen
• Farrokh Mistree

AI-artifacts in engineering change management – a systematic literature review

• Peter Burggräf
• Johannes Wagner
• Ognjen Radisic-Aberger

Novel method for shape complexity evaluation: a threshold from machining to additive manufacturing in the early design phase

• Mouna Ben Slama
• Sami Chatti
• Borhen Louhichi

Don’t let perfect be the enemy of good: how perfectionism influences human-centred designing engagement and communal design production in civil engineering

• Nathalie Al Kakoun
• Frederic Boy
• Patricia Xavier

Towards the definition of assembly-oriented modular product architectures: a systematic review

• Fabio Marco Monetti
• Antonio Maffei

Analysing Industry 4.0 technology-solution dependencies: a support framework for successful Industry 4.0 adoption in the product generation process

• Matthias R. Guertler
• David Schneider
• Nathalie Sick

Understanding design education with a bibliometric approach: a story of 50 years

• İbrahim Delen
• Fatma Özüdoğru
• Ercan Akpinar

Method for potential assessment and adaptation for additive manufacturing of conventionally manufactured components

• Nadja Siller
• Sebastian Werner
• Dietmar Göhlich

Dormant deficiency: a novel concept to direct cause–effect CAD model analysis

• Harald E. Otto
• Ferruccio Mandorli

Creating predictive social impact models of engineered products using synthetic populations

• Phillip D. Stevenson
• Christopher A. Mattson
• John L. Salmon

Quantification of “novelty” based on free-energy principle and its application for “aesthetic liking” for industrial products

• Hiromasa Sasaki
• Hideyoshi Yanagisawa

Cross-disciplinary system value overview towards value-oriented design

• Emilia Lavi
• Yoram Reich

Empirical studies on conceptual design synthesis of multiple-state mechanical devices

• Anubhab Majumder
• Somasekhara Rao Todeti
• Amaresh Chakrabarti

Design for circularity and durability: an integrated approach from DFX guidelines

• Jaime A. Mesa

Product representation via networks methodology for exposing project risks

• Shlomi Efrati

Development of a diagnostic tool for product circularity: a redesign approach

• Arturo González-Quiroga

Towards an integrated design methodology for mechatronic systems

• J. A. Vazquez-Santacruz
• R. Portillo-Velez
• E. Portilla-Flores

Engineering complexity beyond the surface: discerning the viewpoints, the drivers, and the challenges

• Gisela A. Garza Morales
• Kostas Nizamis
• G. Maarten Bonnema

Multiple technology infusion assessment: a framework and case study

• Eun Suk Suh

Improving the elicitation of critical customer requirements through an understanding of their sensitivity

• Yijiang Chen

A new approach for evaluating experienced assembly complexity based on Multi Expert-Multi Criteria Decision Making method

• Elisa Verna
• Gianfranco Genta
• Maurizio Galetto

Lean development and its impacts on the performance of new product processes: an analysis of innovative Brazilian companies

• José Carlos de Toledo
• Larissa Maria Prisco Pinheiro
• Mario Orestes Aguirre González

Journal transformation, 2022 closure, and reviewers’ gratitude

An early-phase design process to enable long-term flexibility in assembly systems.

• Natalia Svensson Harari
• Anders Fundin

Research methods in engineering design: a synthesis of recent studies using a systematic literature review

• David Escudero-Mancebo
• Nieves Fernández-Villalobos
• Alejandra Martínez-Monés

Studying interaction density in co-design sessions involving spatial augmented reality

• Fatma Ben Guefrech
• Jean-François Boujut
• Gaetano Cascini

An augmented formulation for robust design optimization of structures using stochastic simulation method

• Mohd Aman Khalid
• Sahil Bansal
• Varun Ramamohan

A systematic approach for product modelling and function integration to support adaptive redesign of product variants

• Foo Shing Wong
• David C. Wynn

Beyond assembly features: systematic review of the core concepts and perspectives towards a unified approach to assembly information representation

• Nathaly Rea Minango

Managing multi-goal design problems using adaptive leveling-weighting-clustering algorithm

• Jelena Milisavljevic-Syed

Front-end issues in product family design: systematic literature review and meta-synthesis

• Leandro Gauss
• Daniel P. Lacerda
• Paulo A. Cauchick Miguel

Concepts of change propagation analysis in engineering design

• Arindam Brahma

Feeling the heat: investigating the influence of novice designers’ trait empathy, and their beliefs, attitudes, and intentions towards sustainability on their identification of problem requirements

• Rohan Prabhu
• Mohammad Alsager Alzayed
• Elizabeth M. Starkey

Comparing the effect of virtual and in-person instruction on students’ performance in a design for additive manufacturing learning activity

• Anastasia M. K. Schauer
• Kenton B. Fillingim
• Katherine Fu

‘Earning your scars’: an exploratory interview study of design for manufacturing at hardware startups

• Hannah D. Budinoff
• Julia Kramer

How do you play that makerspace game? An ethnographic exploration of the habitus of engineering makerspaces

• Melissa W. Alemán
• Megan E. Tomko
• Robert L. Nagel

Development and validity evidence investigation of a design for additive manufacturing self-efficacy scale

• Timothy W. Simpson
• Nicholas A. Meisel

“Why couldn’t we do this more often?”: exploring the feasibility of virtual and distributed work in product design engineering

• Sharon Ferguson
• Kimberly Lai
• Alison Olechowski

Quantifying the maximum possible improvement in \(2^{k}\) experiments

• Nandan Sudarsanam
• Anusha Kumar
• Daniel D. Frey

Organizing the fragmented landscape of multidisciplinary product development: a mapping of approaches, processes, methods and tools from the scientific literature

• Julia Guérineau
• Matthieu Bricogne
• Alexandre Durupt

A modeling framework to support the implementation of engineering changes in designing complex products

• Roland Lachmayer

How the type of methodology used, when working in a natural environment, affects the designer's creativity

• Vicente Chulvi
• Carlos García-García

Feedback systems in the design and development process

• Anja M. Maier

An underpinning theory and approach to applicability testing of constructive computational mechanisms

• Imre Horváth
• Zoltán Rusák

Codesign in resource-limited societies: theoretical perspectives, inputs, outputs and influencing factors

• Santosh Jagtap

Comparing parallel and iterative prototyping strategies during engineering design

• Alexander R. Murphy
• Erin A. Floresca
• Julie S. Linsey

Journal innovations, 2021 closure, and reviewers’ gratitude

• Find a journal
• Publish with us
• Track your research

• Reviews / Why join our community?
• For companies
• Frequently asked questions

Design Research

What is design research.

Design research is the practice of gaining insights by observing users and understanding industry and market shifts. For example, in service design it involves designers’ using ethnography—an area of anthropology—to access study participants, to gain the best insights and so be able to start to design popular services.

“We think we listen, but very rarely do we listen with real understanding, true empathy. Yet listening, of this very special kind, is one of the most potent forces for change that I know.” — Carl Rogers, Psychologist and founding father of the humanistic approach & psychotherapy research

Service design expert and Senior Director of User Research at Twitch Kendra Shimmell explains what goes into good design research in this video.

• Transcript loading…

Get Powerful Insights with Proper Design Research

When you do user research well, you can fuel your design process with rich insights into how your target users interact—or might interact—in contexts to do the things they must do to achieve their goals using whatever they need on the way. That’s why it’s essential to choose the right research methods and execute them properly. Then, you’ll be able to reach those participants who agree to be test users/customers, so they’ll be comfortable enough to give you accurate, truthful insights about their needs, desires, pain points and much more. As service design can involve highly intricate user journeys , things can be far more complex than in “regular” user experience (UX) design . That’s where design research comes in, with its two main ingredients:

Qualitative research – to understand core human behaviors, habits and tasks/goals

Industry and Market research – to understand shifts in technology and in business models and design-relevant signs

An ideal situation—where you have enough resources and input from experts—is to combine the above to obtain the clearest view of the target customers of your proposed—or improved—service and get the most accurate barometer reading of what your market wants and why. In any case, ethnography is essential. It’s your key to decoding this very human economy of habits, motivations, pain points, values and other hard-to-spot factors that influence what people think, feel, say and do on their user journeys. It’s your pathway to creating personas —fictitious distillations that prove you empathize with your target users as customers—and to gain the best insights means you carefully consider how to access these people on their level. When you do ethnographic field studies, you strive for accurate observations of your users/customers in the context of using a service .

© Interaction Design Foundation, CC BY-SA 4.0

How to Leverage Ethnography to Do Proper Design Research

Whatever your method or combination of methods (e.g., semi-structured interviews and video ethnography), the “golden rules” are:

Build rapport – Your “test users” will only open up in trusting, relaxed, informal, natural settings. Simple courtesies such as thanking them and not pressuring them to answer will go a long way. Remember, human users want a human touch, and as customers they will have the final say on a design’s success.

Hide/Forget your own bias – This is a skill that will show in how you ask questions, which can subtly tell users what you might want to hear. Instead of asking (e.g.) “The last time you used a pay app on your phone, what was your worst security concern?”, try “Can you tell me about the last time you used an app on your phone to pay for something?”. Questions that betray how you might view things can make people distort their answers.

Embrace the not-knowing mindset and a blank-slate approach – to help you find users’ deep motivations and why they’ve created workarounds. Trying to forget—temporarily—everything you’ve learned about one or more things can be challenging. However, it can pay big dividends if you can ignore the assumptions that naturally creep into our understanding of our world.

Accept ambiguity – Try to avoid imposing a rigid binary (black-and-white/“yes”-or-“no”) scientific framework over your users’ human world.

Don’t jump to conclusions – Try to stay objective. The patterns we tend to establish to help us make sense of our world more easily can work against you as an observer if you let them. It’s perfectly human to rely on these patterns so we can think on our feet. But your users/customers already will be doing this with what they encounter. If you add your own subjectivity, you’ll distort things.

Keep an open mind to absorb the users’ world as present it – hence why it’s vital to get some proper grounding in user research. It takes a skilled eye, ear and mouth to zero in on everything there is to observe, without losing sight of anything by catering to your own agendas, etc.

Gentle encouragement helps; Silence is golden – a big part of keeping a naturalistic setting means letting your users stay comfortable at their own pace (within reason). Your “Mm-mmhs” of encouragement and appropriate silent stretches can keep your research safe from users’ suddenly putting politeness ahead of honesty if they feel (or feel that you’re) uncomfortable.

Overall, remember that two people can see the same thing very differently, and it takes an open-minded, inquisitive, informal approach to find truly valuable insights to understand users’ real problems.

Learn More about Design Research

Take our Service Design course, featuring many helpful templates: Service Design: How to Design Integrated Service Experiences

This Smashing Magazine piece nicely explores the human dimensions of design research: How To Get To Know Your Users

Let Invision expand your understanding of design research’s value, here: 4 types of research methods all designers should know .

Literature on Design Research

Here’s the entire UX literature on Design Research by the Interaction Design Foundation, collated in one place:

Learn more about Design Research

Take a deep dive into Design Research with our course Service Design: How to Design Integrated Service Experiences .

Services are everywhere! When you get a new passport, order a pizza or make a reservation on AirBnB, you're engaging with services. How those services are designed is crucial to whether they provide a pleasant experience or an exasperating one. The experience of a service is essential to its success or failure no matter if your goal is to gain and retain customers for your app or to design an efficient waiting system for a doctor’s office.

In a service design process, you use an in-depth understanding of the business and its customers to ensure that all the touchpoints of your service are perfect and, just as importantly, that your organization can deliver a great service experience every time . It’s not just about designing the customer interactions; you also need to design the entire ecosystem surrounding those interactions.

In this course, you’ll learn how to go through a robust service design process and which methods to use at each step along the way. You’ll also learn how to create a service design culture in your organization and set up a service design team . We’ll provide you with lots of case studies to learn from as well as interviews with top designers in the field. For each practical method, you’ll get downloadable templates that guide you on how to use the methods in your own work.

This course contains a series of practical exercises that build on one another to create a complete service design project . The exercises are optional, but you’ll get invaluable hands-on experience with the methods you encounter in this course if you complete them, because they will teach you to take your first steps as a service designer. What’s equally important is that you can use your work as a case study for your portfolio to showcase your abilities to future employers! A portfolio is essential if you want to step into or move ahead in a career in service design.

Your primary instructor in the course is Frank Spillers . Frank is CXO of award-winning design agency Experience Dynamics and a service design expert who has consulted with companies all over the world. Much of the written learning material also comes from John Zimmerman and Jodi Forlizzi , both Professors in Human-Computer Interaction at Carnegie Mellon University and highly influential in establishing design research as we know it today.

You’ll earn a verifiable and industry-trusted Course Certificate once you complete the course. You can highlight it on your resume, CV, LinkedIn profile or on your website.

All open-source articles on Design Research

Adding quality to your design research with an ssqs checklist.

• 8 years ago

Open Access—Link to us!

We believe in Open Access and the  democratization of knowledge . Unfortunately, world-class educational materials such as this page are normally hidden behind paywalls or in expensive textbooks.

If you want this to change , cite this page , link to us, or join us to help us democratize design knowledge !

Privacy Settings

Our digital services use necessary tracking technologies, including third-party cookies, for security, functionality, and to uphold user rights. Optional cookies offer enhanced features, and analytics.

Experience the full potential of our site that remembers your preferences and supports secure sign-in.

Governs the storage of data necessary for maintaining website security, user authentication, and fraud prevention mechanisms.

Enhanced Functionality

Saves your settings and preferences, like your location, for a more personalized experience.

Referral Program

We use cookies to enable our referral program, giving you and your friends discounts.

Error Reporting

We share user ID with Bugsnag and NewRelic to help us track errors and fix issues.

Optimize your experience by allowing us to monitor site usage. You’ll enjoy a smoother, more personalized journey without compromising your privacy.

Analytics Storage

Collects anonymous data on how you navigate and interact, helping us make informed improvements.

Differentiates real visitors from automated bots, ensuring accurate usage data and improving your website experience.

Lets us tailor your digital ads to match your interests, making them more relevant and useful to you.

Advertising Storage

Stores information for better-targeted advertising, enhancing your online ad experience.

Personalization Storage

Permits storing data to personalize content and ads across Google services based on user behavior, enhancing overall user experience.

Advertising Personalization

Allows for content and ad personalization across Google services based on user behavior. This consent enhances user experiences.

Enables personalizing ads based on user data and interactions, allowing for more relevant advertising experiences across Google services.

Receive more relevant advertisements by sharing your interests and behavior with our trusted advertising partners.

Enables better ad targeting and measurement on Meta platforms, making ads you see more relevant.

Allows for improved ad effectiveness and measurement through Meta’s Conversions API, ensuring privacy-compliant data sharing.

LinkedIn Insights

Tracks conversions, retargeting, and web analytics for LinkedIn ad campaigns, enhancing ad relevance and performance.

LinkedIn CAPI

Enhances LinkedIn advertising through server-side event tracking, offering more accurate measurement and personalization.

Google Ads Tag

Tracks ad performance and user engagement, helping deliver ads that are most useful to you.

Share the knowledge!

Share this content on:

or copy link

Cite according to academic standards

Simply copy and paste the text below into your bibliographic reference list, onto your blog, or anywhere else. You can also just hyperlink to this page.

New to UX Design? We’re Giving You a Free ebook!

Download our free ebook The Basics of User Experience Design to learn about core concepts of UX design.

In 9 chapters, we’ll cover: conducting user interviews, design thinking, interaction design, mobile UX design, usability, UX research, and many more!

A Systems View Across Time and Space

• Open access
• Published: 13 April 2023

Design thinking as an effective method for problem-setting and needfinding for entrepreneurial teams addressing wicked problems

• Rahmin Bender-Salazar   ORCID: orcid.org/0000-0002-5783-6314 1  

Journal of Innovation and Entrepreneurship volume  12 , Article number:  24 ( 2023 ) Cite this article

11k Accesses

4 Citations

Metrics details

Organizations in a wide array of fields and disciplines are increasingly using design thinking as an innovative process to create products or services that address wicked problems in their industries. Design thinking, a method of creative and collaborative problem solving originating in the tactics of designers, is a product design and development process that is, more and more, being used as a tool to move innovation forward and structure creation processes in diverse disciplines, from product development to food creation to social science research. Increasingly design thinking has become popular beyond the confines of creative and design disciplines and into the realm of wicked problems in social and ecological systems. While design thinking has many forms and applications, this study uses a refined version built upon the key themes of inspiration, ideation, and implementation as defined by Tim Brown, CEO of IDEO (2009), and situates it within the social science discipline—namely, systems thinking, organizational learning, and action research. Through a distilled design structure this flexible methodology combines insights from organizational development, social psychology, systems theory, and design research. By embedding learning and reflective practices into the structure of design thinking, a hybrid model of design thinking emerges that is a more effective tool for framing, setting in context, and solving these types of problems within teams.

From large private companies to small NGOs, academic institutions, and government entities, all are striving to learn about and create innovative services, products, and experiences that address the problems the relevant stakeholders in their industries face. Design thinking, a methodology for problem solving that has its origins in designers’ approaches, tactics, and needs to make this multi-disciplinary process explicit (Gregory, 1966 ), has increasingly emerged in recent decades as a powerful method to drive the innovation process in the pursuit of improvement. Design thinking, as described by the emerging management and innovation scholar Michael Luchs, is “…a creative problem-solving approach—or, more completely, a systematic and collaborative approach for identifying and creatively solving problems” ( 2015 , p. 1). Design thinking’s holistic approach to stakeholders and systems, coupled with its participatory nature, has made it an approachable technique to use beyond the fields of art, architecture, engineering, and technology that traditionally have design disciplines. The theories and practice of design thinking have grown in popularity and have been more heavily used in the academic discourses on management and in the business industry over the past several decades. Thus, this discipline has emerged as a problem solving tool beyond the traditional confines of design (Johansson-Sköldberg et al., 2013 ).

This leads to the following research question: to what extent does the application of design thinking, tasked with addressing wicked problems, represent an effective means for team problem setting and problem solving in organizations?

To fully grasp the concepts discussed in this proposal, it is helpful to clarify a few definitions before proceeding. Wicked problems: these are difficult and challenging problems, which appear in all fields and organizations; the most complex, multifaceted, and intractable problems with systemic impact are referred to as wicked problems (Churchman, 1967 ; Rittel & Webber, 1973 ; Roberts, 2000 ). Organizations: This term is defined as “social units (or human groupings) deliberately constructed and reconstructed to seek specific goals” (Etzioni, 1964 , p. 3) and, in this study, they are defined as seeking to solve problems through the creation of a new product or service. Design thinking: The definition of design thinking in this study can be simply understood as the use of methods and research practices to solve problems that are traditionally not in the fields of design, architecture, or engineering.

A brief history of design thinking

Design thinking was evangelized and popularized by IDEO beginning in the early 1990s (Brown, 2009 ); however, it existed in the academic discourse much earlier in various forms. To understand the current and evolving use of design thinking, a historical review of this process is beneficial. Specifically, it is essential to examine the early work examining designers’ practice and research, occurring in the latter half of the twentieth century, by the parents of modern design thought: Lawson ( 1980 ), Rowe ( 1987 ), Archer ( 1979 ), and Cross ( 1991 ).

An initial push to make a more rigorous discipline out of design thinking sprang from what Michael Barry and Sarah Beckman—current researchers exploring learning in design thinking—refer to as “…a need to make design thinking explicit and a need to embrace the many disciplines that are engaged in some way with design” (Beckman & Barry, 2007 , p. 26). The movement towards an explicit design method began in the 1960s, which would later be referred to as the first generation, and the subsequent movement in the 1970s and 1980s, known as the second generation (Rittell, 1984 ). This second generation of design thought began to emphasize the social aspects of design, by including active participants in the process (Beckman & Barry, 2007 ).

As described by Archer, “there exists a designerly way of thinking and communication that is both different from scientific and scholarly methods of enquiry when applied to its own kinds of problems” (Archer, 1979 , p. 18). This assertion from Archer accents not only the thinking aspect but the unique way of communicating used by designers applying the design thinking method towards problem solving. Similar to this, Cross explains that the design thought process is a research practice and a way of processing information, described as “designerly ways of knowing” ( 2001 ), that is an independent methodology with rich theory and should not be dependent on social science theory ( 2007 ). These two scholars lay the groundwork for design thinking to emerge as a distinct discipline for tackling problems in a myriad of disciplines.

In addition, Rowe outlined a systematic design process to problem solving that emphasized the role of the designer to address the needs of the client ( 1987 ). He described this user-centered process as design thinking, which was one of the earliest uses of the term. In Rowe’s design thinking process, a designer intervenes in a client organization; interprets the evidence gathered through quantitative and qualitative investigation; and makes an effort to address the challenges presented in the form of a product or service. In Lawson’s work, the process of design thinking, though not explicitly called that, is explored as a process that utilizes experimentation and information gathering tactics to tailor products ( 1980 ). Lawson’s definition predates Rowe’s use of the term of design thinking but similarly focuses on the designer’s expert role in assessing the needs of a client and testing possible solutions. This process is a tool that designers can masterfully use, informed by their expertise and designerly ways of knowing (Cross, 2001 ), to ultimately solve challenges that often fall into the definition of wicked problems. Rowe and Lawson focus on the intrinsically unique features of design thinking, with an emphasis on how the use of data gathering and testing make it an ideal tool for finding appropriate and optimal solutions.

These foundations of design thinking led us to Tim Brown’s definition of three overlapping, sometimes non-sequential elements—inspiration, ideation, and implementation—as outlined in Change by Design ( 2009 ) and popularized by IDEO. This simple structure serves as the foundation in which to organize the foundational theories for the proposed method in this article. This definition of design thinking is informed by the work of Lawson ( 1980 ), Rowe ( 1987 ), Archer ( 1979 ), and Cross ( 1991 , 2001 ). This foundational design method is broadly defined as the three key elements can be repeated, can overlap, and can be non-sequential (Brown & Wyatt, 2010 ).

Design thinking adapted towards addressing wicked problems

For this exploration of design thinking’s effect and innovative potential in addressing wicked problems, it is essential to understand the corresponding academic discourse and how it has evolved with design thinking. The theory was first described in an editorial by management theorist Churchman ( 1967 ) as a reaction to the term, first coined by Horst Rittel. The article was an exploration of these difficult, virtually unsolvable problems in the management science discourse and responsibility of society and academia to accept their intractability and find innovation solutions to live with them (Churchman, 1967 ). This first formal definition of the concept was further expanded with more defined parameters with the article of Rittel and Melvin Webber in 1973 as uniquely complex problems. Rittel and Webber’s ( 1973 ) work framed wicked problems within the context of social policy planning, where problems are often not clear, and contrasted that with problems in mathematics and chess, where there are clear cut solutions. As stated by modern theorists Brian Head and Wei-Ning Xiang, “…the ubiquity of wicked problems is the norm, and present in almost every pressing issue area that matters to human society today…” ( 2016 , p. 1). This description describes the growing relevance and prevalence of wicked problems on human systems and how it has grown in importance from its inception.

Herbert Simon, a pioneer in design research and artificial intelligence, wanted to use a design approach, in the vein of the one described above, as a unique discipline, to tackle “ill-structured problems,” which he described as problems with undefined characteristics ( 1969 ). Simon described his approach to design as a means of “…devising artifacts to attain goals…” (Simon, 1969 , p. 114), which continued a trend of describing design as a solution making and transformative process. This interpretation of design thinking continued to gain momentum amongst theorists and practitioners throughout the twentieth century, which resulted in design thinking as a methodology becoming synonymous with problem solving, especially as a multidisciplinary practice for framing wicked problems (Buchanan, 1992 ). Design thinking as a method to solve problems outside the creative domain began with Herbert Simon, who applied design methodologies to science and his field of artificial intelligence ( 1969 ). This movement of applying the design thinking discipline to fields not traditionally associated with design continued with the product development process used by IDEO, know as Human Centered Design or HCD (Brown, 2008 ; IDEO, 2011 ). The degree of client participation and at which stages of the process vary between methods, but they agree on a key area of design thinking—that the client or product user is the primary focus.

As design thinking moves beyond the traditional creative sphere and enters the realm of addressing wicked problems across a wide spectrum of topics, the discipline is enriched by the rigorous research practices that the social sciences have to offer. The stand-alone discipline of design thinking explored in this article integrates some of the social science methodologies to effectively adapt to the new terrain of designing for social systems. Specifically, this discipline is informed by systems theory (Bertalanffy, 1969 ; Dentoni et al., 2023 ; Meadows, 2008 ; Senge, 1996 ), organizational learning (Argyris & Schön, 1978 ; Kolb, 1984 ; Senge, 1990 ) and action research (Lewin, 1946 ).

Design and systems

Systems are an essential element to implementing a design thinking process that addresses wicked problems, because they allow the designer to see a more expansive view of the problem. To understand how to design a specific product or service, the designer often analyzes the various systems that are involved, such as social, technological, ecological, or political systems. By understanding the inner workings of these systems and collaborating with relevant stakeholders, a designer can co-create a product or service that acts as a targeted intervention to improve the system. This perspective has its origins in general systems theory, formulated by biologist Ludwig Von Bertalanffy ( 1969 ), which expands the understanding of systems beyond science and analyzes all systems in an intricate, open, and holistic manner. The majority of design thinking approaches are human-centric perspectives on general systems theory in that they focus not only on the systems involved with a specific intervention but also on how the different systems interact with each other. Though most design thinking processes are human-centered, they are not exclusively focused on social systems, because the ecological and built environment are also considered. Expanding on this viewpoint is organisimic theory (Goldstein, 1995 ), which emphasizes human interconnectedness—that humans are intrinsically and inextricably intertwined with the natural environment and the ecological systems therein. In addition, Barry Commoner, in his work The Closing Circle , further stated that everything in living systems is connected to each other and what has an effect on one affects all (Commoner, 1971 ). These ideas inform systems thinking (Dentoni et al., 2023 ; Senge, 1996 ), which is an application of systems theory to interpret the intertwined and dynamic interactions among multiple interdependent elements to inform possible interventions. This approach to interconnected systems informs the design thinking approach through the very foundation of the process—placing the human at the center of the research and looking at all the ways this individual connects with the product, service, or system.

Design thinking to stimulate learning

The principles of design thinking are human-centered, that is, the results are specifically tailored to the end-user, and are created using a process of collaboration, active engagement, and reflection (IDEO, 2011 ). This process can be further explained using the double loop learning theory (Argyris & Schön, 1978 ), which informs how reflective practice foundationally builds on learning. Double loop learning involves single loop learning—repeated attempts to address the same issue with the same method—while additionally engaging in reflective practice to learn from past performance and emphasize repeat attempts to refine approaches (Argyris & Schön, 1978 ).

David Kolb, a scholar in learning science, similarly, outlines an experiential learning model ( 1984 ) rooted in social psychology, which focuses on concrete action, learning from experience, reflection, and experimentation. This theory involves an axis of learning with the y -axis containing two opposing methods of processing experience and an x -axis of opposing methods of transforming experience. This axis of learning can be seen in Fig.  1 , and display experience processing in learning from a spectrum of concrete examples as one extreme and abstract conceptualization of ideas as the opposition. The processing of information is similarly balanced that with two opposing methods of transforming experience (Beckman & Barry, 2007 ; Kolb, 1984 ). The two diametrically opposed information transformation processes include reflective observation on one end and active experimentation on the other (Beckman & Barry, 2007 ). In simple terms, the process as seen in Fig.  1 shows two forces of learning that of processing reality and transforming it within each there is a tangible and intangible component. The work of Kolb, Argrys, and Schön increase the potential to learn from the design thinking process with rapid prototyping practice—reacting and changing the product, system, or service based on reflective practices and adapting based on those reflections. Rapid prototyping is influenced by social learning models, which emphasize interaction in learning and the importance of experimentation with both thought and action.

Kolb Learning model as adapted from Beckman and Barry ( 2007 ), Kolb ( 1984 ) and Kolb and Kolb ( 2005 )

Charles Owen, a design academic from the Illinois Institute of Technology who has advocated for design as an engine for innovation ( 2006a ), builds on the prototyping practice from Kolb, Argrys, and Schön. Owen theorized that the design process has discernable phases that, while often not in order, generally begin with the analytic research stage and end with the synthetic experimentation and creation stage (Owen, 1993). This innovation model begins with creating ideas and concepts from research and then applying them to experiments for testing. When used through the lens of learning, this proposed process, as illustrated in Fig.  2 , begins to take shape as a non-sequential, innovative method to interpret and address complex problems. This process is illustrated in the work of Beckman and Barry ( 2007 ) who combined the elements of Owen ( 2006b ) in a simple vestige of two axes and four quadrants. In this prescribed and infinitely repeatable process, concrete analysis brings about observable research that can then be applied to abstract analysis, that is, frameworks and theories. Finally, this leads to abstract synthesis, which is the creation of ideas that can be clearly synthesized to become concrete solutions.

Innovation process as adapted from Beckman and Barry ( 2007 )

Using design thinking in concert with action research

Design thinking, as described by Owen, seeks to form knowledge through action (1997), which is similar in style and approach to Action Research (Lewin, 1946 ) in the social sciences. Action research was first created for researchers to take a participatory and active role in their studies to mold and guide their experience (Lewin, 1946 ), which echoes the role of the designer in a design thinking process. The designer or researcher needs to take account of their subjects and make observations, which is a traditional research paradigm while also understanding their impact as a participant in the process. In addition, reflective practice (Argyris & Schön, 1978 ) is a means to review and learn from past experience, and with this tool, a designer or researcher is able to build on observations of the research subject or client and create the best solutions for them. A similar approach to the use of knowledge aggregated from observations and reflective practice, is the needfinding model, which is an exploration of addressing the needs of a particular subject and working to create a solution tailored to solve this problem for them (Faste, 1987 ). Needfinding in design thinking does not occur as a sequential step after reflection and observation, but rather as a method to guide both of those processes to address the needs of the intended client or product user. Similarly, in action research, needfinding is necessary for the researcher to undertake to gain context of motivations of organizations and individuals involved. In action research, the subject and researchers are all participants and collaborators in the change process and its essential to understand their needs in this context, which parallels the collaborative and solution creating work of a designer.

Schön described design, in its traditional form, as a tacit process with designers’ knowledge that is difficult to transfer or explain ( 1983 ). This situates designers as having specific expertise that is difficult for those without the professional know-how to comprehend or utilize. Design thinking seeks to clarify the discipline of design into a process more akin to implicit knowledge (Nonaka & Takechi, 1995 ), allowing design expertise to be disseminated to a larger audience, including both the designer and the client or product user. This implies that the interaction between the designer and the client is a reciprocal transaction or a communication between interacting components and systems (Germain, 1991 ; Luhmann, 1995 ). This interactive method represents the action research process, where both parties contribute to the creation process, with the designer leading the exercise. The change desired in the design thinking process, rather than research study, is an output in the form of a product or service made in collaboration with the client.

This approach to learning is common within design in that it is meant to create the ideal solution through experimentation, iteration, and continually learning from both. Using participatory action research, that is focusing on rapid learning, repetition of the practice-driven design thinking framework, and reflection, is essential for innovating and solving wicked problems (Argyris & Schön, 1991 ; Lewin, 1946 ).

Innovating through design thinking

Innovation, described as the “core renewal process” in an organization purposed with creating new products and services (Bessant et al., 2005 ), is the mechanism for addressing wicked problems. To innovate effectively to remain competitive, organizations have increasingly turned to the application of design thinking as a process for product development in recent decades (Johansson-Sköldberg et al., 2013 ; Lockwood, 2010 ). Design thinking-driven problem solving is a powerful and disruptive method that creates innovative products and services that seek to address these types of problems across diverse fields.

This article uses a foundational approach to design thinking-driven problem solving, which is, in essence, a flexible framework that does not adhere to a strict structure. Rather, it is able to ebb and flow within the design challenge and cater to the relevant stakeholders. As stated by Sydney Gregory in the seminal work The Design Method , “[the] design method is a pattern of behavior employed in inventing things…which do not yet exist. Science is analytic; design is constructive” ( 1966 , p. 6). Design, in this context, is used as an engine of product, system, and service creation that addresses individuals’ needs and challenges.

The design thinking process explained above can be considered an innovation process (Brown & Wyatt, 2010 ) and has a social learning component (Beckman & Barry, 2007 ). More specifically, this process can be defined as a problem setting method (Schön, 1983 ). Problem setting, as explained by design cognition scholar Willemien Visser is “…the process by which we define the decision to be made, the ends to be achieved, and the means that may be chose[n]” ( 2010 , p. 4). Problem setting is the first step towards innovation and tackling a wicked problem. By defining the problem and understanding all of the pieces that interact with it, one can begin to address, but not necessarily solve a wicked problem. To understand how to use design thinking as a method within this innovative problem setting process, one must understand the context of the current design thinking discourse.

Towards a refined design thinking model

Organizations are consistently looking for innovative ways to advance their products, profits, and goals, and design thinking, though not clearly defined, has emerged as a driving force to meet these challenges. Despite the varying definitions (Brown, 2008 ; Dorst, 2006 , 2010 ; Kimbell, 2015 ), there are enough similarities that describe the key elements of design thinking that bring it in line with other design and social science research methodologies. By combining a few of the fundamental elements into a hybrid model of design thinking, it can be used as a powerful tool to address wicked problems that organizations face. This method, as illustrated in Fig.  3 , brings together the elements of Charles Owen’s map of innovation ( 1998 , 2006a , 2006b ), Kolb’s experiential learning ( 1984 ), and Tim Brown’s three signature elements of the design thinking process ( 2009 ).

Hybrid model of design thinking, which is a design process workaround with design thinking and innovation adapted from the work of Beckman and Barry ( 2007 ), Brown ( 2008 , 2009 ), Brown and Wyatt ( 2010 ), Brown and Katz ( 2011 )

The components of inspiration, ideation, and implementation (Brown, 2009 ) serve as the foundation of this hybrid model. Using Brown’s simplified construction could be interpreted as embracing the recent, popular versions of design thinking as a third or independent discipline. However, its approachable three-pronged structure provides a categorical separation between steps and meshes well with Owen’s concepts of innovation—the interplay of analysis and synthesis with abstract and concrete ( 1998 , 2006a , 2006b ). This powerful combination creates a streamlined and flexible framework, where innovation can occur in a non-sequential order, dictated by the needs of the problem. Interestingly, Archer foresaw this hybrid approach when he stated, “time is rapidly approaching when design decision making and management decision making techniques will have so much in common that the one will become no more than the extension of the other” ( 1967 , p. 51). Archer’s foresight in the above hybrid design approach is in line with his third-way ( 1979 ) thought process but differs in that this design discipline works in concert with social science instead of wholly separate from it. Using this innovative hybrid design thinking model, wicked problems can be quickly identified and addressed, with an outlook towards finding specific solutions to fit users’ needs.

Research design

Building on the theoretical model, based on the literature review above, a case study was undertaken to better understand the model in practice. The case study used a participatory design thinking exercise with a cohort of students enrolled in an applied entrepreneurial Masters-level course at Wageningen University. This course was targeted at students interested in entrepreneurship and circular economy, and worked with eight student teams that were developing business ideas using renewable materials in garment production. Disruptive innovation—a product, service, or approach that fundamentally upends the status quo of an industry or field (Christensen, 1997 )—serves as a lens in this case study to analyze the effect of design thinking on problem solving and concept development of the student teams’ entrepreneurial ventures The course was focused on circular economic systems, which seeks to reuse resources in a closed, infinitely repeatable loop, which is in contrast to traditional linear economic models that use finite resources and create waste (Geissdoerfer et al., 2017 ). The Ellen MacArthur Foundation, a leader in applying the circular transition, define the concept as the following:

A circular economy is an industrial system that is restorative or regenerative by intention and design. It replaces the “end-of-life” concept with restoration, shifts towards the use of renewable energy, eliminates the use of toxic chemicals, which impair reuse, and aims for the elimination of waste through the superior design of materials, products, systems, and, within this, business models. (Ellen MacArthur Foundation, 2012, p. 7)

Circular economy seeks to reduce humanity’s impact on the environment and climate by decreasing waste and using resources more efficiently, thus attempting to solve the wicked problem of negative human impact on the environment.

Creating a baseline

Participants in the study came from two types of academic backgrounds: a science-based one, and one rooted in the social sciences. There was an observable difference between each group in their ability to learn and apply design thinking. Students from a science-based background, such as environmental science or biochemistry, were able to learn and use design thinking concepts with greater ease than those with a social science, humanities, or management studies background. This noticeable difference may be attributable to the science-based students’ ability to mix and match frameworks as needed to find solutions to complex problems. For example, in physics, students have been taught to use one formula for one situation with its own set of variables, and another formula for another situation with a second set of variables. In other words, the situation dictates what tools are used. Similarly, in the hybrid model of design thinking, which the students were exposed to, specific elements are only applied in certain circumstances and situations. Thus, as design thinking contains elements of the scientific method, this may have resonated more with the science-based students’ usual ways of learning and applying methods.

The overall purpose of creating a baseline was to see what portion of the design thinking concepts had permeated in participants’ minds and how they described those concepts. As such, I used what participants shared as their interpretation or impression of design thinking in their own words. In many cases their descriptions were of a concept without the use of the concept name (e.g., prototype, ideation), and I compared these explanations with the concepts used in the hybrid model of design thinking in an effort to make connections where possible. The students displayed their knowledge of design thinking during the interviews and through the course by describing important elements of the process, namely, creating prototypes, building on failed attempts, and repeated reflection on the implementation of their ideas. To establish a baseline, it was not necessary for participants to use the exact names or descriptions of the design thinking concepts, as the real test of whether they understood these concepts and could apply them would be uncovered during the design thinking in action (DTiA) section of data collection.

This qualitative methods study, informed by design thinking, was conducted in three phases: Phase 1 consisted of an ethnographic observational study and Phase 2 consisted of a series of six interviews (see Table 1 ) with past participants to assess their knowledge of and ability to apply design thinking to a real world problem.

The purpose of these two phases was to collectively gather data to understand the relationship between design thinking and problem solving in a team. Specifically, the data from the two phases seeks to answer to what extent design thinking represents an effective method for team problem setting and problem solving of wicked problems in organizations. Once collected, the data was codified (see Table 2 ) into four major themes: (1) the interviewee’s personal motivation in life and vocational goals; (2) their professed knowledge in the aspects, uses, and approaches of design thinking; (3) the interviewee’s application of design thinking in a scenario; and (4) their assessment of the effectiveness of design thinking.

The research findings examine the research question, “To what extent does the application of design thinking, tasked with addressing wicked problems, represent an effective means for team problem setting and problem solving in organizations?" To answer this question, I used the four themes outlined above to conduct the data analysis, and the interpretation of the data will continue to follow these themes. For the interpretation, I split the four overarching themes into two categories. The first category incorporates the first two themes (personal motivation and knowledge of design thinking) and acts as a baseline to gauge, where the individual is academically and what design thinking concepts they have retained. This is useful information, because it paints a clearer picture of the participants’ individual characteristics, which I then paired with the second category of themes to understand whether these characteristics play a role in the participants’ application of design thinking to solve a wicked problem. The richest set of data comes from the second category. The latter two themes (application of design thinking and perceived effectiveness) are included in this second category as a way to analyze DTiA through role-playing scenarios, which gives insight into the participants’ practical knowledge and application of the hybrid design thinking model used for this experiment.

This DTiA exercise revealed three key features of the hybrid model, which combines behavioral science and traditional design methods to create a flexible and foundational model for addressing wicked problems. Three key aspects within the hybrid model that were particularly apparent in this second category were “problem setting”, “needfinding”, and “double-loop learning”. First, interviewees successfully applied problem setting by outlining all the necessary information that would be required to solve an assignment—in this case, the hypothetical scenario of working with Apple to improve the iPhone’s falling market share. Interviewees correctly prioritized the following: (1) setting up a component team to tackle the issue; (2) collecting data on competitors to compare best practices; (3) understanding the needs of potential and past customers; and (4) creating a process to experiment and iterate on failures. These priorities exemplify the hybrid model’s three central elements and how organizational learning, needfinding, and problem setting are key to the success of the model in addressing wicked problems. What’s more, the interviewees were able to link ecological systems, such as environmental value chains and social systems while looking at both consumers and stakeholders to put the question into context. Second, participants used needfinding to distinguish what aspects of the real world problem were most important to take into consideration when evaluating possible solutions. These aspects focused mostly on the needs of human and ecological systems that were involved with the problem. Third, participants used double-loop learning to test possible solutions to the problems they faced and made iterative changes based on the positive or negative results. Specifically, the interviewees showed how they questioned all of the parameters of the prompt and laid a plan for testing, retesting, and iteration of ideas.

This study’s findings suggest that the hybrid model of design thinking is an effective framework for addressing wicked problems. Namely, participants were able to recall various terms, such as “prototyping” and “ideation” when defining this hybrid model. Furthermore, they displayed implicit knowledge by successfully using aspects of the model, including “double-loop learning,” “iteration,” and “reflective practices,” to find solutions during the DTiA exercise. For example, Interviewee C specifically defined “prototyping” as “a method to create quick test solutions that can then be iterated upon and improved with future versions towards a suitable solution.” Being an explicit definition of this design thinking concept, it is clear that Interviewee C understood and retained the information learned during the course. By contrast, Interviewee A did not identify “prototyping” by name but displayed use of the concept during the role-playing exercise.

The course participants used design thinking in the formulation of their entrepreneurial ventures, which were created to address the wicked problem of environmental sustainability. Two groups of participants in particular, Epsilon and Zeta, used design thinking to address very specific problems they identified within environmental sustainability, which are outlined below.

Epsilon team’s use of the hybrid design thinking method

Epsilon’s innovative solution was developed in response to the lack of incubation spaces for sustainable entrepreneurs in Wageningen, Netherlands—that is, workspaces and offices, where like-minded entrepreneurs can work and have access to investors and experts to grow their businesses. The team focused on Wageningen specifically, because they had the most experience in this city, as students at the local university and as entrepreneurs who had attempted a previous venture here already. Note that this was the team’s second venture attempt for this study. They first explored how to grow a mushroom skin, related to the “living skin” research project, so that they could experiment with different types of coating to make the material waterproof. They planned to sell the waterproof coating to companies to make durable clothing, bags, or car interiors. Through experimentation and the prototyping process, the team tried to grow mushrooms but faced challenges with a lack of expertise and a space to grow the fungi. The team expressed frustration about these obstacles and through reflection realized that getting expert assistance and finding a space to experiment were essential to their success as a venture; however, perhaps, these were problems they could address. As such, the team shifted their focus to a new venture, which was to find an innovative solution to the lack of incubation spaces in Wageningen.

The team researched and tested their new venture concept of creating an organic, sustainably, and locally sourced café that is an office space for ventures in the city, has a network of experts to help entrepreneurs, and offers a location for entrepreneurs to sell and test their products and services. With this shift, the team then went to collect data and surveyed people around the city and the results showed that there was, in fact, demand from residents and sustainable entrepreneurs for this type of space and that Wageningen did not currently have any locations that met these entrepreneurs’ needs. Specifically, they found that a co-working space and having access to experts are actually crucial for entrepreneurs in the early stages of their ventures, because it allows them to test their ideas and learn from others as they iterate on better solutions. Similarly, the team itself was able to learn from the failure and challenges of their first venture attempt, which inspired them to address that problem directly with a different venture. Epsilon’s venture evolved to become a café, store, and incubation space for entrepreneurs in Wageningen that sought to create products or services that are environmentally sustainable and have closed-loop, circular waste streams. Their final venture concept included a plan for further development, testing, and iteration to continue learning as they grow and improve their products.

This team’s journey from one venture to another provides an exemplary use of the hybrid design thinking model. This shift embodies Argyris and Schön’s definition of double-loop learning, the students not only explored their original question related to their venture but also if it was the right question in itself. Argyris and Schön ( 1978 ) described the concept with the following metaphor:

Single loop learning can be compared with a thermostat that learns when it is too hot or too cold and then turns the heat on or off. The thermostat is able to perform this task, because it can receive information (the temperature of the room) and, therefore, take corrective action. If the thermostat could question itself about whether it should be set at 68 degrees, it would be capable not only of detecting error but of questioning the underlying policies and goals as well as its own program. That is a second and more comprehensive inquiry; hence it might be called double loop learning. (pp. 2–3)

I shared the metaphor above with the students during the beginning of the course, and this group exemplified double-loop learning in the selection and refinement of their venture. Team Epsilon showed their understanding of the context of a venture and how that can change the very nature of a proposed solution as it was for them, when they shifted the problem they focused on. Furthermore, their reaction to changing circumstance can be interpreted as the team displaying Schön’s ( 1983 ) concept of “reflection-in-action” (p. 79). The team struggled with their concept and made changes that ebbed and flowed with the challenges they faced, which in Schön’s definition would be part of the designer’s reflective “conversation with the situation.” Their use of double-loop learning in regard to building on lessons learned and changing approaches based on feedback led them to their new venture and guided how they continued to iterate and improve that new venture. Furthermore, they expertly displayed problem setting and understanding the context of a venture and how that can change the very nature of a proposed solution as it was for them, when they shifted their problem. The final project from this team was well thought out, fit to context and was an exemplary use of the hybrid model.

Zeta team’s use of the hybrid design thinking method

The Zeta team faced very different challenges in creating their venture. The team members, who came from diverse backgrounds and had varying interests and skillsets, came up with a plethora of ideas and had a difficult time choosing one idea to move forward with. The ideation and brainstorming process was not decisive or iterative, and the students expressed their frustration as the process rolled on without a clear venture in sight. The team worried that they had fallen behind and would not have enough time to complete all aspects of the project. With design thinking coaching by the researcher, the team was encouraged to refocus their efforts to think about any problem, not necessarily related to environmental sustainability, and see how they could collectively address it. Once they had decided on a problem, they could then begin introducing aspects related to reducing waste streams and circular economy in an organic way that would connect the problem they chose to the bigger, wicked problem of environmental sustainability.

The team used needfinding to find the requirements of the problem and then utilized framing and reframing to make their venture work in that context. This venture’s process exemplifies frame innovation, coined by Dorst ( 2015 ), which he describes as a “key entrepreneurial activity” (p. 149). The team shifted frames, from seeing their venture as a means to solve an aspect of environmental sustainability, to solving a real-world problem that can be connected to environmental sustainability. The Zeta team went through further consultation and began discussing one team member’s proposed problem based on her experience working with the United Nations (UN) on disaster recovery in Latin America. She described the problem of people needing quick housing when a disaster strikes; the logistic challenges of getting temporary, single use housing into the disaster area; and the waste the homes leave once they are no longer used. This discussion led the group to connect this issue to the “living skin” fungi material to create temporary housing that could be lighter weight, biodegradable, and reusable. This idea connects the problem posed within the problem of environmental sustainability, which was their task. Furthermore, this shift exemplifies an understanding of systems thinking and interconnectedness of social and ecological systems. Once the initial concept was developed, they began to refine the idea using team members’ expertise working in international development and aid as well as environmental sustainability. They then turned to the questions of how to make this into a venture and who would be their target audience. This process led them to brainstorm how they could balance the needs of potential clients (disaster response organizations), potential users (disaster victims), and the natural environment (ecological footprint). The team conducted surveys and found that potential clients would be interested in cost and scale of the potential solution, while potential users would be most interested in comfort and durability. Those considerations were then balanced with creating the minimalist ecological footprint and having a viable business model so the venture would thrive. They made two crucial decisions at this juncture: first, they decided not to manufacture the material but to source it from a third party, and second, they decided to structure their venture as a non-profit focused on the UN and disaster recovery agencies.

Using the design thinking concepts of rapid prototyping and reflection they were able to quickly figure out which ideas were working and abandon those that were not, which ultimately led to a venture they described as “living houses.” This iterative process they embodied shows the power of using design thinking for concept refinement. The team’s final venture concept was a not-for-profit organization that sourced biodegradable and reusable materials to create light-weight, temporary housing to be sold to NGOs, governments, and public international institutions for disaster victims around the globe. Their plan included next steps for further testing and iteration to improve the product and business model. In both cases, the Epsilon and Zeta teams used the hybrid design thinking model to problem set and problem solve as they set up and executed their ventures. This clearly helps address the central research question of the study by showing the utility of design thinking as tool for addressing wicked problems both in the internal venture creation process and the problem the venture sought to address, environmental sustainability.

Connecting team’s use of design thinking hybrid method to interview data

While these team examples provide evidence to support the positive impact of design thinking on problem setting and solving for wicked problems, the most interesting results came from the Phase 3 interviews that took place 1 year after completion of the course. During these interviews the participants were tasked with using the hybrid design thinking model in a theoretical applied scenario. Through these participant interviews, I was able to explore which features of design thinking they had internalized and how they might apply those to a real world problem. As explained in the following discussion, the participants’ ability to use design thinking concepts implicitly and explicitly over a year later shows that the concepts were adopted as a modus operandi, at least in part. As shown in the matrix in Fig.  4 , the participants all showed a high ability to apply the competencies regardless of their ability to define them as. In addition, the participants who did not recall the definitions were able apply the competencies to a higher level of specificity and knowledge than two out of the three interviewees that could.

Matrix showing interviewees’ ability to define ( x -axis) and apply ( y -axis) on key design thinking competencie s

In the scenario with the interview, participants were tasked with describing the steps they would take to tackle the problem of declining market share of the iPhone. Without being specifically prompted, all interviewees included some form of waste reduction and environmental sustainability into their action plan in the scenario. Some causation for the inclusion of these environmental themes could be the students’ backgrounds, their association with the course’s focus on this particular wicked problem, and/or a general growing awareness of the global climate crisis. That said, their ability to connect a problem to a deeper, wicked problem demonstrates their use of the competencies of system thinking and problem setting from the hybrid design thinking model. They were able to place a practical task within a wider context and connect it with wicked problems involved, such as climate change and electronic waste.

Much like in the case of the Zeta team described above, any seemingly unrelated problem can be used as a gateway to begin discerning the mechanics needed to address a specific, wicked problem, which will lead to creating experimental solutions that can be further tested. Furthermore, the participants were able to identify, in name or description, the three core elements of the hybrid design thinking model—inspiration, ideation, and implementation—and delineate corresponding activities for each while also explicitly and implicitly describing design thinking’s approach to solving wicked problems. The participants’ perception of and demonstrated application of design thinking elements in their problem solving procedure in the interview sheds light on the effectiveness of design thinking as a problem setting and solving tool. This suggests that the participants embraced design thinking, specifically the three-pronged hybrid model that melds design methodologies and behavioral science, as a useful process for problem solving. More important than the interviewees identification of the steps of the model, was their application of problem setting and problem solving strategies that follow the three main elements of design thinking. Participants were able to show the use of brainstorming (inspiration), prototyping (ideation), and iteration (implementation) in various ways and interchangeably. This nimble and engrained use of the concept shows its effectiveness as a problem setting and problem solving tool as well as its impact on users.

Connecting findings to the existing literature

This study was informed by a literature review which examined the history, theories, and application of design thinking in addressing wicked problems. In this study, design thinking is considered a “third discipline” or independent area of study that applies behavioral science and design methodologies to a proposed hybrid model. This hybrid design thinking model strengthens typical design methodologies by including (1) systems thinking, taking into account interconnectedness of ecological and social systems; (2) organizational learning, using double-loop learning, reflective practice, and iterative prototyping; and (3) elements of action research, such as collaborative and cyclical feedback with designer and client. This integrated process is particularly pertinent when working on problems beyond traditional design, for it lends a structural framework to behavioral science research using the three phases of ideation, prototyping, and implementation. In the hybrid design thinking model, behavioral and organizational considerations are not merely optional, but rather an essential element that works in congress with design methodologies.

As outlined above, the findings of this study are in line with the literature and research that indicate that design thinking is a potent tool for addressing wicked problems. By their nature, wicked problems are intractable and complex, so when testing ways to solve them effectively the method must be able to adapt with that nature. Specifically, this research suggests that design thinking represents an innovative process uniquely equipped to address wicked problems through its use of “problem setting.” That is, the effective use of needfinding—looking for solutions for relevant stakeholders—and double-loop learning—applying iterative knowledge and testing assumptions while doing. Although the participants in this study represent a very small treatment group in a specific educational setting focused on tackling environmental wicked problems, there is potential to test this experiment more broadly in educational settings focused on a variety of wicked problems.

Implications for future research

There are four overarching implications that result from this study that academic researchers and practitioners should take into consideration when exploring how to use design thinking as an effective method to address wicked problems. First, future research should conduct experiments using design thinking to address wicked problems that occur within other thematic areas, such as gender inequality, wealth distribution, employment with new technologies, and religious tensions, among others. Second, future research should test a variety of team compositions and study settings beyond that of a university. For example, team members could be part of a research institution, corporation, government, or NGO, and studies could be conducted within those organizations or across disciplines. Third, future research should explore what other aspects of design thinking are effective and learn why they are or are not successful in tackling wicked problems. Fourth, future research should test the hybrid design thinking model’s effectiveness using other forms of design thinking as a control. Finally, beyond academia there are implications of this study for professional practice. Gleanings from this study and use of the hybrid model in the field can occur immediately if used as an adaptable and editable tool for problem solving. This can be used in NGO’s, governments, universities and companies working on wicked problems in their work.

Limitations

This was a qualitative methods study that included a participatory design exercise focused on students enrolled in an entrepreneurship and circular economy course, where they were tasked to use design thinking as a method for creating innovative solutions to the wicked problem of environmental sustainability. While designed to examine how effective design thinking is for setting and solving wicked problems for teams, there is a clear limitation of its application on settings outside education, such as in business and practices outside of academia. Although the course was hands-on, involved the creation of a nonprofit or for-profit business, and was team-based, it still took place in an educational setting rather than in the open marketplace. In addition, this study unfolded in a European context and specifically within the Netherlands, which limits its scope further. As stated earlier, there are wider implications for this data beyond being held in an academic setting that influence the results and potential uses of design thinking. As stated above, future studies should be conducted with teams outside of academia who are tackling different wicked problems other than environmental sustainability. Different results could occur in different settings and problems and future research can explore those possibilities.

Beyond the components of the research, this study had limitations with time, as it had to be carried out during a specific semester and was dependent on student availability. In addition, due to university considerations, including the time needed for proposal review and IRB approvals, there were delays in conducting the interviews which were originally set for May 2018, but were carried out in December 2018 and January 2019. However, this allowed for a shift in focus of looking at how the knowledge and practice of design thinking remained implicitly and explicitly in the interviewees’ problem solving practices. A final limitation is that this study was a doctoral dissertation, which means it had a limited budget and a specific time period in which it was required to be completed.

Final thoughts

Analysis of designers’ thinking and doing has been explored for over a half century, and design thinking, in particular, has evolved over the last three decades from a process only used by designers to more expansive use. Along with the expanded use of design thinking is the rightful criticism, skepticism, and curiosity with the approach, which can offer an opportunity for further refinement and transdisciplinary use. This evolution has expanded design thinking from traditionally creative fields to help create products to practical, ergonomic and aesthetic standards to being used by governments, social policy researchers, non-governmental organizations, and many more to solve societal problems and the most difficult among them, wicked problems. The hybrid design thinking model strengthens design methodologies with systems thinking, organizational learning, and action research, which can help deepen and inform the design methods when working on problems beyond traditional design. IDEO’s popularized design thinking process with the three elements of inspiration, ideation, and implementation provides a structure that can be used as a basis to add insights and tactics from social sciences—namely, systems thinking, organizational learning, and action research—and designer’s methods more broadly. Systems thinking offers an opportunity for teams to zoom out and have a macro view of the dynamic, interconnected elements of the wicked problem they seek to address through iterative solutions and reflection. Organizational learning offers a posture of learning which can strengthen the iteration, testing, and reflection processes in design thinking. Finally, action research informed practice with design thinking enables teams to be active participants, researchers, and designers in finding possible solutions to wicked problems. Design thinking when applied to solving problems in an entrepreneurial education setting will add to the effectiveness and innovative nature of the solutions created. Through creative brainstorming, experimentation and reflection being integrated into the creation of entrepreneurial solutions to wicked problems there is great potential ramifications beyond educational settings, such as industry, government, and civil society.

Availability of data and materials

The data and materials used in the research are available through the ProQuest dissertation database as part of graduation requirements for the PhD at Fielding Graduate University.

Abbreviations

Design thinking in action

Institutional Review Board

Archer, L. B. (1967). Design management. Decision, 1 (4), 47–51.

Google Scholar  

Archer, L. B. (1979). Whatever became of design methodology? Design Studies, 1 , 17–20.

Article   Google Scholar  

Argyris, C., & Schön, D. (1978). Organizational learning: a theory of action perspective . Addison-Wesley.

Argyris, C., & Schön, D. A. (1991). Participatory action research and action science compared: a commentary. In W. F. Whyte (Ed.), SAGE focus editions: Participatory action research (pp. 85–96). SAGE Publications Ltd. https://doi.org/10.4135/9781412985383

Chapter   Google Scholar  

Beckman, S., & Barry, M. (2007). Innovation as a learning process: Embedding design thinking. California Management Review, 50 (1), 25–56.

Bertalanffy, L. V. (1969). General systems theory: Foundations, development, applications (Rev. Ed) . George Braziller.

Bessant, J., Lamming, R., Noke, H., & Phillips, W. (2005). Managing innovation beyond the steady state. Technovation, 25 (12), 1366–1376.

Brown, T. (2008). Design thinking. Harvard Business Review., 86 (6), 84–92.

Brown, T. (2009). Change by design: How design thinking transforms organizations and inspires innovation . Harper Business.

Brown, T., & Katz, B. (2011). Change by design. Journal of Product Innovation Management, 28 , 381–383. https://doi.org/10.1111/j.1540-5885.2011.00806.x

Brown, T., & Wyatt, J. (2010). Design thinking for social innovation. Stanford Social Innovation Review., 12 , 31–35.

Buchanan, R. (1992). Wicked problems in design thinking. Design Issues, 8 (2), 5–21.

Christensen, CM. (1997).  The innovator's dilemma: when new technologies cause great firms to fail . Harvard Business School Press, ISBN 978-0-87584-585-2

Churchman, C. (1967). Guest editorial: Wicked problems. Management Science, 14 (4), B141–B142.

Commoner, B. (1971). The closing circle: Nature, man, and technology . Random House Knopf.

Cross, N. (1991). Research in design thinking . Delft University of Technology.

Cross, N. (2001). Designerly ways of knowing: Design discipline versus design science. Design Issues, 17 (3), 49–55.

Cross, N. (2007). Designerly ways of knowing (p. 16). Basel: Birkhäuser.

Dentoni, D., Cucchi, C., Roglic, M., Lubberink, R., Bender-Salazar, R., & Manyise, T. (2023). Systems thinking, mapping and change in food and agriculture. Bio-Based and Applied Economics. https://doi.org/10.36253/bae-13930

Dorst, K. (2006). Design problems and design paradoxes. Design Issues., 22 , 4–17.

Dorst, K. (2010). The nature of design thinking. Proceedings of the 8th Design Thinking Research Symposium (131–139). Sydney University of Technology, Sydney, New South Wales

Dorst, K. (2015). Frame Innovation: Create New Thinking by Design. The MIT Press . https://doi.org/10.7551/mitpress/10096.001.0001

Etzioni, A. (1964). Modern organizations . Prentice-Hall.

Faste, R. A. (1987). Perceiving needs. SAE Journal, Society of Automotive Engineers., 871534 , 1–5. https://doi.org/10.4271/871534

Geissdoerfer, M., Savaget, P., Bocken, N. M. P., & Hultink, E. J. (2017). The circular economy: A new sustainability paradigm? Journal of Cleaner Production, 143 , 757–768.

Germain, C. B. (1991). Human behavior in the social environment: An ecological view . Columbia University Press.

Goldstein, K. (1995). The organism: A holistic approach to biology derived from pathological data in man . Zone Books.

Gregory, S. A. (1966). Design and the design method. In S. A. Gregory (Ed.), The design Method. Plenum Press.

Head, B., & Xiang, W.-N. (2016). Working with wicked problems in socio-ecological systems: More awareness, greater acceptance, and better adaptation. Landscape and Urban Planning., 110 (2013), 1–4.

IDEO. (2011). Human centered design toolkit , 200.

IDEO. (n.d.). About. Retrieved from https://www.ideo.com/about

Johansson-Sköldberg, U., Woodilla, J., & Çetinkaya, M. (2013). Design thinking: Past, present and possible futures. Creativity and Innovation Management, 22 (2), 121–146.

Kimbell, L. (2011). Rethinking design thinking: Part I. Design and Culture, 3 (3), 285–306. https://doi.org/10.2752/175470811X13071166525216

Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development . PrenticeHall.

Kolb, A. Y., & Kolb, D. A. (2005). Learning Styles and Learning Spaces: Enhancing Experiential Learning in Higher Education. Academy of Management Learning & Education, 4 (2), 193–212. https://doi.org/10.2307/40214287?refreqid=search-gateway:69b4159ab026cceba348641bddd445b6

Lawson, B. (1980). How designers think: The design process demystified . Burlington: Elsevier Ltd.

Lewin, K. (1946). Action research and minority problems. Journal of Social Issues, 2 (4), 34–46.

Lockwood, T. (2010). Design Thinking: Integrating Innovation, Customer Experience and BrandValue . New York, NY: Allworth Press.

Luchs, M. G. (2015). A Brief introduction to design thinking. In M. G. Luchs, K. S. Swan, & A. Griffin (Eds.), Design thinking (pp. 1–11). Hoboken: Wiley.

Luhmann, N. (1995). Social systems . Stanford University Press.

Meadows, D. H. (2008). Thinking in systems: A primer . Chelsea Green Publishing.

Nonaka, I., & Takeuchi, H. (1995). The knowledge creating company . Oxford University Press.

Owen, C. L. (1998). Design research: Building the knowledge base. Design Studies, 19 (1), 9–20. https://doi.org/10.1016/s0142-694x(97)00030-6

Owen CL. (2006a) Design thinking: Driving innovation. The Business Process Management Institute BPMI.org . September 2006a

Owen, C. L. (2006b). Design thinking—notes on its nature and use. Design Research Quarterly, 1 (2), 16–27.

Rittel, H. (1984). Second-generation design methods. In N. Cross (Ed.), Developments in design methodology (pp. 317–327). Wiley.

Rittel, H. W. J., & Webber, M. M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4 (2), 155–169. https://doi.org/10.1007/BF01405730

Roberts, N. (2000). Wicked problems and network approaches to resolution. International Public Management Review, 1 (1), 1–19.

Rowe, P. (1987). Design thinking . The MIT Press.

Schön, D. A. (1983). The reflective practitioner: How professionals think in action . New York: Basic Books.

Senge, P. (1990). The Fifth Discipline: The art and practice of the learning organization . Doubleday.

Senge, P. (1996). Systems thinking. Executive Excellence, 13 (1), 15.

Simon, H. (1969). The sciences of the artificial . MIT Press.

Visser, W. (2010). Schön: Design as a reflective practice. Collection, 2 , 21–25.

Download references

Acknowledgements

Thank you to Wageningen University & Research and Fielding Graduate University for the opportunity to conduct this research in an entrepreneurial classroom setting. Ethical Approval through institutional review board (IRB) is detailed in Appendix B . This work was completed as part of doctoral research of Rahmin Bender (-Salazar) conducted for the Fielding Graduate University and at Wageningen University & Research and published with ProQuest as part of graduation requirements.

Not applicable.

Author information

Authors and affiliations.

Kemmy Business School, University of Limerick, Limerick, Ireland

Rahmin Bender-Salazar

You can also search for this author in PubMed   Google Scholar

Contributions

The author read and approved the final manuscript.

Corresponding author

Correspondence to Rahmin Bender-Salazar .

Ethics declarations

Competing interests.

The author declares no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix A: Interview Protocol—November 2018

[To open the conversation a bit of small talk and catching up with the former student, what they have been up to and what do they have planned next and this lines up to the informal questions below (in no particular order).]

Welcome and thank you for this time and to explore some of these concepts with you and get your perspective. Now that you have completed the Design Thinking course, I would like to explore with you whether, in your future career, you would consider design thinking as a way for teams to tackle difficult problems, and any ideas you may have on the subject. This is not designed in any way to test your knowledge about design thinking, or to reflect on how you did in class. I would simply like to understand whether, with what you’ve learned, you feel that design thinking is a good way to tackle tough problems, and how you would go about doing that.

Questions to warm up and understand context—5 ~ min

What is your major/main subject of study?

How do you want to use your education and what do you want to do as your vocation?

Design thinking and problem solving—40 min

[The purpose of the first question is to begin to brush on problem setting and begging the design thinking process, the parameters and elements. The goal is to solicit data from participants through storytelling and their thoughts on the topic.]

Can you tell me a story about your experience with design thinking in the class that you thought was memorable?

Are there other examples of things that struck you about design thinking?

What is it about the design thinking approach that you like the most?

Is there anything that you don’t like, or would do differently?

Let’s do some role playing. Let’s say, tomorrow you get hired by Apple to be the head of their new development team. They have a serious problem: the iPhone has reached a saturation point. You are tasked to come up with an entirely new set of functions that will totally reinvent the iPhone. How would you go about doing that, if you were using the design thinking approach? If you can, break it down using the three-phase hybrid model we discussed: Ideation-Prototyping-Implementation.

Is there anything about design thinking you feel you need to know more about, before you could confidently begin to use it?

Wrap up—10–15 min

So in sum, do you think design thinking a good method to produce disruptive innovation, or would you use other methods?

Does design thinking need to be adapted to the fast pace of disruptive change today?

Appendix B: Ethical Approval for Research—April 2018

1) IRB Approval Information

Name: Rahmin Bender.

IRB#: 17–1107

Title: Applying Design Thinking and Practice to team projects seeking to create regenerative and sustainable products to address the wicked problem of sustainable garments

Faculty: Fredrick Steier.

Type: Title Change and General Revisions.

2) Study Summary

The dissertation project seeks to explore through participatory action research, how the application of design methods to address wicked problems represents a disruptive innovation in the process of solution creation and if so or not, to what extent. The disruptive innovation is framed within the context of the Netherlands, the public University education system and the field of sustainable fashion and garment production. The specific context of this study will be at Wageningen University and Research in the Netherlands working with student teams creating business ideas, using design thinking and aligned methods, with the renewable materials in garment production. The forty Masters students in a circular economy course will be split into eight teams that will work with designers using these materials to create business and product concepts using design thinking processes facilitated by me.

3) Revision Checklist

I. Change title to: Applying design thinking to entrepreneurial learning spaces purposed with addressing wicked problems.

Title changed to emphasize more on the application of design thinking on the learn space and how it addresses the wicked problem, rather than focusing more and more on the

II. Change question 2 element (c) from “(c) how design process impacts team dynamics of product creation team” to (c) how design process impacts the co-creation of the entrepreneurial learning space.

Question changed to focus additionally on how using the design process not only impacts the outputs of the course but the course itself.

III. Change question 3’s following elements.

Change this bullet: “World Café held after the course to accumulate data and feedback from participants and put into context with the notes.”

New Text: Changed to Design Charrette held after the course to accumulate data, feedback and put notes into context through a participatory designing of future iterations of the course.

Change this bullet: “Depending on IRB is performed data collection will be focused on the World Café portion that will be held in January post course and the course and work will be looked at historically.”

New text: IRB includes data from the course that ended in the end of 2017 as well as data from the participatory design workshop titled a design charrette occurring in 25 April 2017.

Add the following bullet

Design-based Research informed by action research and design thinking will serve as the research method for analyzing the historic data from the course and data collected in the design charrette to address the research questions posed.

The above changes are made to reflect a change from a World Café method to a more intimate design charrette. This change was made because of difficulty getting a large enough participation for a World Café to work, ideally 20 or more people. The design charrette will use the same research element but be in a smaller setting, which will allow for more interaction. Finally, the addition of design-based research to emphasize the element of the entrepreneurial learning space and how that was actively formed and influenced by the use of design methods.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Bender-Salazar, R. Design thinking as an effective method for problem-setting and needfinding for entrepreneurial teams addressing wicked problems. J Innov Entrep 12 , 24 (2023). https://doi.org/10.1186/s13731-023-00291-2

Download citation

Received : 19 October 2022

Accepted : 05 April 2023

Published : 13 April 2023

DOI : https://doi.org/10.1186/s13731-023-00291-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Design thinking
• Problem setting
• Design process
• Wicked problems
• Systems thinking
• Organizational learning
• Needfinding

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is a Research Design | Types, Guide & Examples

What Is a Research Design | Types, Guide & Examples

Published on June 7, 2021 by Shona McCombes . Revised on November 20, 2023 by Pritha Bhandari.

A research design is a strategy for answering your   research question  using empirical data. Creating a research design means making decisions about:

• Your overall research objectives and approach
• Whether you’ll rely on primary research or secondary research
• Your sampling methods or criteria for selecting subjects
• Your data collection methods
• The procedures you’ll follow to collect data
• Your data analysis methods

A well-planned research design helps ensure that your methods match your research objectives and that you use the right kind of analysis for your data.

Table of contents

Step 1: consider your aims and approach, step 2: choose a type of research design, step 3: identify your population and sampling method, step 4: choose your data collection methods, step 5: plan your data collection procedures, step 6: decide on your data analysis strategies, other interesting articles, frequently asked questions about research design.

• Introduction

Before you can start designing your research, you should already have a clear idea of the research question you want to investigate.

There are many different ways you could go about answering this question. Your research design choices should be driven by your aims and priorities—start by thinking carefully about what you want to achieve.

The first choice you need to make is whether you’ll take a qualitative or quantitative approach.

Qualitative research designs tend to be more flexible and inductive , allowing you to adjust your approach based on what you find throughout the research process.

Quantitative research designs tend to be more fixed and deductive , with variables and hypotheses clearly defined in advance of data collection.

It’s also possible to use a mixed-methods design that integrates aspects of both approaches. By combining qualitative and quantitative insights, you can gain a more complete picture of the problem you’re studying and strengthen the credibility of your conclusions.

Practical and ethical considerations when designing research

As well as scientific considerations, you need to think practically when designing your research. If your research involves people or animals, you also need to consider research ethics .

• How much time do you have to collect data and write up the research?
• Will you be able to gain access to the data you need (e.g., by travelling to a specific location or contacting specific people)?
• Do you have the necessary research skills (e.g., statistical analysis or interview techniques)?
• Will you need ethical approval ?

At each stage of the research design process, make sure that your choices are practically feasible.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Within both qualitative and quantitative approaches, there are several types of research design to choose from. Each type provides a framework for the overall shape of your research.

Types of quantitative research designs

Quantitative designs can be split into four main types.

• Experimental and   quasi-experimental designs allow you to test cause-and-effect relationships
• Descriptive and correlational designs allow you to measure variables and describe relationships between them.

With descriptive and correlational designs, you can get a clear picture of characteristics, trends and relationships as they exist in the real world. However, you can’t draw conclusions about cause and effect (because correlation doesn’t imply causation ).

Experiments are the strongest way to test cause-and-effect relationships without the risk of other variables influencing the results. However, their controlled conditions may not always reflect how things work in the real world. They’re often also more difficult and expensive to implement.

Types of qualitative research designs

Qualitative designs are less strictly defined. This approach is about gaining a rich, detailed understanding of a specific context or phenomenon, and you can often be more creative and flexible in designing your research.

The table below shows some common types of qualitative design. They often have similar approaches in terms of data collection, but focus on different aspects when analyzing the data.

Your research design should clearly define who or what your research will focus on, and how you’ll go about choosing your participants or subjects.

In research, a population is the entire group that you want to draw conclusions about, while a sample is the smaller group of individuals you’ll actually collect data from.

Defining the population

A population can be made up of anything you want to study—plants, animals, organizations, texts, countries, etc. In the social sciences, it most often refers to a group of people.

For example, will you focus on people from a specific demographic, region or background? Are you interested in people with a certain job or medical condition, or users of a particular product?

The more precisely you define your population, the easier it will be to gather a representative sample.

• Sampling methods

Even with a narrowly defined population, it’s rarely possible to collect data from every individual. Instead, you’ll collect data from a sample.

To select a sample, there are two main approaches: probability sampling and non-probability sampling . The sampling method you use affects how confidently you can generalize your results to the population as a whole.

Probability sampling is the most statistically valid option, but it’s often difficult to achieve unless you’re dealing with a very small and accessible population.

For practical reasons, many studies use non-probability sampling, but it’s important to be aware of the limitations and carefully consider potential biases. You should always make an effort to gather a sample that’s as representative as possible of the population.

Case selection in qualitative research

In some types of qualitative designs, sampling may not be relevant.

For example, in an ethnography or a case study , your aim is to deeply understand a specific context, not to generalize to a population. Instead of sampling, you may simply aim to collect as much data as possible about the context you are studying.

In these types of design, you still have to carefully consider your choice of case or community. You should have a clear rationale for why this particular case is suitable for answering your research question .

For example, you might choose a case study that reveals an unusual or neglected aspect of your research problem, or you might choose several very similar or very different cases in order to compare them.

Data collection methods are ways of directly measuring variables and gathering information. They allow you to gain first-hand knowledge and original insights into your research problem.

You can choose just one data collection method, or use several methods in the same study.

Survey methods

Surveys allow you to collect data about opinions, behaviors, experiences, and characteristics by asking people directly. There are two main survey methods to choose from: questionnaires and interviews .

Observation methods

Observational studies allow you to collect data unobtrusively, observing characteristics, behaviors or social interactions without relying on self-reporting.

Observations may be conducted in real time, taking notes as you observe, or you might make audiovisual recordings for later analysis. They can be qualitative or quantitative.

Other methods of data collection

There are many other ways you might collect data depending on your field and topic.

If you’re not sure which methods will work best for your research design, try reading some papers in your field to see what kinds of data collection methods they used.

Secondary data

If you don’t have the time or resources to collect data from the population you’re interested in, you can also choose to use secondary data that other researchers already collected—for example, datasets from government surveys or previous studies on your topic.

With this raw data, you can do your own analysis to answer new research questions that weren’t addressed by the original study.

Using secondary data can expand the scope of your research, as you may be able to access much larger and more varied samples than you could collect yourself.

However, it also means you don’t have any control over which variables to measure or how to measure them, so the conclusions you can draw may be limited.

Prevent plagiarism. Run a free check.

As well as deciding on your methods, you need to plan exactly how you’ll use these methods to collect data that’s consistent, accurate, and unbiased.

Planning systematic procedures is especially important in quantitative research, where you need to precisely define your variables and ensure your measurements are high in reliability and validity.

Operationalization

Some variables, like height or age, are easily measured. But often you’ll be dealing with more abstract concepts, like satisfaction, anxiety, or competence. Operationalization means turning these fuzzy ideas into measurable indicators.

If you’re using observations , which events or actions will you count?

If you’re using surveys , which questions will you ask and what range of responses will be offered?

You may also choose to use or adapt existing materials designed to measure the concept you’re interested in—for example, questionnaires or inventories whose reliability and validity has already been established.

Reliability and validity

Reliability means your results can be consistently reproduced, while validity means that you’re actually measuring the concept you’re interested in.

For valid and reliable results, your measurement materials should be thoroughly researched and carefully designed. Plan your procedures to make sure you carry out the same steps in the same way for each participant.

If you’re developing a new questionnaire or other instrument to measure a specific concept, running a pilot study allows you to check its validity and reliability in advance.

Sampling procedures

As well as choosing an appropriate sampling method , you need a concrete plan for how you’ll actually contact and recruit your selected sample.

That means making decisions about things like:

• How many participants do you need for an adequate sample size?
• What inclusion and exclusion criteria will you use to identify eligible participants?
• How will you contact your sample—by mail, online, by phone, or in person?

If you’re using a probability sampling method , it’s important that everyone who is randomly selected actually participates in the study. How will you ensure a high response rate?

If you’re using a non-probability method , how will you avoid research bias and ensure a representative sample?

Data management

It’s also important to create a data management plan for organizing and storing your data.

Will you need to transcribe interviews or perform data entry for observations? You should anonymize and safeguard any sensitive data, and make sure it’s backed up regularly.

Keeping your data well-organized will save time when it comes to analyzing it. It can also help other researchers validate and add to your findings (high replicability ).

On its own, raw data can’t answer your research question. The last step of designing your research is planning how you’ll analyze the data.

Quantitative data analysis

In quantitative research, you’ll most likely use some form of statistical analysis . With statistics, you can summarize your sample data, make estimates, and test hypotheses.

Using descriptive statistics , you can summarize your sample data in terms of:

• The distribution of the data (e.g., the frequency of each score on a test)
• The central tendency of the data (e.g., the mean to describe the average score)
• The variability of the data (e.g., the standard deviation to describe how spread out the scores are)

The specific calculations you can do depend on the level of measurement of your variables.

Using inferential statistics , you can:

• Make estimates about the population based on your sample data.
• Test hypotheses about a relationship between variables.

Regression and correlation tests look for associations between two or more variables, while comparison tests (such as t tests and ANOVAs ) look for differences in the outcomes of different groups.

Your choice of statistical test depends on various aspects of your research design, including the types of variables you’re dealing with and the distribution of your data.

Qualitative data analysis

In qualitative research, your data will usually be very dense with information and ideas. Instead of summing it up in numbers, you’ll need to comb through the data in detail, interpret its meanings, identify patterns, and extract the parts that are most relevant to your research question.

Two of the most common approaches to doing this are thematic analysis and discourse analysis .

There are many other ways of analyzing qualitative data depending on the aims of your research. To get a sense of potential approaches, try reading some qualitative research papers in your field.

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A research design is a strategy for answering your   research question . It defines your overall approach and determines how you will collect and analyze data.

A well-planned research design helps ensure that your methods match your research aims, that you collect high-quality data, and that you use the right kind of analysis to answer your questions, utilizing credible sources . This allows you to draw valid , trustworthy conclusions.

Quantitative research designs can be divided into two main categories:

• Correlational and descriptive designs are used to investigate characteristics, averages, trends, and associations between variables.
• Experimental and quasi-experimental designs are used to test causal relationships .

Qualitative research designs tend to be more flexible. Common types of qualitative design include case study , ethnography , and grounded theory designs.

The priorities of a research design can vary depending on the field, but you usually have to specify:

• Your research questions and/or hypotheses
• Your overall approach (e.g., qualitative or quantitative )
• The type of design you’re using (e.g., a survey , experiment , or case study )
• Your data collection methods (e.g., questionnaires , observations)
• Your data collection procedures (e.g., operationalization , timing and data management)
• Your data analysis methods (e.g., statistical tests  or thematic analysis )

A sample is a subset of individuals from a larger population . Sampling means selecting the group that you will actually collect data from in your research. For example, if you are researching the opinions of students in your university, you could survey a sample of 100 students.

In statistics, sampling allows you to test a hypothesis about the characteristics of a population.

Operationalization means turning abstract conceptual ideas into measurable observations.

For example, the concept of social anxiety isn’t directly observable, but it can be operationally defined in terms of self-rating scores, behavioral avoidance of crowded places, or physical anxiety symptoms in social situations.

Before collecting data , it’s important to consider how you will operationalize the variables that you want to measure.

A research project is an academic, scientific, or professional undertaking to answer a research question . Research projects can take many forms, such as qualitative or quantitative , descriptive , longitudinal , experimental , or correlational . What kind of research approach you choose will depend on your topic.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, November 20). What Is a Research Design | Types, Guide & Examples. Scribbr. Retrieved April 5, 2024, from https://www.scribbr.com/methodology/research-design/

Is this article helpful?

Shona McCombes

Other students also liked, guide to experimental design | overview, steps, & examples, how to write a research proposal | examples & templates, ethical considerations in research | types & examples, unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Help | Advanced Search

Computer Science > Human-Computer Interaction

Title: "that's not good science": an argument for the thoughtful use of formative situations in research through design.

Abstract: Most currently accepted approaches to evaluating Research through Design (RtD) presume that design prototypes are finalized and ready for robust testing in laboratory or in-the-wild settings. However, it is also valuable to assess designs at intermediate phases with mid-fidelity prototypes, not just to inform an ongoing design process, but also to glean knowledge of broader use to the research community. We propose 'formative situations' as a frame for examining mid-fidelity prototypes-in-process in this way. We articulate a set of criteria to help the community better assess the rigor of formative situations, in the service of opening conversation about establishing formative situations as a valuable contribution type within the RtD community.

Submission history

Access paper:.

• HTML (experimental)
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

• How it works

How to Write a Research Design – Guide with Examples

Published by Alaxendra Bets at August 14th, 2021 , Revised On October 3, 2023

A research design is a structure that combines different components of research. It involves the use of different data collection and data analysis techniques logically to answer the  research questions .

It would be best to make some decisions about addressing the research questions adequately before starting the research process, which is achieved with the help of the research design.

Below are the key aspects of the decision-making process:

• Data type required for research
• Research resources
• Participants required for research
• Hypothesis based upon research question(s)
• Data analysis  methodologies
• Variables (Independent, dependent, and confounding)
• The location and timescale for conducting the data
• The time period required for research

The research design provides the strategy of investigation for your project. Furthermore, it defines the parameters and criteria to compile the data to evaluate results and conclude.

Your project’s validity depends on the data collection and  interpretation techniques.  A strong research design reflects a strong  dissertation , scientific paper, or research proposal .

Step 1: Establish Priorities for Research Design

Before conducting any research study, you must address an important question: “how to create a research design.”

The research design depends on the researcher’s priorities and choices because every research has different priorities. For a complex research study involving multiple methods, you may choose to have more than one research design.

Multimethodology or multimethod research includes using more than one data collection method or research in a research study or set of related studies.

If one research design is weak in one area, then another research design can cover that weakness. For instance, a  dissertation analyzing different situations or cases will have more than one research design.

For example:

• Experimental research involves experimental investigation and laboratory experience, but it does not accurately investigate the real world.
• Quantitative research is good for the  statistical part of the project, but it may not provide an in-depth understanding of the  topic .
• Also, correlational research will not provide experimental results because it is a technique that assesses the statistical relationship between two variables.

While scientific considerations are a fundamental aspect of the research design, It is equally important that the researcher think practically before deciding on its structure. Here are some questions that you should think of;

• Do you have enough time to gather data and complete the write-up?
• Will you be able to collect the necessary data by interviewing a specific person or visiting a specific location?
• Do you have in-depth knowledge about the  different statistical analysis and data collection techniques to address the research questions  or test the  hypothesis ?

If you think that the chosen research design cannot answer the research questions properly, you can refine your research questions to gain better insight.

Step 2: Data Type you Need for Research

Decide on the type of data you need for your research. The type of data you need to collect depends on your research questions or research hypothesis. Two types of research data can be used to answer the research questions:

Primary Data Vs. Secondary Data

Qualitative vs. quantitative data.

Also, see; Research methods, design, and analysis .

Need help with a thesis chapter?

• Hire an expert from ResearchProspect today!
• Statistical analysis, research methodology, discussion of the results or conclusion – our experts can help you no matter how complex the requirements are.

Step 3: Data Collection Techniques

Once you have selected the type of research to answer your research question, you need to decide where and how to collect the data.

It is time to determine your research method to address the  research problem . Research methods involve procedures, techniques, materials, and tools used for the study.

For instance, a dissertation research design includes the different resources and data collection techniques and helps establish your  dissertation’s structure .

The following table shows the characteristics of the most popularly employed research methods.

Research Methods

Step 4: Procedure of Data Analysis

Use of the  correct data and statistical analysis technique is necessary for the validity of your research. Therefore, you need to be certain about the data type that would best address the research problem. Choosing an appropriate analysis method is the final step for the research design. It can be split into two main categories;

Quantitative Data Analysis

The quantitative data analysis technique involves analyzing the numerical data with the help of different applications such as; SPSS, STATA, Excel, origin lab, etc.

This data analysis strategy tests different variables such as spectrum, frequencies, averages, and more. The research question and the hypothesis must be established to identify the variables for testing.

Qualitative Data Analysis

Qualitative data analysis of figures, themes, and words allows for flexibility and the researcher’s subjective opinions. This means that the researcher’s primary focus will be interpreting patterns, tendencies, and accounts and understanding the implications and social framework.

You should be clear about your research objectives before starting to analyze the data. For example, you should ask yourself whether you need to explain respondents’ experiences and insights or do you also need to evaluate their responses with reference to a certain social framework.

Step 5: Write your Research Proposal

The research design is an important component of a research proposal because it plans the project’s execution. You can share it with the supervisor, who would evaluate the feasibility and capacity of the results  and  conclusion .

Read our guidelines to write a research proposal  if you have already formulated your research design. The research proposal is written in the future tense because you are writing your proposal before conducting research.

The  research methodology  or research design, on the other hand, is generally written in the past tense.

How to Write a Research Design – Conclusion

A research design is the plan, structure, strategy of investigation conceived to answer the research question and test the hypothesis. The dissertation research design can be classified based on the type of data and the type of analysis.

Above mentioned five steps are the answer to how to write a research design. So, follow these steps to  formulate the perfect research design for your dissertation .

ResearchProspect writers have years of experience creating research designs that align with the dissertation’s aim and objectives. If you are struggling with your dissertation methodology chapter, you might want to look at our dissertation part-writing service.

Our dissertation writers can also help you with the full dissertation paper . No matter how urgent or complex your need may be, ResearchProspect can help. We also offer PhD level research paper writing services.

Frequently Asked Questions

What is research design.

Research design is a systematic plan that guides the research process, outlining the methodology and procedures for collecting and analysing data. It determines the structure of the study, ensuring the research question is answered effectively, reliably, and validly. It serves as the blueprint for the entire research project.

How to write a research design?

To write a research design, define your research question, identify the research method (qualitative, quantitative, or mixed), choose data collection techniques (e.g., surveys, interviews), determine the sample size and sampling method, outline data analysis procedures, and highlight potential limitations and ethical considerations for the study.

How to write the design section of a research paper?

In the design section of a research paper, describe the research methodology chosen and justify its selection. Outline the data collection methods, participants or samples, instruments used, and procedures followed. Detail any experimental controls, if applicable. Ensure clarity and precision to enable replication of the study by other researchers.

How to write a research design in methodology?

To write a research design in methodology, clearly outline the research strategy (e.g., experimental, survey, case study). Describe the sampling technique, participants, and data collection methods. Detail the procedures for data collection and analysis. Justify choices by linking them to research objectives, addressing reliability and validity.

You May Also Like

Find how to write research questions with the mentioned steps required for a perfect research question. Choose an interesting topic and begin your research.

How to write a hypothesis for dissertation,? A hypothesis is a statement that can be tested with the help of experimental or theoretical research.

Not sure how to approach a company for your primary research study? Don’t worry. Here we have some tips for you to successfully gather primary study.

USEFUL LINKS

LEARNING RESOURCES

COMPANY DETAILS

• How It Works

• < Previous

Home > DRS Conference Proceedings > DRS Conferences > DRS2016 > Papers > Design practice and design re...

DRS2016 Research Papers

Design practice and design research: finally together?

Kees Dorst , University of Technology Sydney and Eindhoven University of Technology

Early design research was driven by the ambition to create a coherent Science of Design – an ambition that was later abandoned in favour of a more pluralist approach. But despite great progress in the last 50 years, Design Research can still be criticised for being (1) too disconnected from design practice, (2) internally scattered and confused (3) not achieving the impact that was hoped for. In this paper we will discuss possible solutions to these conundrums by learning from three professional and academic fields: Marketing, Art Theory and Management, respectively. Based on these three discussions an attempt will be made to create an integrated answer by considering how design research and practice might come together in the creation of a new field, “Academic Design”.

design research; design practice; academic design

https://doi.org/10.21606/drs.2016.212

Dorst, K. (2016) Design practice and design research: finally together?, in Lloyd, P. and Bohemia, E. (eds.), Future Focused Thinking - DRS International Conference 2016 , 27 - 30 June, Brighton, United Kingdom. https://doi.org/10.21606/drs.2016.212

Creative Commons License

Since October 06, 2020

To view the content in your browser, please download Adobe Reader or, alternately, you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.

Advanced Search

• Notify me via email or RSS
• Collections
• - DRS Conferences
• - Nordes Conferences
• - IASDR Conferences
• - Learn X Design
• - Pluriversal Design
• - Doctoral Education
• - DRS 1971-1980
• - DRS Archive
• DRS conferences peer review policy
• Notifications
• Join the DRS

Home | About | My Account | Accessibility Statement | Privacy | Copyright | Contact Us

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology

Research Design | Step-by-Step Guide with Examples

Published on 5 May 2022 by Shona McCombes . Revised on 20 March 2023.

A research design is a strategy for answering your research question  using empirical data. Creating a research design means making decisions about:

• Your overall aims and approach
• The type of research design you’ll use
• Your sampling methods or criteria for selecting subjects
• Your data collection methods
• The procedures you’ll follow to collect data
• Your data analysis methods

A well-planned research design helps ensure that your methods match your research aims and that you use the right kind of analysis for your data.

Table of contents

Step 1: consider your aims and approach, step 2: choose a type of research design, step 3: identify your population and sampling method, step 4: choose your data collection methods, step 5: plan your data collection procedures, step 6: decide on your data analysis strategies, frequently asked questions.

• Introduction

Before you can start designing your research, you should already have a clear idea of the research question you want to investigate.

There are many different ways you could go about answering this question. Your research design choices should be driven by your aims and priorities – start by thinking carefully about what you want to achieve.

The first choice you need to make is whether you’ll take a qualitative or quantitative approach.

Qualitative research designs tend to be more flexible and inductive , allowing you to adjust your approach based on what you find throughout the research process.

Quantitative research designs tend to be more fixed and deductive , with variables and hypotheses clearly defined in advance of data collection.

It’s also possible to use a mixed methods design that integrates aspects of both approaches. By combining qualitative and quantitative insights, you can gain a more complete picture of the problem you’re studying and strengthen the credibility of your conclusions.

Practical and ethical considerations when designing research

As well as scientific considerations, you need to think practically when designing your research. If your research involves people or animals, you also need to consider research ethics .

• How much time do you have to collect data and write up the research?
• Will you be able to gain access to the data you need (e.g., by travelling to a specific location or contacting specific people)?
• Do you have the necessary research skills (e.g., statistical analysis or interview techniques)?
• Will you need ethical approval ?

At each stage of the research design process, make sure that your choices are practically feasible.

Prevent plagiarism, run a free check.

Within both qualitative and quantitative approaches, there are several types of research design to choose from. Each type provides a framework for the overall shape of your research.

Types of quantitative research designs

Quantitative designs can be split into four main types. Experimental and   quasi-experimental designs allow you to test cause-and-effect relationships, while descriptive and correlational designs allow you to measure variables and describe relationships between them.

With descriptive and correlational designs, you can get a clear picture of characteristics, trends, and relationships as they exist in the real world. However, you can’t draw conclusions about cause and effect (because correlation doesn’t imply causation ).

Experiments are the strongest way to test cause-and-effect relationships without the risk of other variables influencing the results. However, their controlled conditions may not always reflect how things work in the real world. They’re often also more difficult and expensive to implement.

Types of qualitative research designs

Qualitative designs are less strictly defined. This approach is about gaining a rich, detailed understanding of a specific context or phenomenon, and you can often be more creative and flexible in designing your research.

The table below shows some common types of qualitative design. They often have similar approaches in terms of data collection, but focus on different aspects when analysing the data.

Your research design should clearly define who or what your research will focus on, and how you’ll go about choosing your participants or subjects.

In research, a population is the entire group that you want to draw conclusions about, while a sample is the smaller group of individuals you’ll actually collect data from.

Defining the population

A population can be made up of anything you want to study – plants, animals, organisations, texts, countries, etc. In the social sciences, it most often refers to a group of people.

For example, will you focus on people from a specific demographic, region, or background? Are you interested in people with a certain job or medical condition, or users of a particular product?

The more precisely you define your population, the easier it will be to gather a representative sample.

Sampling methods

Even with a narrowly defined population, it’s rarely possible to collect data from every individual. Instead, you’ll collect data from a sample.

To select a sample, there are two main approaches: probability sampling and non-probability sampling . The sampling method you use affects how confidently you can generalise your results to the population as a whole.

Probability sampling is the most statistically valid option, but it’s often difficult to achieve unless you’re dealing with a very small and accessible population.

For practical reasons, many studies use non-probability sampling, but it’s important to be aware of the limitations and carefully consider potential biases. You should always make an effort to gather a sample that’s as representative as possible of the population.

Case selection in qualitative research

In some types of qualitative designs, sampling may not be relevant.

For example, in an ethnography or a case study, your aim is to deeply understand a specific context, not to generalise to a population. Instead of sampling, you may simply aim to collect as much data as possible about the context you are studying.

In these types of design, you still have to carefully consider your choice of case or community. You should have a clear rationale for why this particular case is suitable for answering your research question.

For example, you might choose a case study that reveals an unusual or neglected aspect of your research problem, or you might choose several very similar or very different cases in order to compare them.

Data collection methods are ways of directly measuring variables and gathering information. They allow you to gain first-hand knowledge and original insights into your research problem.

You can choose just one data collection method, or use several methods in the same study.

Survey methods

Surveys allow you to collect data about opinions, behaviours, experiences, and characteristics by asking people directly. There are two main survey methods to choose from: questionnaires and interviews.

Observation methods

Observations allow you to collect data unobtrusively, observing characteristics, behaviours, or social interactions without relying on self-reporting.

Observations may be conducted in real time, taking notes as you observe, or you might make audiovisual recordings for later analysis. They can be qualitative or quantitative.

Other methods of data collection

There are many other ways you might collect data depending on your field and topic.

If you’re not sure which methods will work best for your research design, try reading some papers in your field to see what data collection methods they used.

Secondary data

If you don’t have the time or resources to collect data from the population you’re interested in, you can also choose to use secondary data that other researchers already collected – for example, datasets from government surveys or previous studies on your topic.

With this raw data, you can do your own analysis to answer new research questions that weren’t addressed by the original study.

Using secondary data can expand the scope of your research, as you may be able to access much larger and more varied samples than you could collect yourself.

However, it also means you don’t have any control over which variables to measure or how to measure them, so the conclusions you can draw may be limited.

As well as deciding on your methods, you need to plan exactly how you’ll use these methods to collect data that’s consistent, accurate, and unbiased.

Planning systematic procedures is especially important in quantitative research, where you need to precisely define your variables and ensure your measurements are reliable and valid.

Operationalisation

Some variables, like height or age, are easily measured. But often you’ll be dealing with more abstract concepts, like satisfaction, anxiety, or competence. Operationalisation means turning these fuzzy ideas into measurable indicators.

If you’re using observations , which events or actions will you count?

If you’re using surveys , which questions will you ask and what range of responses will be offered?

You may also choose to use or adapt existing materials designed to measure the concept you’re interested in – for example, questionnaires or inventories whose reliability and validity has already been established.

Reliability and validity

Reliability means your results can be consistently reproduced , while validity means that you’re actually measuring the concept you’re interested in.

For valid and reliable results, your measurement materials should be thoroughly researched and carefully designed. Plan your procedures to make sure you carry out the same steps in the same way for each participant.

If you’re developing a new questionnaire or other instrument to measure a specific concept, running a pilot study allows you to check its validity and reliability in advance.

Sampling procedures

As well as choosing an appropriate sampling method, you need a concrete plan for how you’ll actually contact and recruit your selected sample.

That means making decisions about things like:

• How many participants do you need for an adequate sample size?
• What inclusion and exclusion criteria will you use to identify eligible participants?
• How will you contact your sample – by mail, online, by phone, or in person?

If you’re using a probability sampling method, it’s important that everyone who is randomly selected actually participates in the study. How will you ensure a high response rate?

If you’re using a non-probability method, how will you avoid bias and ensure a representative sample?

Data management

It’s also important to create a data management plan for organising and storing your data.

Will you need to transcribe interviews or perform data entry for observations? You should anonymise and safeguard any sensitive data, and make sure it’s backed up regularly.

Keeping your data well organised will save time when it comes to analysing them. It can also help other researchers validate and add to your findings.

On their own, raw data can’t answer your research question. The last step of designing your research is planning how you’ll analyse the data.

Quantitative data analysis

In quantitative research, you’ll most likely use some form of statistical analysis . With statistics, you can summarise your sample data, make estimates, and test hypotheses.

Using descriptive statistics , you can summarise your sample data in terms of:

• The distribution of the data (e.g., the frequency of each score on a test)
• The central tendency of the data (e.g., the mean to describe the average score)
• The variability of the data (e.g., the standard deviation to describe how spread out the scores are)

The specific calculations you can do depend on the level of measurement of your variables.

Using inferential statistics , you can:

• Make estimates about the population based on your sample data.
• Test hypotheses about a relationship between variables.

Regression and correlation tests look for associations between two or more variables, while comparison tests (such as t tests and ANOVAs ) look for differences in the outcomes of different groups.

Your choice of statistical test depends on various aspects of your research design, including the types of variables you’re dealing with and the distribution of your data.

Qualitative data analysis

In qualitative research, your data will usually be very dense with information and ideas. Instead of summing it up in numbers, you’ll need to comb through the data in detail, interpret its meanings, identify patterns, and extract the parts that are most relevant to your research question.

Two of the most common approaches to doing this are thematic analysis and discourse analysis .

There are many other ways of analysing qualitative data depending on the aims of your research. To get a sense of potential approaches, try reading some qualitative research papers in your field.

A sample is a subset of individuals from a larger population. Sampling means selecting the group that you will actually collect data from in your research.

For example, if you are researching the opinions of students in your university, you could survey a sample of 100 students.

Statistical sampling allows you to test a hypothesis about the characteristics of a population. There are various sampling methods you can use to ensure that your sample is representative of the population as a whole.

Operationalisation means turning abstract conceptual ideas into measurable observations.

For example, the concept of social anxiety isn’t directly observable, but it can be operationally defined in terms of self-rating scores, behavioural avoidance of crowded places, or physical anxiety symptoms in social situations.

Before collecting data , it’s important to consider how you will operationalise the variables that you want to measure.

The research methods you use depend on the type of data you need to answer your research question .

• If you want to measure something or test a hypothesis , use quantitative methods . If you want to explore ideas, thoughts, and meanings, use qualitative methods .
• If you want to analyse a large amount of readily available data, use secondary data. If you want data specific to your purposes with control over how they are generated, collect primary data.
• If you want to establish cause-and-effect relationships between variables , use experimental methods. If you want to understand the characteristics of a research subject, use descriptive methods.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

McCombes, S. (2023, March 20). Research Design | Step-by-Step Guide with Examples. Scribbr. Retrieved 2 April 2024, from https://www.scribbr.co.uk/research-methods/research-design/

Is this article helpful?

Shona McCombes

Organizing Academic Research Papers: Types of Research Designs

• Purpose of Guide
• Design Flaws to Avoid
• Glossary of Research Terms
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Choosing a Title
• Making an Outline
• Paragraph Development
• Executive Summary
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tertiary Sources
• What Is Scholarly vs. Popular?
• Qualitative Methods
• Quantitative Methods
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Annotated Bibliography
• Dealing with Nervousness
• Using Visual Aids
• Grading Someone Else's Paper
• How to Manage Group Projects
• Multiple Book Review Essay
• Reviewing Collected Essays
• About Informed Consent
• Writing Field Notes
• Writing a Policy Memo
• Writing a Research Proposal
• Acknowledgements

Introduction

Before beginning your paper, you need to decide how you plan to design the study .

The research design refers to the overall strategy that you choose to integrate the different components of the study in a coherent and logical way, thereby, ensuring you will effectively address the research problem; it constitutes the blueprint for the collection, measurement, and analysis of data. Note that your research problem determines the type of design you can use, not the other way around!

General Structure and Writing Style

Action research design, case study design, causal design, cohort design, cross-sectional design, descriptive design, experimental design, exploratory design, historical design, longitudinal design, observational design, philosophical design, sequential design.

Kirshenblatt-Gimblett, Barbara. Part 1, What Is Research Design? The Context of Design. Performance Studies Methods Course syllabus . New York University, Spring 2006; Trochim, William M.K. Research Methods Knowledge Base . 2006.

The function of a research design is to ensure that the evidence obtained enables you to effectively address the research problem as unambiguously as possible. In social sciences research, obtaining evidence relevant to the research problem generally entails specifying the type of evidence needed to test a theory, to evaluate a program, or to accurately describe a phenomenon. However, researchers can often begin their investigations far too early, before they have thought critically about about what information is required to answer the study's research questions. Without attending to these design issues beforehand, the conclusions drawn risk being weak and unconvincing and, consequently, will fail to adequate address the overall research problem.

 Given this, the length and complexity of research designs can vary considerably, but any sound design will do the following things:

• Identify the research problem clearly and justify its selection,
• Review previously published literature associated with the problem area,
• Clearly and explicitly specify hypotheses [i.e., research questions] central to the problem selected,
• Effectively describe the data which will be necessary for an adequate test of the hypotheses and explain how such data will be obtained, and
• Describe the methods of analysis which will be applied to the data in determining whether or not the hypotheses are true or false.

Kirshenblatt-Gimblett, Barbara. Part 1, What Is Research Design? The Context of Design. Performance Studies Methods Course syllabus . New Yortk University, Spring 2006.

Definition and Purpose

The essentials of action research design follow a characteristic cycle whereby initially an exploratory stance is adopted, where an understanding of a problem is developed and plans are made for some form of interventionary strategy. Then the intervention is carried out (the action in Action Research) during which time, pertinent observations are collected in various forms. The new interventional strategies are carried out, and the cyclic process repeats, continuing until a sufficient understanding of (or implement able solution for) the problem is achieved. The protocol is iterative or cyclical in nature and is intended to foster deeper understanding of a given situation, starting with conceptualizing and particularizing the problem and moving through several interventions and evaluations.

What do these studies tell you?

• A collaborative and adaptive research design that lends itself to use in work or community situations.
• Design focuses on pragmatic and solution-driven research rather than testing theories.
• When practitioners use action research it has the potential to increase the amount they learn consciously from their experience. The action research cycle can also be regarded as a learning cycle.
• Action search studies often have direct and obvious relevance to practice.
• There are no hidden controls or preemption of direction by the researcher.

What these studies don't tell you?

• It is harder to do than conducting conventional studies because the researcher takes on responsibilities for encouraging change as well as for research.
• Action research is much harder to write up because you probably can’t use a standard format to report your findings effectively.
• Personal over-involvement of the researcher may bias research results.
• The cyclic nature of action research to achieve its twin outcomes of action (e.g. change) and research (e.g. understanding) is time-consuming and complex to conduct.

Gall, Meredith. Educational Research: An Introduction . Chapter 18, Action Research. 8th ed. Boston, MA: Pearson/Allyn and Bacon, 2007; Kemmis, Stephen and Robin McTaggart. “Participatory Action Research.” In Handbook of Qualitative Research . Norman Denzin and Yvonna S. Locoln, eds. 2nd ed. (Thousand Oaks, CA: SAGE, 2000), pp. 567-605.; Reason, Peter and Hilary Bradbury. Handbook of Action Research: Participative Inquiry and Practice . Thousand Oaks, CA: SAGE, 2001.

A case study is an in-depth study of a particular research problem rather than a sweeping statistical survey. It is often used to narrow down a very broad field of research into one or a few easily researchable examples. The case study research design is also useful for testing whether a specific theory and model actually applies to phenomena in the real world. It is a useful design when not much is known about a phenomenon.

• Approach excels at bringing us to an understanding of a complex issue through detailed contextual analysis of a limited number of events or conditions and their relationships.
• A researcher using a case study design can apply a vaiety of methodologies and rely on a variety of sources to investigate a research problem.
• Design can extend experience or add strength to what is already known through previous research.
• Social scientists, in particular, make wide use of this research design to examine contemporary real-life situations and provide the basis for the application of concepts and theories and extension of methods.
• The design can provide detailed descriptions of specific and rare cases.
• A single or small number of cases offers little basis for establishing reliability or to generalize the findings to a wider population of people, places, or things.
• The intense exposure to study of the case may bias a researcher's interpretation of the findings.
• Design does not facilitate assessment of cause and effect relationships.
• Vital information may be missing, making the case hard to interpret.
• The case may not be representative or typical of the larger problem being investigated.
• If the criteria for selecting a case is because it represents a very unusual or unique phenomenon or problem for study, then your intepretation of the findings can only apply to that particular case.

Anastas, Jeane W. Research Design for Social Work and the Human Services . Chapter 4, Flexible Methods: Case Study Design. 2nd ed. New York: Columbia University Press, 1999; Stake, Robert E. The Art of Case Study Research . Thousand Oaks, CA: SAGE, 1995; Yin, Robert K. Case Study Research: Design and Theory . Applied Social Research Methods Series, no. 5. 3rd ed. Thousand Oaks, CA: SAGE, 2003.

Causality studies may be thought of as understanding a phenomenon in terms of conditional statements in the form, “If X, then Y.” This type of research is used to measure what impact a specific change will have on existing norms and assumptions. Most social scientists seek causal explanations that reflect tests of hypotheses. Causal effect (nomothetic perspective) occurs when variation in one phenomenon, an independent variable, leads to or results, on average, in variation in another phenomenon, the dependent variable.

Conditions necessary for determining causality:

• Empirical association--a valid conclusion is based on finding an association between the independent variable and the dependent variable.
• Appropriate time order--to conclude that causation was involved, one must see that cases were exposed to variation in the independent variable before variation in the dependent variable.
• Nonspuriousness--a relationship between two variables that is not due to variation in a third variable.
• Causality research designs helps researchers understand why the world works the way it does through the process of proving a causal link between variables and eliminating other possibilities.
• Replication is possible.
• There is greater confidence the study has internal validity due to the systematic subject selection and equity of groups being compared.
• Not all relationships are casual! The possibility always exists that, by sheer coincidence, two unrelated events appear to be related [e.g., Punxatawney Phil could accurately predict the duration of Winter for five consecutive years but, the fact remains, he's just a big, furry rodent].
• Conclusions about causal relationships are difficult to determine due to a variety of extraneous and confounding variables that exist in a social environment. This means causality can only be inferred, never proven.
• If two variables are correlated, the cause must come before the effect. However, even though two variables might be causally related, it can sometimes be difficult to determine which variable comes first and therefore to establish which variable is the actual cause and which is the  actual effect.

Bachman, Ronet. The Practice of Research in Criminology and Criminal Justice . Chapter 5, Causation and Research Designs. 3rd ed.  Thousand Oaks, CA: Pine Forge Press, 2007; Causal Research Design: Experimentation. Anonymous SlideShare Presentation ; Gall, Meredith. Educational Research: An Introduction . Chapter 11, Nonexperimental Research: Correlational Designs. 8th ed. Boston, MA: Pearson/Allyn and Bacon, 2007; Trochim, William M.K. Research Methods Knowledge Base . 2006.

Often used in the medical sciences, but also found in the applied social sciences, a cohort study generally refers to a study conducted over a period of time involving members of a population which the subject or representative member comes from, and who are united by some commonality or similarity. Using a quantitative framework, a cohort study makes note of statistical occurrence within a specialized subgroup, united by same or similar characteristics that are relevant to the research problem being investigated, r ather than studying statistical occurrence within the general population. Using a qualitative framework, cohort studies generally gather data using methods of observation. Cohorts can be either "open" or "closed."

• Open Cohort Studies [dynamic populations, such as the population of Los Angeles] involve a population that is defined just by the state of being a part of the study in question (and being monitored for the outcome). Date of entry and exit from the study is individually defined, therefore, the size of the study population is not constant. In open cohort studies, researchers can only calculate rate based data, such as, incidence rates and variants thereof.
• Closed Cohort Studies [static populations, such as patients entered into a clinical trial] involve participants who enter into the study at one defining point in time and where it is presumed that no new participants can enter the cohort. Given this, the number of study participants remains constant (or can only decrease).
• The use of cohorts is often mandatory because a randomized control study may be unethical. For example, you cannot deliberately expose people to asbestos, you can only study its effects on those who have already been exposed. Research that measures risk factors  often relies on cohort designs.
• Because cohort studies measure potential causes before the outcome has occurred, they can demonstrate that these “causes” preceded the outcome, thereby avoiding the debate as to which is the cause and which is the effect.
• Cohort analysis is highly flexible and can provide insight into effects over time and related to a variety of different types of changes [e.g., social, cultural, political, economic, etc.].
• Either original data or secondary data can be used in this design.
• In cases where a comparative analysis of two cohorts is made [e.g., studying the effects of one group exposed to asbestos and one that has not], a researcher cannot control for all other factors that might differ between the two groups. These factors are known as confounding variables.
• Cohort studies can end up taking a long time to complete if the researcher must wait for the conditions of interest to develop within the group. This also increases the chance that key variables change during the course of the study, potentially impacting the validity of the findings.
• Because of the lack of randominization in the cohort design, its external validity is lower than that of study designs where the researcher randomly assigns participants.

Healy P, Devane D. “Methodological Considerations in Cohort Study Designs.” Nurse Researcher 18 (2011): 32-36;  Levin, Kate Ann. Study Design IV: Cohort Studies. Evidence-Based Dentistry 7 (2003): 51–52; Study Design 101 . Himmelfarb Health Sciences Library. George Washington University, November 2011; Cohort Study . Wikipedia.

Cross-sectional research designs have three distinctive features: no time dimension, a reliance on existing differences rather than change following intervention; and, groups are selected based on existing differences rather than random allocation. The cross-sectional design can only measure diffrerences between or from among a variety of people, subjects, or phenomena rather than change. As such, researchers using this design can only employ a relative passive approach to making causal inferences based on findings.

• Cross-sectional studies provide a 'snapshot' of the outcome and the characteristics associated with it, at a specific point in time.
• Unlike the experimental design where there is an active intervention by the researcher to produce and measure change or to create differences, cross-sectional designs focus on studying and drawing inferences from existing differences between people, subjects, or phenomena.
• Entails collecting data at and concerning one point in time. While longitudinal studies involve taking multiple measures over an extended period of time, cross-sectional research is focused on finding relationships between variables at one moment in time.
• Groups identified for study are purposely selected based upon existing differences in the sample rather than seeking random sampling.
• Cross-section studies are capable of using data from a large number of subjects and, unlike observational studies, is not geographically bound.
• Can estimate prevalence of an outcome of interest because the sample is usually taken from the whole population.
• Because cross-sectional designs generally use survey techniques to gather data, they are relatively inexpensive and take up little time to conduct.
• Finding people, subjects, or phenomena to study that are very similar except in one specific variable can be difficult.
• Results are static and time bound and, therefore, give no indication of a sequence of events or reveal historical contexts.
• Studies cannot be utilized to establish cause and effect relationships.
• Provide only a snapshot of analysis so there is always the possibility that a study could have differing results if another time-frame had been chosen.
• There is no follow up to the findings.

Hall, John. “Cross-Sectional Survey Design.” In Encyclopedia of Survey Research Methods. Paul J. Lavrakas, ed. (Thousand Oaks, CA: Sage, 2008), pp. 173-174; Helen Barratt, Maria Kirwan. Cross-Sectional Studies: Design, Application, Strengths and Weaknesses of Cross-Sectional Studies . Healthknowledge, 2009. Cross-Sectional Study . Wikipedia.

Descriptive research designs help provide answers to the questions of who, what, when, where, and how associated with a particular research problem; a descriptive study cannot conclusively ascertain answers to why. Descriptive research is used to obtain information concerning the current status of the phenomena and to describe "what exists" with respect to variables or conditions in a situation.

• The subject is being observed in a completely natural and unchanged natural environment. True experiments, whilst giving analyzable data, often adversely influence the normal behavior of the subject.
• Descriptive research is often used as a pre-cursor to more quantitatively research designs, the general overview giving some valuable pointers as to what variables are worth testing quantitatively.
• If the limitations are understood, they can be a useful tool in developing a more focused study.
• Descriptive studies can yield rich data that lead to important recommendations.
• Appoach collects a large amount of data for detailed analysis.
• The results from a descriptive research can not be used to discover a definitive answer or to disprove a hypothesis.
• Because descriptive designs often utilize observational methods [as opposed to quantitative methods], the results cannot be replicated.
• The descriptive function of research is heavily dependent on instrumentation for measurement and observation.

Anastas, Jeane W. Research Design for Social Work and the Human Services . Chapter 5, Flexible Methods: Descriptive Research. 2nd ed. New York: Columbia University Press, 1999;  McNabb, Connie. Descriptive Research Methodologies . Powerpoint Presentation; Shuttleworth, Martyn. Descriptive Research Design , September 26, 2008. Explorable.com website.

A blueprint of the procedure that enables the researcher to maintain control over all factors that may affect the result of an experiment. In doing this, the researcher attempts to determine or predict what may occur. Experimental Research is often used where there is time priority in a causal relationship (cause precedes effect), there is consistency in a causal relationship (a cause will always lead to the same effect), and the magnitude of the correlation is great. The classic experimental design specifies an experimental group and a control group. The independent variable is administered to the experimental group and not to the control group, and both groups are measured on the same dependent variable. Subsequent experimental designs have used more groups and more measurements over longer periods. True experiments must have control, randomization, and manipulation.

• Experimental research allows the researcher to control the situation. In so doing, it allows researchers to answer the question, “what causes something to occur?”
• Permits the researcher to identify cause and effect relationships between variables and to distinguish placebo effects from treatment effects.
• Experimental research designs support the ability to limit alternative explanations and to infer direct causal relationships in the study.
• Approach provides the highest level of evidence for single studies.
• The design is artificial, and results may not generalize well to the real world.
• The artificial settings of experiments may alter subject behaviors or responses.
• Experimental designs can be costly if special equipment or facilities are needed.
• Some research problems cannot be studied using an experiment because of ethical or technical reasons.
• Difficult to apply ethnographic and other qualitative methods to  experimental designed research studies.

Anastas, Jeane W. Research Design for Social Work and the Human Services . Chapter 7, Flexible Methods: Experimental Research. 2nd ed. New York: Columbia University Press, 1999; Chapter 2: Research Design, Experimental Designs . School of Psychology, University of New England, 2000; Experimental Research. Research Methods by Dummies. Department of Psychology. California State University, Fresno, 2006; Trochim, William M.K. Experimental Design . Research Methods Knowledge Base. 2006; Rasool, Shafqat. Experimental Research . Slideshare presentation.

An exploratory design is conducted about a research problem when there are few or no earlier studies to refer to. The focus is on gaining insights and familiarity for later investigation or undertaken when problems are in a preliminary stage of investigation.

The goals of exploratory research are intended to produce the following possible insights:

• Familiarity with basic details, settings and concerns.
• Well grounded picture of the situation being developed.
• Generation of new ideas and assumption, development of tentative theories or hypotheses.
• Determination about whether a study is feasible in the future.
• Issues get refined for more systematic investigation and formulation of new research questions.
• Direction for future research and techniques get developed.
• Design is a useful approach for gaining background information on a particular topic.
• Exploratory research is flexible and can address research questions of all types (what, why, how).
• Provides an opportunity to define new terms and clarify existing concepts.
• Exploratory research is often used to generate formal hypotheses and develop more precise research problems.
• Exploratory studies help establish research priorities.
• Exploratory research generally utilizes small sample sizes and, thus, findings are typically not generalizable to the population at large.
• The exploratory nature of the research inhibits an ability to make definitive conclusions about the findings.
• The research process underpinning exploratory studies is flexible but often unstructured, leading to only tentative results that have limited value in decision-making.
• Design lacks rigorous standards applied to methods of data gathering and analysis because one of the areas for exploration could be to determine what method or methodologies could best fit the research problem.

Cuthill, Michael. “Exploratory Research: Citizen Participation, Local Government, and Sustainable Development in Australia.” Sustainable Development 10 (2002): 79-89; Taylor, P. J., G. Catalano, and D.R.F. Walker. “Exploratory Analysis of the World City Network.” Urban Studies 39 (December 2002): 2377-2394; Exploratory Research . Wikipedia.

The purpose of a historical research design is to collect, verify, and synthesize evidence from the past to establish facts that defend or refute your hypothesis. It uses secondary sources and a variety of primary documentary evidence, such as, logs, diaries, official records, reports, archives, and non-textual information [maps, pictures, audio and visual recordings]. The limitation is that the sources must be both authentic and valid.

• The historical research design is unobtrusive; the act of research does not affect the results of the study.
• The historical approach is well suited for trend analysis.
• Historical records can add important contextual background required to more fully understand and interpret a research problem.
• There is no possibility of researcher-subject interaction that could affect the findings.
• Historical sources can be used over and over to study different research problems or to replicate a previous study.
• The ability to fulfill the aims of your research are directly related to the amount and quality of documentation available to understand the research problem.
• Since historical research relies on data from the past, there is no way to manipulate it to control for contemporary contexts.
• Interpreting historical sources can be very time consuming.
• The sources of historical materials must be archived consistentally to ensure access.
• Original authors bring their own perspectives and biases to the interpretation of past events and these biases are more difficult to ascertain in historical resources.
• Due to the lack of control over external variables, historical research is very weak with regard to the demands of internal validity.
• It rare that the entirety of historical documentation needed to fully address a research problem is available for interpretation, therefore, gaps need to be acknowledged.

Savitt, Ronald. “Historical Research in Marketing.” Journal of Marketing 44 (Autumn, 1980): 52-58;  Gall, Meredith. Educational Research: An Introduction . Chapter 16, Historical Research. 8th ed. Boston, MA: Pearson/Allyn and Bacon, 2007.

A longitudinal study follows the same sample over time and makes repeated observations. With longitudinal surveys, for example, the same group of people is interviewed at regular intervals, enabling researchers to track changes over time and to relate them to variables that might explain why the changes occur. Longitudinal research designs describe patterns of change and help establish the direction and magnitude of causal relationships. Measurements are taken on each variable over two or more distinct time periods. This allows the researcher to measure change in variables over time. It is a type of observational study and is sometimes referred to as a panel study.

• Longitudinal data allow the analysis of duration of a particular phenomenon.
• Enables survey researchers to get close to the kinds of causal explanations usually attainable only with experiments.
• The design permits the measurement of differences or change in a variable from one period to another [i.e., the description of patterns of change over time].
• Longitudinal studies facilitate the prediction of future outcomes based upon earlier factors.
• The data collection method may change over time.
• Maintaining the integrity of the original sample can be difficult over an extended period of time.
• It can be difficult to show more than one variable at a time.
• This design often needs qualitative research to explain fluctuations in the data.
• A longitudinal research design assumes present trends will continue unchanged.
• It can take a long period of time to gather results.
• There is a need to have a large sample size and accurate sampling to reach representativness.

Anastas, Jeane W. Research Design for Social Work and the Human Services . Chapter 6, Flexible Methods: Relational and Longitudinal Research. 2nd ed. New York: Columbia University Press, 1999; Kalaian, Sema A. and Rafa M. Kasim. "Longitudinal Studies." In Encyclopedia of Survey Research Methods . Paul J. Lavrakas, ed. (Thousand Oaks, CA: Sage, 2008), pp. 440-441; Ployhart, Robert E. and Robert J. Vandenberg. "Longitudinal Research: The Theory, Design, and Analysis of Change.” Journal of Management 36 (January 2010): 94-120; Longitudinal Study . Wikipedia.

This type of research design draws a conclusion by comparing subjects against a control group, in cases where the researcher has no control over the experiment. There are two general types of observational designs. In direct observations, people know that you are watching them. Unobtrusive measures involve any method for studying behavior where individuals do not know they are being observed. An observational study allows a useful insight into a phenomenon and avoids the ethical and practical difficulties of setting up a large and cumbersome research project.

• Observational studies are usually flexible and do not necessarily need to be structured around a hypothesis about what you expect to observe (data is emergent rather than pre-existing).
• The researcher is able to collect a depth of information about a particular behavior.
• Can reveal interrelationships among multifaceted dimensions of group interactions.
• You can generalize your results to real life situations.
• Observational research is useful for discovering what variables may be important before applying other methods like experiments.
• Observation researchd esigns account for the complexity of group behaviors.
• Reliability of data is low because seeing behaviors occur over and over again may be a time consuming task and difficult to replicate.
• In observational research, findings may only reflect a unique sample population and, thus, cannot be generalized to other groups.
• There can be problems with bias as the researcher may only "see what they want to see."
• There is no possiblility to determine "cause and effect" relationships since nothing is manipulated.
• Sources or subjects may not all be equally credible.
• Any group that is studied is altered to some degree by the very presence of the researcher, therefore, skewing to some degree any data collected (the Heisenburg Uncertainty Principle).

Atkinson, Paul and Martyn Hammersley. “Ethnography and Participant Observation.” In Handbook of Qualitative Research . Norman K. Denzin and Yvonna S. Lincoln, eds. (Thousand Oaks, CA: Sage, 1994), pp. 248-261; Observational Research. Research Methods by Dummies. Department of Psychology. California State University, Fresno, 2006; Patton Michael Quinn. Qualitiative Research and Evaluation Methods . Chapter 6, Fieldwork Strategies and Observational Methods. 3rd ed. Thousand Oaks, CA: Sage, 2002; Rosenbaum, Paul R. Design of Observational Studies . New York: Springer, 2010.

Understood more as an broad approach to examining a research problem than a methodological design, philosophical analysis and argumentation is intended to challenge deeply embedded, often intractable, assumptions underpinning an area of study. This approach uses the tools of argumentation derived from philosophical traditions, concepts, models, and theories to critically explore and challenge, for example, the relevance of logic and evidence in academic debates, to analyze arguments about fundamental issues, or to discuss the root of existing discourse about a research problem. These overarching tools of analysis can be framed in three ways:

• Ontology -- the study that describes the nature of reality; for example, what is real and what is not, what is fundamental and what is derivative?
• Epistemology -- the study that explores the nature of knowledge; for example, on what does knowledge and understanding depend upon and how can we be certain of what we know?
• Axiology -- the study of values; for example, what values does an individual or group hold and why? How are values related to interest, desire, will, experience, and means-to-end? And, what is the difference between a matter of fact and a matter of value?
• Can provide a basis for applying ethical decision-making to practice.
• Functions as a means of gaining greater self-understanding and self-knowledge about the purposes of research.
• Brings clarity to general guiding practices and principles of an individual or group.
• Philosophy informs methodology.
• Refine concepts and theories that are invoked in relatively unreflective modes of thought and discourse.
• Beyond methodology, philosophy also informs critical thinking about epistemology and the structure of reality (metaphysics).
• Offers clarity and definition to the practical and theoretical uses of terms, concepts, and ideas.
• Limited application to specific research problems [answering the "So What?" question in social science research].
• Analysis can be abstract, argumentative, and limited in its practical application to real-life issues.
• While a philosophical analysis may render problematic that which was once simple or taken-for-granted, the writing can be dense and subject to unnecessary jargon, overstatement, and/or excessive quotation and documentation.
• There are limitations in the use of metaphor as a vehicle of philosophical analysis.
• There can be analytical difficulties in moving from philosophy to advocacy and between abstract thought and application to the phenomenal world.

Chapter 4, Research Methodology and Design . Unisa Institutional Repository (UnisaIR), University of South Africa;  Labaree, Robert V. and Ross Scimeca. “The Philosophical Problem of Truth in Librarianship.” The Library Quarterly 78 (January 2008): 43-70; Maykut, Pamela S. Beginning Qualitative Research: A Philosophic and Practical Guide . Washington, D.C.: Falmer Press, 1994; Stanford Encyclopedia of Philosophy . Metaphysics Research Lab, CSLI, Stanford University, 2013.

• The researcher has a limitless option when it comes to sample size and the sampling schedule.
• Due to the repetitive nature of this research design, minor changes and adjustments can be done during the initial parts of the study to correct and hone the research method. Useful design for exploratory studies.
• There is very little effort on the part of the researcher when performing this technique. It is generally not expensive, time consuming, or workforce extensive.
• Because the study is conducted serially, the results of one sample are known before the next sample is taken and analyzed.
• The sampling method is not representative of the entire population. The only possibility of approaching representativeness is when the researcher chooses to use a very large sample size significant enough to represent a significant portion of the entire population. In this case, moving on to study a second or more sample can be difficult.
• Because the sampling technique is not randomized, the design cannot be used to create conclusions and interpretations that pertain to an entire population. Generalizability from findings is limited.
• Difficult to account for and interpret variation from one sample to another over time, particularly when using qualitative methods of data collection.

Rebecca Betensky, Harvard University, Course Lecture Note slides ; Cresswell, John W. Et al. “Advanced Mixed-Methods Research Designs.” In Handbook of Mixed Methods in Social and Behavioral Research . Abbas Tashakkori and Charles Teddle, eds. (Thousand Oaks, CA: Sage, 2003), pp. 209-240; Nataliya V. Ivankova. “Using Mixed-Methods Sequential Explanatory Design: From Theory to Practice.” Field Methods 18 (February 2006): 3-20; Bovaird, James A. and Kevin A. Kupzyk. “Sequential Design.” In Encyclopedia of Research Design . Neil J. Salkind, ed. Thousand Oaks, CA: Sage, 2010; Sequential Analysis . Wikipedia.  

• << Previous: Purpose of Guide
• Next: Design Flaws to Avoid >>
• Last Updated: Jul 18, 2023 11:58 AM
• URL: https://library.sacredheart.edu/c.php?g=29803
• QuickSearch
• Library Catalog
• Databases A-Z
• Publication Finder
• Course Reserves
• Citation Linker
• Digital Commons
• Our Website

Research Support

• Ask a Librarian
• Appointments
• Interlibrary Loan (ILL)
• Research Guides
• Databases by Subject
• Citation Help

Using the Library

• Reserve a Group Study Room
• Renew Books
• Honors Study Rooms
• Off-Campus Access
• Library Policies
• Library Technology

User Information

• Grad Students
• Online Students
• COVID-19 Updates
• Staff Directory
• News & Announcements
• Library Newsletter

My Accounts

• Interlibrary Loan
• Staff Site Login

FIND US ON  

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Perspect Clin Res
• v.9(4); Oct-Dec 2018

Study designs: Part 1 – An overview and classification

Priya ranganathan.

Department of Anaesthesiology, Tata Memorial Centre, Mumbai, Maharashtra, India

Rakesh Aggarwal

1 Department of Gastroenterology, Sanjay Gandhi Postgraduate Institute of Medical Sciences, Lucknow, Uttar Pradesh, India

There are several types of research study designs, each with its inherent strengths and flaws. The study design used to answer a particular research question depends on the nature of the question and the availability of resources. In this article, which is the first part of a series on “study designs,” we provide an overview of research study designs and their classification. The subsequent articles will focus on individual designs.

INTRODUCTION

Research study design is a framework, or the set of methods and procedures used to collect and analyze data on variables specified in a particular research problem.

Research study designs are of many types, each with its advantages and limitations. The type of study design used to answer a particular research question is determined by the nature of question, the goal of research, and the availability of resources. Since the design of a study can affect the validity of its results, it is important to understand the different types of study designs and their strengths and limitations.

There are some terms that are used frequently while classifying study designs which are described in the following sections.

A variable represents a measurable attribute that varies across study units, for example, individual participants in a study, or at times even when measured in an individual person over time. Some examples of variables include age, sex, weight, height, health status, alive/dead, diseased/healthy, annual income, smoking yes/no, and treated/untreated.

Exposure (or intervention) and outcome variables

A large proportion of research studies assess the relationship between two variables. Here, the question is whether one variable is associated with or responsible for change in the value of the other variable. Exposure (or intervention) refers to the risk factor whose effect is being studied. It is also referred to as the independent or the predictor variable. The outcome (or predicted or dependent) variable develops as a consequence of the exposure (or intervention). Typically, the term “exposure” is used when the “causative” variable is naturally determined (as in observational studies – examples include age, sex, smoking, and educational status), and the term “intervention” is preferred where the researcher assigns some or all participants to receive a particular treatment for the purpose of the study (experimental studies – e.g., administration of a drug). If a drug had been started in some individuals but not in the others, before the study started, this counts as exposure, and not as intervention – since the drug was not started specifically for the study.

Observational versus interventional (or experimental) studies

Observational studies are those where the researcher is documenting a naturally occurring relationship between the exposure and the outcome that he/she is studying. The researcher does not do any active intervention in any individual, and the exposure has already been decided naturally or by some other factor. For example, looking at the incidence of lung cancer in smokers versus nonsmokers, or comparing the antenatal dietary habits of mothers with normal and low-birth babies. In these studies, the investigator did not play any role in determining the smoking or dietary habit in individuals.

For an exposure to determine the outcome, it must precede the latter. Any variable that occurs simultaneously with or following the outcome cannot be causative, and hence is not considered as an “exposure.”

Observational studies can be either descriptive (nonanalytical) or analytical (inferential) – this is discussed later in this article.

Interventional studies are experiments where the researcher actively performs an intervention in some or all members of a group of participants. This intervention could take many forms – for example, administration of a drug or vaccine, performance of a diagnostic or therapeutic procedure, and introduction of an educational tool. For example, a study could randomly assign persons to receive aspirin or placebo for a specific duration and assess the effect on the risk of developing cerebrovascular events.

Descriptive versus analytical studies

Descriptive (or nonanalytical) studies, as the name suggests, merely try to describe the data on one or more characteristics of a group of individuals. These do not try to answer questions or establish relationships between variables. Examples of descriptive studies include case reports, case series, and cross-sectional surveys (please note that cross-sectional surveys may be analytical studies as well – this will be discussed in the next article in this series). Examples of descriptive studies include a survey of dietary habits among pregnant women or a case series of patients with an unusual reaction to a drug.

Analytical studies attempt to test a hypothesis and establish causal relationships between variables. In these studies, the researcher assesses the effect of an exposure (or intervention) on an outcome. As described earlier, analytical studies can be observational (if the exposure is naturally determined) or interventional (if the researcher actively administers the intervention).

Directionality of study designs

Based on the direction of inquiry, study designs may be classified as forward-direction or backward-direction. In forward-direction studies, the researcher starts with determining the exposure to a risk factor and then assesses whether the outcome occurs at a future time point. This design is known as a cohort study. For example, a researcher can follow a group of smokers and a group of nonsmokers to determine the incidence of lung cancer in each. In backward-direction studies, the researcher begins by determining whether the outcome is present (cases vs. noncases [also called controls]) and then traces the presence of prior exposure to a risk factor. These are known as case–control studies. For example, a researcher identifies a group of normal-weight babies and a group of low-birth weight babies and then asks the mothers about their dietary habits during the index pregnancy.

Prospective versus retrospective study designs

The terms “prospective” and “retrospective” refer to the timing of the research in relation to the development of the outcome. In retrospective studies, the outcome of interest has already occurred (or not occurred – e.g., in controls) in each individual by the time s/he is enrolled, and the data are collected either from records or by asking participants to recall exposures. There is no follow-up of participants. By contrast, in prospective studies, the outcome (and sometimes even the exposure or intervention) has not occurred when the study starts and participants are followed up over a period of time to determine the occurrence of outcomes. Typically, most cohort studies are prospective studies (though there may be retrospective cohorts), whereas case–control studies are retrospective studies. An interventional study has to be, by definition, a prospective study since the investigator determines the exposure for each study participant and then follows them to observe outcomes.

The terms “prospective” versus “retrospective” studies can be confusing. Let us think of an investigator who starts a case–control study. To him/her, the process of enrolling cases and controls over a period of several months appears prospective. Hence, the use of these terms is best avoided. Or, at the very least, one must be clear that the terms relate to work flow for each individual study participant, and not to the study as a whole.

Classification of study designs

Figure 1 depicts a simple classification of research study designs. The Centre for Evidence-based Medicine has put forward a useful three-point algorithm which can help determine the design of a research study from its methods section:[ 1 ]

Classification of research study designs

• Does the study describe the characteristics of a sample or does it attempt to analyze (or draw inferences about) the relationship between two variables? – If no, then it is a descriptive study, and if yes, it is an analytical (inferential) study
• If analytical, did the investigator determine the exposure? – If no, it is an observational study, and if yes, it is an experimental study
• If observational, when was the outcome determined? – at the start of the study (case–control study), at the end of a period of follow-up (cohort study), or simultaneously (cross sectional).

In the next few pieces in the series, we will discuss various study designs in greater detail.

Financial support and sponsorship

Conflicts of interest.

There are no conflicts of interest.

Baltimore bridge collapse: a bridge engineer explains what happened, and what needs to change

Associate Professor, Civil Engineering, Monash University

Disclosure statement

Colin Caprani receives funding from the Department of Transport (Victoria) and the Level Crossing Removal Project. He is also Chair of the Confidential Reporting Scheme for Safer Structures - Australasia, Chair of the Australian Regional Group of the Institution of Structural Engineers, and Australian National Delegate for the International Association for Bridge and Structural Engineering.

Monash University provides funding as a founding partner of The Conversation AU.

View all partners

When the container ship MV Dali, 300 metres long and massing around 100,000 tonnes, lost power and slammed into one of the support piers of the Francis Scott Key Bridge in Baltimore, the bridge collapsed in moments . Six people are presumed dead, several others injured, and the city and region are expecting a months-long logistical nightmare in the absence of a crucial transport link.

It was a shocking event, not only for the public but for bridge engineers like me. We work very hard to ensure bridges are safe, and overall the probability of being injured or worse in a bridge collapse remains even lower than the chance of being struck by lightning.

However, the images from Baltimore are a reminder that safety can’t be taken for granted. We need to remain vigilant.

So why did this bridge collapse? And, just as importantly, how might we make other bridges more safe against such collapse?

A 20th century bridge meets a 21st century ship

The Francis Scott Key Bridge was built through the mid 1970s and opened in 1977. The main structure over the navigation channel is a “continuous truss bridge” in three sections or spans.

The bridge rests on four supports, two of which sit each side of the navigable waterway. It is these two piers that are critical to protect against ship impacts.

And indeed, there were two layers of protection: a so-called “dolphin” structure made from concrete, and a fender. The dolphins are in the water about 100 metres upstream and downstream of the piers. They are intended to be sacrificed in the event of a wayward ship, absorbing its energy and being deformed in the process but keeping the ship from hitting the bridge itself.

The fender is the last layer of protection. It is a structure made of timber and reinforced concrete placed around the main piers. Again, it is intended to absorb the energy of any impact.

Fenders are not intended to absorb impacts from very large vessels . And so when the MV Dali, weighing more than 100,000 tonnes, made it past the protective dolphins, it was simply far too massive for the fender to withstand.

Read more: I've captained ships into tight ports like Baltimore, and this is how captains like me work with harbor pilots to avoid deadly collisions

Video recordings show a cloud of dust appearing just before the bridge collapsed, which may well have been the fender disintegrating as it was crushed by the ship.

Once the massive ship had made it past both the dolphin and the fender, the pier – one of the bridge’s four main supports – was simply incapable of resisting the impact. Given the size of the vessel and its likely speed of around 8 knots (15 kilometres per hour), the impact force would have been around 20,000 tonnes .

Bridges are getting safer

This was not the first time a ship hit the Francis Scott Bridge. There was another collision in 1980 , damaging a fender badly enough that it had to be replaced.

Around the world, 35 major bridge collapses resulting in fatalities were caused by collisions between 1960 and 2015, according to a 2018 report from the World Association for Waterborne Transport Infrastructure. Collisions between ships and bridges in the 1970s and early 1980s led to a significant improvement in the design rules for protecting bridges from impact.

Further impacts in the 1970s and early 1980s instigated significant improvements in the design rules for impact.

The International Association for Bridge and Structural Engineering’s Ship Collision with Bridges guide, published in 1993, and the American Association of State Highway and Transporation Officials’ Guide Specification and Commentary for Vessel Collision Design of Highway Bridges (1991) changed how bridges were designed.

In Australia, the Australian Standard for Bridge Design (published in 2017) requires designers to think about the biggest vessel likely to come along in the next 100 years, and what would happen if it were heading for any bridge pier at full speed. Designers need to consider the result of both head-on collisions and side-on, glancing blows. As a result, many newer bridges protect their piers with entire human-made islands.

Of course, these improvements came too late to influence the design of the Francis Scott Key Bridge itself.

Lessons from disaster

So what are the lessons apparent at this early stage?

First, it’s clear the protection measures in place for this bridge were not enough to handle this ship impact. Today’s cargo ships are much bigger than those of the 1970s, and it seems likely the Francis Scott Key Bridge was not designed with a collision like this in mind.

So one lesson is that we need to consider how the vessels near our bridges are changing. This means we cannot just accept the structure as it was built, but ensure the protection measures around our bridges are evolving alongside the ships around them.

Second, and more generally, we must remain vigilant in managing our bridges. I’ve written previously about the current level of safety of Australian bridges, but also about how we can do better.

This tragic event only emphasises the need to spend more on maintaining our ageing infrastructure. This is the only way to ensure it remains safe and functional for the demands we put on it today.

• Engineering
• Infrastructure
• Urban infrastructure
• container ships
• Baltimore bridge collapse

School of Social Sciences – Public Policy and International Relations opportunities

School of Social Sciences – Human Geography opportunities

School of Social Sciences – Criminology opportunities

School of Social Sciences – Academic appointment opportunities

Audience Development Coordinator (fixed-term maternity cover)

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 28 March 2024

New water accounting reveals why the Colorado River no longer reaches the sea

• Brian D. Richter   ORCID: orcid.org/0000-0001-7216-1397 1 , 2 ,
• Gambhir Lamsal   ORCID: orcid.org/0000-0002-2593-8949 3 ,
• Landon Marston   ORCID: orcid.org/0000-0001-9116-1691 3 ,
• Sameer Dhakal   ORCID: orcid.org/0000-0003-4941-1559 3 ,
• Laljeet Singh Sangha   ORCID: orcid.org/0000-0002-0986-1785 4 ,
• Richard R. Rushforth 4 ,
• Dongyang Wei   ORCID: orcid.org/0000-0003-0384-4340 5 ,
• Benjamin L. Ruddell 4 ,
• Kyle Frankel Davis   ORCID: orcid.org/0000-0003-4504-1407 5 , 6 ,
• Astrid Hernandez-Cruz   ORCID: orcid.org/0000-0003-0776-5105 7 ,
• Samuel Sandoval-Solis 8 &
• John C. Schmidt 9  

Communications Earth & Environment volume  5 , Article number:  134 ( 2024 ) Cite this article

7927 Accesses

699 Altmetric

Metrics details

• Water resources

Persistent overuse of water supplies from the Colorado River during recent decades has substantially depleted large storage reservoirs and triggered mandatory cutbacks in water use. The river holds critical importance to more than 40 million people and more than two million hectares of cropland. Therefore, a full accounting of where the river’s water goes en route to its delta is necessary. Detailed knowledge of how and where the river’s water is used can aid design of strategies and plans for bringing water use into balance with available supplies. Here we apply authoritative primary data sources and modeled crop and riparian/wetland evapotranspiration estimates to compile a water budget based on average consumptive water use during 2000–2019. Overall water consumption includes both direct human uses in the municipal, commercial, industrial, and agricultural sectors, as well as indirect water losses to reservoir evaporation and water consumed through riparian/wetland evapotranspiration. Irrigated agriculture is responsible for 74% of direct human uses and 52% of overall water consumption. Water consumed for agriculture amounts to three times all other direct uses combined. Cattle feed crops including alfalfa and other grass hays account for 46% of all direct water consumption.

Similar content being viewed by others

Disappearing cities on US coasts

Leonard O. Ohenhen, Manoochehr Shirzaei, … Robert J. Nicholls

Meta-analysis shows the impacts of ecological restoration on greenhouse gas emissions

Tiehu He, Weixin Ding, … Quanfa Zhang

Irrigation-driven groundwater depletion in the Ganges-Brahmaputra basin decreases the streamflow in the Bay of Bengal

Fadji Z. Maina, Augusto Getirana, … Ravi Appana

Introduction

Barely a trickle of water is left of the iconic Colorado River of the American Southwest as it approaches its outlet in the Gulf of California in Mexico after watering many cities and farms along its 2330-kilometer course. There were a few years in the 1980s in which enormous snowfall in the Rocky Mountains produced a deluge of spring snowmelt runoff capable of escaping full capture for human uses, but for most of the past 60 years the river’s water has been fully consumed before reaching its delta 1 , 2 . In fact, the river was overconsumed (i.e., total annual water consumption exceeding runoff supplies) in 16 of 21 years during 2000–2020 3 , requiring large withdrawals of water stored in Lake Mead and Lake Powell to accommodate the deficits. An average annual overdraft of 10% during this period 2 caused these reservoirs– the two largest in the US – to drop to three-quarters empty by the end of 2022 4 , triggering urgent policy decisions on where to cut consumption.

Despite the river’s importance to more than 40 million people and more than two million hectares (>5 million acres) of cropland—producing most of the vegetable produce for American and Canadian plates in wintertime and also feeding many additional people worldwide via exports—a full sectoral and crop-specific accounting of where all that water goes en route to its delta has never been attempted, until now. Detailed knowledge of how and where the river’s water is used can aid design of strategies and plans for bringing water use into balance with available supplies.

There are interesting historical reasons to explain why this full water budget accounting has not been accomplished previously, beginning a full century ago when the apportionment of rights to use the river’s water within the United States was inscribed into the Colorado River Compact of 1922 5 . That Compact was ambiguous and confusing in its allocation of water inflowing to the Colorado River from the Gila River basin in New Mexico and Arizona 6 , even though it accounts for 24% of the drainage area of the Colorado River Basin (Fig.  1 ). Because of intense disagreements over the rights to the Gila and other tributaries entering the Colorado River downstream of the Grand Canyon, the Compact negotiators decided to leave the allocation of those waters rights to a later time so that the Compact could proceed 6 . Arizona’s formal rights to the Gila and other Arizona tributaries were finally affirmed in a US Supreme Court decision in 1963 that also specified the volumes of Colorado River water allocated to California, Arizona, and Nevada 7 . Because the rights to the Gila’s waters lie outside of the Compact allocations, the Gila has not been included in formal accounting of the Colorado River Basin water budget to date 8 . Additionally, the Compact did not specify how much water Mexico—at the river’s downstream end—should receive. Mexico’s share of the river was not formalized until 22 years later, in the 1944 international treaty on “Utilization of the Waters of the Colorado and Tijuana Rivers and of the Rio Grande” (1944 Water Treaty) 9 . As a result of these political circumstances, full accounting for direct water consumption at the sectoral level—in which water use is accounted according to categories such as municipal, industrial, commercial, or agricultural uses—has not previously been compiled for the Gila River basin’s water, and sectoral accounting for Mexico was not published until 2023 10 .

The physical boundary of the Colorado River Basin is outlined in black. Hatched areas outside of the basin boundary receive Colorado River water via inter-basin transfers (also known as ‘exports’). The Gila River basin is situated in the far southern portion of the CRB in Arizona, New Mexico, and Mexico. Map courtesy of Center for Colorado River Studies, Utah State University.

The US Bureau of Reclamation (“Reclamation”)—which owns and operates massive water infrastructure in the Colorado River Basin—has served as the primary accountant of Colorado River water. In 2012, the agency produced a “Colorado River Basin Water Supply and Demand Study” 8 that accounted for both the sectoral uses of water within the basin’s physical boundaries within the US as well as river water exported outside of the basin (Fig.  1 ). But Reclamation did not attempt to account for water generated from the Gila River basin because of that sub-basin’s exclusion from the Colorado River Compact, and it did not attempt to explain how water crossing the border into Mexico is used. The agency estimated riparian vegetation evapotranspiration for the lower Colorado River but not the remainder of the extensive river system. Richter et al. 11 published a water budget for the Colorado River that included sectoral and crop-specific water consumption but it too did not include water used in Mexico, nor reservoir evaporation or riparian evapotranspiration, and it did not account for water exported outside of the Colorado River Basin’s physical boundary as illustrated in Fig.  1 . Given that nearly one-fifth (19%) of the river’s water is exported from the basin or used in Mexico, and that the Gila is a major tributary to the Colorado, this incomplete accounting has led to inaccuracies and misinterpretations of “where the Colorado River’s water goes” and has created uncertainty in discussions based on the numbers. This paper provides fuller accounting of the fate of all river water during 2000–2019, including averaged annual consumption in each of the sub-basins including exports, consumption in major sectors of the economy, consumption in the production of specific types of crops, and water consumed by reservoir evaporation and riparian/wetland evapotranspiration.

Rising awareness of water overuse and prolonged drought has driven intensifying dialog among the seven US states sharing the basin’s waters as well as between the United States, Mexico, and 30 tribal nations within the US. Since 2000, six legal agreements affecting the US states and two international agreements with Mexico have had the effect of reducing water use from the Colorado River 7 :

In 2001, the US Secretary of the Interior issued a set of “Interim Surplus Guidelines” to reduce California’s water use by 14% to bring the state within its allocation as determined in the 1963 US Supreme Court case mentioned previously. A subsequent “Quantification Settlement Agreement” executed in 2003 spelled out details about how California was going to achieve the targeted reduction.

In 2007, the US Secretary of the Interior adopted a set of “Colorado River Interim Guidelines for Lower Basin Shortages and the Coordinated Operations for Lake Powell and Lake Mead” that reduced water deliveries to Arizona and Nevada when Lake Mead drops to specified levels, with increasing cutbacks as levels decline.

In 2012, the US and Mexican federal governments signed an addendum to the 1944 Water Treaty known as Minute 319 that reduced deliveries to Mexico as Lake Mead elevations fall.

In 2017, the US and Mexican federal governments established a “Binational Water Scarcity Contingency Plan” as part of Minute 323 that provides for deeper cuts in deliveries to Mexico under specified low reservoir elevations in Lake Mead.i

In 2019, the three Lower Basin states and the US Secretary of the Interior agreed to commitments under the “Lower Basin Drought Contingency Plan” that further reduced water deliveries beyond the levels set in 2007 and added specifications for deeper cuts as Lake Mead drops to levels lower than anticipated in the 2007 Guidelines.

In 2023, the states of California, Arizona and Nevada committed to further reductions in water use through the year 2026 12 .

With each of the above agreements, overall water consumption has been reduced but many scientists assert that these reductions still fall substantially short of balancing consumptive use with 21st century water supplies 2 , 13 . With all of these agreements—excepting the Interim Surplus Guidelines of 2001—set to expire in 2026, management of the Colorado River’s binational water supply is now at a crucial point, emphasizing the need for comprehensive water budget accounting.

Our tabulation of the Colorado River’s full water consumption budget (Table  1 ) provides accounting for all direct human uses of water as either agricultural or MCI (municipal, commercial, industrial), as well as indirect losses of water to reservoir evaporation and evapotranspiration from riparian or wetland vegetation including in the Salton Sea and in a wetland in Mexico (Cienega de Santa Clara) that receives agricultural return flows from irrigated areas in Arizona. We explicitly note that all estimates represent consumptive use , resulting from the subtraction of return flows from total water withdrawals. Table  2 provides a summary based only on direct human uses and does not include indirect consumption of water. We have provided Tables  1 and 2 in English units in our Supplementary Information as Tables SI-1 and SI-2 . We have lumped municipal, commercial, and industrial (MCI) uses together because these sub-categories of consumption are not consistently differentiated within official water delivery data for cities utilizing Colorado River water. More detail on urban water use by cities dependent on the river is available in Richter 14 , among other studies.

We differentiated water consumption geographically using the ‘accounting units’ mapped in Fig.  2 , which are based on the Colorado River Basin map as revised by Schmidt 15 ; importantly, these accounting units align spatially with Reclamation’s accounting systems for the Upper Basin and Lower Basin as described in our Methods, thereby enabling readers accustomed to Reclamation’s water-use reports to easily comprehend our accounting. We have also accounted for all water consumed within the Colorado River Basin boundaries as well as water exported via inter-basin transfers. Water exported outside of the basin includes 47 individual inter-basin transfer systems (i.e., canals, pipelines, pumps) that in aggregate export ~12% of the river’s water. We note that the Imperial Irrigation District of southern California is often counted as a recipient of exported water, but we have followed the rationale of Schmidt 15 by including it as an interior part of the Lower Basin even though it receives its Colorado River water via the All American Canal (Fig.  2 ).

The water budget estimates presented in Tables  1 and 2 are summarized for each of the seven “accounting units” displayed here.

These results confirm previous findings that irrigated agriculture is the dominant consumer of Colorado River water. Irrigated agriculture accounts for 52% of overall consumption (Table  1 ; Figs.  3 and 4 ) and 74% of direct human consumption (Table  2 ) of water from the Colorado River Basin. As highlighted in Richter et al. 11 , cattle-feed crops (alfalfa and other hay) are the dominant water-consuming crops dependent upon irrigation water from the basin (Tables  1 and 2 ; Figs.  3 and 4 ). Those crops account for 32% of all water consumed from the basin, 46% of all direct water consumption, and 62% of all agricultural water consumed (Table  1 ; Fig.  3 ). The percentage of water consumed by irrigated crops is greatest in Mexico, where they account for 86% of all direct human uses (Table  2 ) and 80% of total water consumed (Table  1 ). Cattle-feed crops consume 90% of all water used by irrigated agriculture within the Upper Basin, where the consumed volume associated with these cattle-feed crops amounts to more than three times what is consumed for municipal, commercial, or industrial uses combined.

All estimates based on 2000–2019 averages. Both agriculture and MCI (municipal, commercial, and industrial) uses are herein referred to as “direct human uses.” “Indirect uses” include both reservoir evaporation as well as evapotranspiration by riparian/wetland vegetation.

Water consumed by each sector in the Colorado River Basin and sub-basins (including exports), based on 2000–2019 averages.

Another important finding is that a substantial volume of water (19%) is consumed in supporting the natural environment through riparian and wetland vegetation evapotranspiration along river courses. This analysis—made possible because of recent mapping of riparian vegetation in the Colorado River Basin 16 —is an important addition to the water budget of the Colorado River Basin, given that the only previous accounting for riparian vegetation consumption has limited to the mainstem of the Colorado River below Hoover Dam and does not include vegetation upstream of Hoover Dam nor vegetation along tributary rivers 17 . Given that many of these habitats and associated species have been lost or became imperiled due to river flow depletion 18 —including the river’s vast delta ecosystem in Mexico—an ecologically sustainable approach to water management would need to allow more water to remain in the river system to support riparian and aquatic ecosystems. Additionally, 11% of all water consumed in the Colorado River Basin is lost through evaporation from reservoirs.

It is also important to note a fairly high degree of inter-annual variability in each sector of water use; for example, the range of values portrayed for the four water budget sectors shown in Fig.  5 equates to 24–47% of their 20-year averages. Also notable is a decrease in water consumed in the Lower Basin between the years 2000 and 2019 for both the MCI (−38%) and agricultural sectors (−15%), which can in part be attributed to the policy agreements summarized previously that have mandated water-use reductions.

Inter-annual variability of water consumption within the Lower and Upper Basins, including water exported from these basins. The average (AVG) values shown are used in the water budgets detailed in Tables  1 and 2 .

The water accounting in Richter et al. 11 received a great deal of media attention including a front-page story in the New York Times 19 . These stories focused primarily on our conclusion that more than half (53%) of water consumed in the Colorado River Basin was attributable to cattle-feed crops (alfalfa and other hays) supporting beef and dairy production. However, that tabulation of the river’s water budget had notable shortcomings, as discussed previously. In this more complete accounting that includes Colorado River water exported outside of the basin’s physical boundary as well as indirect water consumption, we find that irrigated agriculture consumes half (52%) of all Colorado River Basin water, and the portion of direct consumption going to cattle-feed crops dropped from 53% as reported in Richter et al. 11 to 46% in this revised analysis.

These differences are explained by the fact that we now account for all exported water and also include indirect losses of water to reservoir evaporation and riparian/wetland evapotranspiration in our revised accounting, as well as improvements in our estimation of crop-water consumption. However, the punch line of our 2020 paper does not change fundamentally. Irrigated agriculture is the dominant consumer of water from the Colorado River, and 62% of agricultural water consumption goes to alfalfa and grass hay production.

Richter et al. 20 found that alfalfa and grass hay were the largest water consumers in 57% of all sub-basins across the western US, and their production is increasing in many western regions. Alfalfa is favored for its ability to tolerate variable climate conditions, especially its ability to persist under greatly reduced irrigation during droughts and its ability to recover production quickly after full irrigation is resumed, acting as a “shock absorber” for agricultural production under unpredictable drought conditions. The plant is also valued for fixing nitrogen in soils, reducing fertilizer costs. Perhaps most importantly, labor costs are comparatively low because alfalfa is mechanically harvested. Alfalfa is increasing in demand and price as a feed crop in the growing dairy industry of the region 21 . Any efforts to reduce water consumed by alfalfa—either through shifting to alternative lower-water crops or through compensated fallowing 20 —will need to compete with these attributes.

This new accounting provides a more comprehensive and complete understanding of how the Colorado River Basin’s water is consumed. During our study period of 2000–2019, an estimated average of 23.7 billion cubic meters (19.3 million acre-feet) of water was consumed each year before reaching its now-dry delta in Mexico. Schmidt et al. 2 have estimated that a reduction in consumptive use in the Upper and Lower Basins of 3–4 billion cubic meters (2.4–3.2 million acre-feet) per year—equivalent to 22–29% of direct use in those basins—will be necessary to stabilize reservoir levels, and an additional reduction of 1–3 billion cubic meters (~811,000–2.4 million acre-feet) per year will likely be needed by 2050 as climate warming continues to reduce runoff in the Colorado River Basin.

We hope that this new accounting will add clarity and a useful informational foundation to the public dialog and political negotiations over Colorado River Basin water allocations and cutbacks that are presently underway 2 . Because a persistent drought and intensifying aridification in the region has placed both people and river ecosystems in danger of water shortages in recent decades, knowledge of where the water goes will be essential in the design of policies for bringing the basin into a sustainable water supply-demand balance.

The data sources and analytical approaches used in this study are summarized below. Unless otherwise noted, all data were assembled for each year from 2000–2019 and then averaged. We acknowledge some inconsistency in the manner in which water consumption is measured or estimated across the various data sources and sectors used in this study, as discussed below, and each of these different approaches entail some degree of inaccuracy or uncertainty. We also note that technical measurement or estimation approaches change over time, and new approaches can yield differing results. For instance, the Upper Colorado River Commission is exploring new approaches for estimating crop evapotranspiration in the Upper Basin 22 . When new estimates become available we will update our water budget accordingly.

MCI and agricultural water consumption

The primary source of data on aggregate MCI (municipal, commercial, and industrial) and agricultural water consumption from the Upper and Lower Basins was the US Bureau of Reclamation. Water consumed from the Upper Basin is published in Reclamation’s five-year reports entitled “Colorado River—Upper Basin Consumptive Uses and Losses.” 23 These annual data have been compiled into a single spreadsheet used for this study 24 . Because measurements of agricultural diversions and return flows in the Upper Basin are not sufficiently complete to allow direct calculation of consumptive use, theoretical and indirect methods are used as described in the Consumptive Uses and Losses reports 25 . Reclamation performs these estimates for Colorado, Wyoming, and Utah, but the State of New Mexico provides its own estimates that are collaboratively reviewed with Reclamation staff. The consumptive use of water in thermoelectric power generation in the Upper Basin is provided to Reclamation by the power companies managing each generation facility. Reclamation derives estimates of consumptive use for municipal and industrial purposes from the US Geological Survey’s reporting series (published every 5 years) titled “Estimated Use of Water in the United States” at an 8-digit watershed scale 26 .

Use of shallow alluvial groundwater is included in the water accounting compiled by Reclamation but use of deeper groundwater sources—such as in Mexico and the Gila River Basin—is explicitly excluded in their accounting, and in ours. Reclamation staff involved with water accounting for the Upper and Lower Basins assume that groundwater use counted in their data reports is sourced from aquifers that are hydraulically connected to rivers and streams in the CRB (James Prairie, US Bureau of Reclamation, personal communication, 2023); because of this high connectivity, much of the groundwater being consumed is likely being sourced from river capture as discussed in Jasechko et al. 27 and Wiele et al. 28 and is soon recharged during higher river flows.

Water consumed from the Lower Basin (excluding water supplied by the Gila River Basin) is published in Reclamation’s annual reports entitled “Colorado River Accounting and Water Use Report: Arizona, California, and Nevada.” 3 These consumptive use data are based on measured deliveries and return flows for each individual water user. These data are either measured by Reclamation or provided to the agency by individual water users, tribes, states, and federal agencies 29 . When not explicitly stated in Reclamation reports, attribution of water volumes to MCI or agricultural uses was based on information obtained from each water user’s website, information provided directly by the water user, or information on export water use provided in Siddik et al. 30 . Water use by entities using less than 1.23 million cubic meters (1000 acre-feet) per year on average was allocated to MCI and agricultural uses according to the overall MCI-agricultural percentages calculated within each sub-basin indicated in Tables  1 and 2 for users of greater than 1.23 million cubic meters/year.

Disaggregation of water consumption by sector was particularly important and challenging for the Central Arizona Project given that this canal accounts for 21% of all direct water consumption in the Lower Basin. Reclamation accounts for the volumes of annual diversions into the Central Arizona Project canal but the structure serves 1071 water delivery subcontracts. We classified every unique Central Arizona Project subcontract delivery between 2000–2019 by its final water use to derive an estimated split between agricultural and MCI uses. Central Arizona Project subcontract delivery data were obtained from the current and archived versions of the project’s website summaries in addition to being directly obtained from the agency through a public information request. Subcontract deliveries were classified based on the final end use, including long-term and temporary leases of project water. This accounting also includes the storage of water in groundwater basins for later MCI or agricultural use. Additionally, water allocated to Native American agricultural uses that was subsequently leased to cities was classified as an MCI use.

Data for the Gila River basin was obtained from two sources. The Arizona Department of Water Resources has published data for surface water use in five “Active Management Areas” (AMAs) located in the Gila River basin: Prescott AMA, Phoenix AMA, Pinal AMA, Tucson AMA, and Santa Cruz AMA 31 . The water-use data for these AMAs is compiled from annual reports submitted by each water user (contractor) and then reviewed by the Arizona Department of Water Resources. The AMA water-use data are categorized by purpose of use, facilitating our separation into MCI and agricultural uses. These data are additionally categorized by water source; only surface water sourced from the Gila River hydrologic system was counted (deep groundwater use was not). The AMA data were supplemented with data for the upper Gila River basin provided by the University of Arizona 32 . We have assumed that all water supplied by the Gila River Basin is fully consumed, as the river is almost always completely dry in its lower reaches (less than 1% flows out of the basin into the Colorado River, on average 33 ).

Data for Mexico were obtained from Hernandez-Cruz et al. 10 based on estimates for 2008–2015. Agricultural demands were estimated from annual reports of irrigated area and water use published by the Ministry of Agriculture and the evapotranspiration estimates of the principal crops published by the National Institute for Forestry, Animal Husbandry, and Agricultural Research of Mexico 10 . The average annual volume of Colorado River water consumption in Mexico estimated by these researchers is within 1% of the cross-border delivery volume estimated by the Bureau of Reclamation for 2000–2019 in its Colorado River Accounting and Water Use Reports 3 .

Exported water consumption

Annual average inter-basin transfer volumes for each of 46 canals and pipelines exporting water outside of the Upper Basin were obtained from Reclamation’s Consumptive Uses and Losses spreadsheet 34 . Data for the Colorado River Aqueduct in the Lower Basin were obtained from Siddik et al. 30 Data for exported water in Mexico was available from Hernandez-Cruz et al. 10 . We assigned any seepage or evaporation losses from inter-basin transfers to their proportional end uses. All uses of exported water are considered to be consumptive uses with respect to the Colorado River, because none of the water exported out of the basin is returned to the Colorado River Basin.

We relied on data from Siddik et al. (2023) to identify whether the water exported out of the Colorado River Basin was for only MCI or agricultural use. When more than one water use purpose was identified, as well as for all major inter-basin transfers, we used government and inter-basin transfer project websites or information obtained directly from the project operator or water manager to determine the volume of water transferred and the end uses. Major recipients of exported water include the Coachella Valley Water District (California); Metropolitan Water District of Southern California (particularly for San Diego County, California); Northern Colorado Water Conservancy District; City of Denver (Colorado); the Central Utah Project; City of Albuquerque (New Mexico); and the Middle Rio Grande Conservancy District (New Mexico). We did not pursue sectoral water-use information for 17 of the 46 Upper Basin inter-basin transfers due to their relatively low volumes of water transferred by each system (<247,000 cubic meters or 2000 acre-feet), and instead assigned the average MCI or agricultural percentage (72% MCI, 28% agricultural) from all other inter-basin transfers in the Upper Basin. The export volume of these 17 inter-basin transfers sums to 9.76 million cubic meters (7910 acre-feet) per year, equivalent to 1% of the total volume exported from the Upper Basin.

Reservoir evaporation

Evaporation estimates for the Upper Basin and Lower Basin are based upon Reclamation’s HydroData repository 35 . Reclamation’s evaporation estimates are based on the standardized Penman-Monteith equation as described in the “Lower Colorado River Annual Summaries of Evapotranspiration and Evaporation” reports 17 . The Penman-Monteith estimates are based on pan evaporation measurements. Evaporation estimates for the Salt River Project reservoirs in the Gila River basin were provided by the Salt River Project in Arizona (Charlie Ester, personal communication, 2023).

Another consideration with reservoirs is the volume of water that seeps into the banks or sediments surrounding the reservoir when reservoir levels are high, but then drains back into the reservoir as water levels decline 36 . This has the effect of either exacerbating reservoir losses (consumptive use) or offsetting evaporation when bank seepage flows back into a reservoir. The flow of water into and out of reservoir banks is non-trivial; during 1999–2008, an estimated 247 million cubic meters (200,000 acre-feet) of water drained from the canyon walls surrounding Lake Powell into the reservoir each year, providing additional water supply 36 . However, the annual rate of alternating gains or losses has not been sufficiently measured at any of the basin’s reservoirs and therefore is not included in Tables  1 and 2 .

Riparian and wetland vegetation evapotranspiration

We exported the total annual evapotranspiration depth at a 30 meter resolution from OpenET 37 using Google Earth Engine from 2016 to 2019 to align with OpenET’s data availability starting in 2016. Total annual precipitation depths, sourced from gridMET 38 , were resampled to align with the evapotranspiration raster resolution. Subsequently, a conservative estimate of the annual water depth utilized by riparian vegetation from the river was derived by subtracting the annual precipitation raster from the evapotranspiration raster for each year. Positive differentials, indicative of river-derived evapotranspiration, were then multiplied by the riparian vegetation area as identified in the CO-RIP 16 dataset to estimate the total annual volumetric water consumption by riparian vegetation across the Upper, Lower, and Gila River Basins. The annual volumetric water consumption calculated over four years were finally averaged to get riparian vegetation evapotranspiration in the three basins. Because the entire flow of the Colorado River is diverted into the Canal Alimentador Central near the international border, very little riparian evapotranspiration occurs along the river south of the international border in the Mexico basin.

In addition to water consumed by riparian evapotranspiration within the Lower Basin, the Salton Sea receives agricultural drain water from both the Imperial Irrigation District and the Coachella Valley Irrigation District, stormwater drainage from the Coachella Valley, and inflows from the New and Alamo Rivers 39 . Combined inflows to the Sea during 2015–2019 were added to our estimates of riparian/wetland evapotranspiration in the Lower Basin.

Similarly, Mexico receives drainage water from the Wellton–Mohawk bypass drain originating in southern Arizona that empties into the Cienega de Santa Clara (a wetland); this drainage water is included as riparian/wetland evapotranspiration in the Mexico basin.

Crop-specific water consumption

The volumes of total agricultural consumption reported for each sub-basin in Tables  1 and 2 were obtained from the same data sources described above for MCI consumption and exported water. The portion (%) of those agricultural consumption volumes going to each individual crop was then allocated according to percentage estimates of each crop’s water consumption in each accounting unit using methods described in Richter et al. 20 and detailed here.

Monthly crop water requirements during 1981–2019 for 13 individual crops, representing 68.8% of total irrigated area in the US in 2019, were estimated using the AquaCrop-OS model (Table SI- 3 ) 40 . For 17 additional crops representing about 25.4% of the total irrigated area, we used a simple crop growth model following Marston et al. 41 as crop parameters needed to run AquaCrop-OS were not available. A list of the crops included in this study is shown in Table SI- 3 . The crop water requirements used in Richter et al. 11 were based on a simplistic crop growth model, often using seasonal crop coefficients whereas we use AquaCrop-OS 40 , a robust crop growth model, to produce more realistic crop growth and crop water estimates for major crops. AquaCrop-OS is an open-source version of the AquaCrop model 42 , a crop growth model capable of simulating herbaceous crops. Additionally, we leverage detailed local data unique to the US, including planting dates and subcounty irrigated crop areas, to produce estimates at a finer spatial resolution than the previous study. We obtained crop-specific planting dates from USDA 43 progress data at the state level. For crops that did not have USDA crop progress data, we used data from FAO 44 and CUP+ model 45 for planting dates. We used climate data (precipitation, minimum and maximum air temperature, reference ET) from gridMET 38 , soil texture data from ISRIC 46 database and crop parameters from AquaCrop-OS to run the model. The modeled crop water requirement was partitioned into blue and green components following the framework from Hoekestra et al. 47 , assuming that blue and green water consumed on a given day is proportional to the amount of green and blue water soil moisture available on that day. When applying a simple crop growth model, daily gridded (2.5 arc minutes) crop-specific evapotranspiration (ETc) was computed by taking the product of reference evapotranspiration (ETo) and crop coefficient (Kc), where ETo was obtained from gridMET. Crop coefficients were calculated using planting dates and crop coefficient curves from FAO and CUP+ model. Kc was set to zero outside of the growing season. We partitioned the daily ETc into blue and green components by following the methods from ref. 41 It is assumed that the crop water demands are met by irrigation whenever it exceeds effective precipitation (the latter calculated using the USDA Soil Conservation Service method (USDA, 1968 48 ). We obtained county level harvested area from USDA 43 and disaggregated to sub-county level using Cropland Data Layer (CDL) 49 and Landsat-based National Irrigation Dataset (LANID) 50 . The CDL is an annual raster layer that provides crop-specific land cover data, while the LANID provides irrigation status information. The CDL and LANID raster were multiplied and aggregated to 2.5 arc minutes to match the AquaCrop-OS output. We produced a gridded crop area map by using this resulting product as weights to disaggregate county level area. CDL is unavailable before 2008. Therefore, we used land use data from ref. 51 in combination with average CDL map and county level harvested area to produce gridded crop harvested area. We computed volumetric water consumption by multiplying the crop water requirement depth by the corresponding crop harvested area.

Data availability

All data compiled and analyzed in this study are publicly available as cited and linked in our Methods section. Our compilation of these data is also available from Hydroshare at: http://www.hydroshare.org/resource/2098ae29ae704d9aacfd08e030690392 .

Code availability

All model code and software used in this study have been accessed from sources cited in our Methods section. We used AquaCrop-OS (v5.0a), an open source version of AquaCrop crop growth model, to run crop simulations. This model is publicly available at http://www.aquacropos.com/ . For estimating riparian evapotranspiration, we used ArcGIS Pro 3.1.3 on the Google Earth Engine. Riparian vegetation distribution maps were sourced from Dryad at https://doi.org/10.5061/dryad.3g55sv8 .

Stromberg, J. C., Andersen, D. C. & Scott, M. L. Riparian floodplain wetlands of the arid and semiarid southwest In Wetland Habitats of North America: Ecology and Conservation Concern s , Chapter 24, pp. 343–356. (University of California Press, 2012). https://www.ucpress.edu/book/9780520271647/wetland-habitats-of-north-america .

Schmidt, J. C., Yackulic, C. B. & Kuhn, E. The Colorado River water crisis: Its origin and the future. WIREs Water https://doi.org/10.1002/wat2.1672 (2023).

Article   Google Scholar  

Colorado River Accounting and Water Use Report: Arizona, California, and Nevada. Interior Region 8: Lower Colorado Basin (US Bureau of Reclamation, 2023). Annual reports available under “Water Accounting Reports” at https://www.usbr.gov/lc/region/g4000/wtracct.html .

Water Operations: Historic Data (US Bureau of Reclamation, 2023). https://www.usbr.gov/rsvrWater/HistoricalApp.html .

Colorado River Compact , 1922 . US Bureau of Reclamation. https://www.usbr.gov/lc/region/pao/pdfiles/crcompct.pdf .

Kuhn, E. & Fleck, J. Science Be Dammed:How Ignoring Inconvenient Science Drained the Colorado River (The University of Arizona Press, 2019) https://uapress.arizona.edu/book/science-be-dammed .

Castle, A. & Fleck, J. The Risk of Curtailment under the Colorado River Compact ( https://doi.org/10.2139/ssrn.3483654 (2019).

US Bureau of Reclamation. Colorado River Basin Water Supply and Demand Study: Technical Report C – Water Demand Assessment https://www.usbr.gov/lc/region/programs/crbstudy/finalreport/Technical%20Report%20C%20-%20Water%20Demand%20Assessment/TR-C-Water_Demand_Assessmemt_FINAL.pdf (2012).

Utilization of the Waters of the Colorado and Tijuana Rivers and of the Rio Grande . International Treaty between the United States and Mexico, February 3, 1944. (International Boundary and Waters Commission, 1944). https://www.ibwc.gov/wp-content/uploads/2022/11/1944Treaty.pdf .

Hernández-Cruz, A. et al. Assessing water management strategies under water scarcity in the Mexican portion of the Colorado River Basin. J. Water Resour. Plan. Manag. 149 , 04023042 (2023).

Richter, B. D. et al. Water scarcity and fish imperilment driven by beef production. Nat. Sustain. 3 , 319–328 (2020).

Biden-Harris Administration announces historic Consensus System Conservation Proposal to protect the Colorado River Basin . US Department of the Interior, May 22, 2023. https://www.doi.gov/pressreleases/biden-harris-administration-announces-historic-consensus-system-conservation-proposal .

Wheeler, K. G. et al. What will it take to stabilize the Colorado River? Science 377 , 373–375 (2022).

Article   ADS   CAS   PubMed   Google Scholar  

Richter, B. D. Decoupling urban water use from population growth in the Colorado River Basin. J. Water Plan. Manag. 149 , 2 (2023).

Google Scholar  

Schmidt, J. C. Maps Matter: A few suggested changes to the Colorado River basin base map . Center for Colorado River Studies. (Utah State University, 2022).

Woodward, B. D. et al. Co-Rip: A riparian vegetation and corridor extent dataset for Colorado river basin streams and rivers. ISPRS Int. J. Geo Inform. 7 , 397 (2018).

Article   ADS   Google Scholar  

Lower Colorado River Annual Summaries of Evapotranspiration and Evaporation . (US Bureau of Reclamation, Lower Colorado Region, 2023). https://www.usbr.gov/lc/region/g4000/wtracct.html .

Richter, B. D., Powell, E. M., Lystash, T. & Faggert, M. Protection and restoration of freshwater ecosystems. Chapter 5 in Miller, Kathleen A., Alan F. Hamlet, Douglas S. Kenney, and Kelly T. Redmond (Eds.) Water Policy and Planning in a Variable and Changing Climate . (CRC Press - Taylor & Francis Group, 2016).

Shao, Elena. “The Colorado River is shrinking. See what’s using all the water.” New York Times , May 22, 2023. https://www.nytimes.com/interactive/2023/05/22/climate/colorado-river-water.html .

Richter, B. D., et al. Alleviating water scarcity by optimizing crop mixes. Nat. Water . https://doi.org/10.1038/s44221-023-00155-9 .

Njuki, E. U.S. dairy productivity increased faster in large farms and across southwestern states . U.S. Economic Research Service, US Department of Agriculture, March 22, 2022. https://www.ers.usda.gov/amber-waves/2022/march/u-s-dairy-productivity-increased-faster-in-large-farms-and-across-southwestern-states/ .

Mefford, B. & Prairie J., eds. Assessing Agricultural Consumptive Use in the Upper Colorado River Basin - Phase III Report U.S. Bureau of Reclamation and the Upper Colorado River Commission. http://www.ucrcommission.com/reports-studies/ (2022).

Upper Basin Consumptive Uses and Losses (Bureau of Reclamation). Annual reports available at https://www.usbr.gov/uc/envdocs/plans.html .

Bureau of Reclamation. “Consumptive Uses and Losses spreadsheet 1971–2020” Colorado River Basin Natural Flow and Salt Data, Supporting data for consumptive uses and losses computation. https://www.usbr.gov/lc/region/g4000/NaturalFlow/documentation.html .

Upper Colorado River Basin Consumptive Uses and Losses Report 2016–2020 . US Department of Interior: Bureau of Reclamation. Five year reports available under “Colorado River-Consumptive Uses and Losses Reports” at https://www.usbr.gov/uc/envdocs/plans.html .

Estimated Use of Water in the United States . US Department of Interior: US Geological Survey. Reports available every five years at https://www.usgs.gov/mission-areas/water-resources/science/water-use-united-states .

Jasechko, S. et al. Widespread potential loss of streamflow into underlying aquifers across the USA. Nature 591 , 391–395 (2021).

Wiele, S. M., Leake, S. A., Owen-Joyce, S. J. & and McGuire, E. H. Update of the Accounting Surface Along the Lower Colorado River US Department of the Interior: US Geological Survey Scientific Investigations Report 2008–5113 (2008).

Bruce, B. W., et al. Comparison of U.S. Geological Survey and Bureau of Reclamation water-use reporting in the Colorado River Basin U.S. Geological Survey Scientific Investigations Report 2018–5021 . https://doi.org/10.3133/sir20185021 (2018).

Siddik, M. A. B., Dickson, K. E., Rising, J., Ruddell, B. L. & Marston, L. T. Interbasin water transfers in the United States and Canada. Sci. Data 10 , 27 (2023). Data spreadsheet provided by M.A.B. Siddik.

Article   PubMed   PubMed Central   Google Scholar  

Active Management Areas : AMA Annual Supply and Demand Dashboard (Arizona Department of Water Resources, 2023). https://azwater.gov/ama/ama-data .

Lacroix, K. M. et al. Wet water and paper water in the Upper Gila River Watershed https://extension.arizona.edu/sites/extension.arizona.edu/files/pubs/az1708-2016_0.pdf The University of Arizona Cooperative Extension, AZ1708. Data spreadsheet provided by A. Hullinger (2016).

Surface-Water Annual Statistics for the Nation: Gila River at Dome, Arizona . US Geological Survey. Available at https://waterdata.usgs.gov/nwis/annual/?referred_module=sw&site_no=09520500&por_09520500_5810=19975,00060,5810,1905,2024&year_type=C&format=html_table&date_format=YYYY-MM-DD&rdb_compression=file&submitted_form=parameter_selection_list .

Consumptive Uses and Losses spreadsheet 1971–2020 . Bureau of Reclamation, Colorado River Basin Natural Flow and Salt Data, Supporting data for consumptive uses and losses computation. https://www.usbr.gov/lc/region/g4000/NaturalFlow/documentation.html .

HydroData: Reservoir Data . US Bureau of Reclamation. https://www.usbr.gov/uc/water/ .

Myers, T. Loss rates from Lake Powell and their impact on management of the Colorado River. J. Am. Water Resour. Assoc. 49 , 1213–1224 (2013).

Melton, F. S. et al. OpenET: filling a critical data gap in water management for the western United States. J. Am. Water Resour. Assoc. 58 , 971–994 (2022).

Abatzoglou, J. T. Development of gridded surface meteorological data for ecological applications and modelling. Int. J. Climatol. 33 , 121–131 (2013).

Salton Sea Management Program: Long-Range Plan Public Draft (2022). California Natural Resources Agency. https://saltonsea.ca.gov/wp-content/uploads/2022/12/Salton-Sea-Long-Range-Plan-Public-Draft-Dec-2022.pdf .

Foster, T. et al. AquaCrop-OS: an open source version of FAO’s crop water productivity model. Agricul. Water Manag. 181 , 18–22 (2017).

Marston, L. T., et al. Reducing water scarcity by improving water productivity in the United States. Environ. Res. Lett. 15 https://doi.org/10.1088/1748-9326/ab9d39 (2020).

Steduto, P., Hsiao, T. C., Fereres, E. & Raes, D. Crop yield response to water (2012). 1028. Rome: Food and Agriculture Organization of the United Nations.

USDA, National Agricultural Statistics Service. “Quick Stats.” http://quickstats.nass.usda.gov .

Allen, R. G., Pereira, L. S., Raes, D. & Smith, M. FAO Irrigation and drainage paper No. 56 56, (e156. Food and Agriculture Organization of the United Nations, Rome, 1998).

Orange, M. N., Scott Matyac, J. & Snyder, R. L. Consumptive use program (CUP) model. IV Int. Symp. Irrig. Horticult. Crops 664 , 461–468 (2003).

Hengl, T. et al. SoilGrids250m: Global gridded soil information based on machine learning. PLoS One 12 , e0169748 (2017).

Hoekstra, A. Y. Green-blue water accounting in a soil water balance. Adv. Water Resour. 129 , 112–117 (2019).

USDA (US Department of Agriculture). A Method for Estimating Volume and Rate of Runoff in Small Watersheds . SCS-TP-149. Washington DC: Soil Conservation Service (1968).

Johnson, D. M., & Mueller, R. 2010. “Cropland Data Layer.” https://nassgeodata.gmu.edu/CropScape/ .

Xie, Y., Gibbs, H. K. & Lark, T. J. Landsat-based Irrigation Dataset (LANID): 30m resolution maps of irrigation distribution, frequency, and change for the US, 1997–2017. Earth Syst. Sci. Data 13 , 5689–5710 (2021).

Sohl, T. et al. Modeled historical land use and land cover for the conterminous United States. J. Land Use Sci. 11 , 476–499 (2016).

Download references

Acknowledgements

This paper is dedicated to our colleague Jack Schmidt in recognition of his retirement and enormous contributions to the science and management of the Colorado River. The authors thank James Prairie of the US Bureau of Reclamation, Luke Shawcross of the Northern Colorado Water Conservancy District, Charlie Ester of the Salt River Project, and Brian Woodward of the University of California Cooperative Extension for their assistance in accessing data used in this study. The authors also thank Rhett Larson at the Sandra Day O’Connor School of Law at Arizona State University for their review of Arizona water budget data, and the Central Arizona Project for providing delivery data by each subcontract. G.L., L.M., and K.F.D. acknowledge support by the United States Department of Agriculture National Institute of Food and Agriculture grant 2022-67019-37180. L.T.M. acknowledges the support the National Science Foundation grant CBET-2144169 and the Foundation for Food and Agriculture Research Grant No. FF-NIA19-0000000084. R.R.R. acknowledges the support the National Science Foundation grant CBET-2115169.

Author information

Authors and affiliations.

World Wildlife Fund, 1250 24th St NW, Washington, DC, 20037, USA

Brian D. Richter

Sustainable Waters, Crozet, Virginia, 22932, USA

The Charles E.Via, Jr, Department of Civil and Environmental Engineering, Virginia Tech, Blacksburg, VA, 24061, USA

Gambhir Lamsal, Landon Marston & Sameer Dhakal

Northern Arizona University, Flagstaff, AZ, 86011, USA

Laljeet Singh Sangha, Richard R. Rushforth & Benjamin L. Ruddell

Department of Geography and Spatial Sciences, University of Delaware, Newark, DE, 19716, USA

Dongyang Wei & Kyle Frankel Davis

Department of Plant and Soil Sciences, University of Delaware, Newark, DE, 19716, USA

Kyle Frankel Davis

Instituto de Investigaciones Oceanologicas, Universidad Autonoma de Baja California, Ensenada, Baja California, México

Astrid Hernandez-Cruz

Department of Land, Air and Water Resources, Univeristy of California at Davis, Davis, CA, 95616, USA

Samuel Sandoval-Solis

Center for Colorado River Studies, Utah State University, Logan, UT, 84322, USA

John C. Schmidt

You can also search for this author in PubMed   Google Scholar

Contributions

B.D.R. designed the study, compiled and analyzed data, wrote the manuscript and supervised co-author contributions. G.L. compiled all crop data, estimated crop evapotranspiration, and prepared figures. S.D. compiled all riparian vegetation data and estimated riparian evapotranspiration. L.S.S. and R.R.R. accessed, compiled, and analyzed data from the Central Arizona Project. D.W. compiled data and prepared figures. A.H.-C. and S.S.-S. compiled and analyzed data for Mexico. J.C.S. compiled and analyzed reservoir evaporation data and edited the manuscript. L.M., B.L.R., and K.F.D. supervised data compilation and analysis and edited the manuscript.

Corresponding author

Correspondence to Brian D. Richter .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Peer review

Peer review information.

Communications Earth & Environment thanks James Booker and Becky Bolinger for their contribution to the peer review of this work. Primary Handling Editors: Aliénor Lavergne and Carolina Ortiz Guerrero. A peer review file is available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Peer review file, supplementary information, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Richter, B.D., Lamsal, G., Marston, L. et al. New water accounting reveals why the Colorado River no longer reaches the sea. Commun Earth Environ 5 , 134 (2024). https://doi.org/10.1038/s43247-024-01291-0

Download citation

Received : 03 October 2023

Accepted : 27 February 2024

Published : 28 March 2024

DOI : https://doi.org/10.1038/s43247-024-01291-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Journal of Materials Chemistry C

Homochirality to design high- t c lead-free ferroelastic semiconductor.

Ferroelastic semiconductor materials have garnered significant research interest due to their promising applications in the fields of shape memory, superelasticity, templated electronic nanostructures, mechanical switching, and optoelectronic transmission. However, the toxicity of lead-based structures and low phase-transition temperature ( T c ) greatly constrain the application scenarios of ferroelastic semiconductors. Here, by H/OH-substitution-induced homochiral strategy, we synthesize a pair of lead-free ferroelastic semiconductors ( R / S -CTA) 2 SbCl 5 (CTA = 3-Chloro-2-hydroxypropyltrimethyllammonium) having semiconductor properties with an indirect bandgap of 3.41 eV. They crystallized in the chiral space group P 2 1 2 1 2 1 at room temperature, and both undergo 422 F 222 type ferroelastic phase transitions with T c up to 410 K, accompanied by a large entropy change of 68.75 and 66.09 J mol K -1 , respectively. Owing to the introduction of chirality, they exhibited temperature-dependent nonlinear second-harmonic generation (SHG) properties. Relatively, the achiral TMCP (TMCP = N , N , N -trimethylchloropropylamine) makes the phase transition properties of centrosymmetric TMCP 2 SbCl 5 ordinary compared to chiral R / S - pair. This is precisely the main starting point of homochiral strategy in phase transition and optical structure research, while arousing research interest. This work, which provides a new avenue for the design of high- T c lead-free ferroelastic semiconductor compounds, is a powerful motivation for the realization of multifunctional materials related to chirality.

• This article is part of the themed collection: Journal of Materials Chemistry C HOT Papers

Supplementary files

• Supplementary information PDF (702K)
• Crystal structure data CIF (3448K)

Article information

Download Citation

Permissions.

B. Deng, Z. Rao, M. Shen, K. Liang, Y. Zhu, Z. Wang, K. Ding, C. Su, M. Lun, Z. Zhang, Y. Zhang and D. Fu, J. Mater. Chem. C , 2024, Accepted Manuscript , DOI: 10.1039/D4TC00428K

This article is licensed under a Creative Commons Attribution-NonCommercial 3.0 Unported Licence . You can use material from this article in other publications, without requesting further permission from the RSC, provided that the correct acknowledgement is given and it is not used for commercial purposes.

To request permission to reproduce material from this article in a commercial publication , please go to the Copyright Clearance Center request page .

If you are an author contributing to an RSC publication, you do not need to request permission provided correct acknowledgement is given.

If you are the author of this article, you do not need to request permission to reproduce figures and diagrams provided correct acknowledgement is given. If you want to reproduce the whole article in a third-party commercial publication (excluding your thesis/dissertation for which permission is not required) please go to the Copyright Clearance Center request page .

Read more about how to correctly acknowledge RSC content .

Social activity

Search articles by author.

This article has not yet been cited.

Advertisements