qualitative research process quiz

100 Questions (and Answers) About Qualitative Research

• Lisa M. Given - Swinburne University, Australia, Charles Sturt University, Australia, RMIT University, Melbourne, Australia

“This is a great companion book for a course on qualitative methods and it is also a great resource as a ‘ready-reference,’ which should be a required companion for all graduate students who will be taking qualitative research methods.”

“It provides an overview of the subject on the nuances of qualitative research.”

“ Very precise in helping students determine if their study is appropriate for this type of research design.”

“The book appears to provide the right combination of breadth and depth . There are a lot of topics covered, but the book seems to provide a succinct, snapshot-like answer for each question.”

“A book like this can provide a useful supplement to major texts and be used as a reference.”

Lisa M. Given

No internet connection.

All search filters on the page have been cleared., your search has been saved..

• All content
• Dictionaries
• Encyclopedias
• Expert Insights
• Foundations
• How-to Guides
• Journal Articles
• Little Blue Books
• Little Green Books
• Project Planner
• Tools Directory
• Sign in to my profile My Profile

• Sign in Signed in
• My profile My Profile

100 Questions (and Answers) About Qualitative Research

• By: Lisa M. Given
• Publisher: SAGE Publications, Inc
• Series: SAGE 100 Questions and Answers
• Publication year: 2016
• Online pub date: January 19, 2023
• Discipline: Sociology , Criminology and Criminal Justice , Business and Management , Communication and Media Studies , Education , Psychology , Health , Social Work , Political Science and International Relations
• Methods: Data collection , Focus groups , Research questions
• DOI: https:// doi. org/10.4135/9781483398655
• Keywords: decision making , discipline , organizations , publications , social media , software , teams Show all Show less
• Print ISBN: 9781483345642
• Online ISBN: 9781483398655
• Buy the book icon link

Subject index

Exploring 100 key questions (and answers) on the nature and practice of qualitative inquiry, this unique book addresses the practical decisions that researchers must make in their work, from the design of the study, through ethics approval, implementation, and writing. The book’s quick-scan, question-and-answer format make it ideal as a supplementary text or as a ready reference for graduate students preparing for comprehensive exams and writing research proposals, undergraduates in affiliated programs who will not be taking a primary course in qualitative research methods, and researchers working across disciplines in academic or practice environments.

Front Matter

• Acknowledgements
• Acknowledgments
• About the Author

Part 1: The Nature of Qualitative Inquiry

• Chapter 1: What Is Qualitative Research?
• Chapter 2: What Disciplines Use Qualitative Approaches and Are There Differences in Disciplinary Approach?
• Chapter 3: Is Qualitative Research Used in Practice or Only in Academic Research?
• Chapter 4: My Supervisor Says That Quantitative Research Is More Objective, So It’s Better Than Qualitative Research. Is That True?
• Chapter 5: What Is the Difference Between “Ontology” and “Epistemology,” and Why Do They Matter?
• Chapter 6: I’ve Heard That Qualitative Research Is More Inductive Rather Than Deductive—What Does That Mean?
• Chapter 7: What Is the Difference Between a Project Designed With a Qualitative “Paradigm” and a Project Designed to Gather Qualitative “Data”?
• Chapter 8: What Is the Difference Between Quantitative Positivism and Qualitative Constructionism?
• Chapter 9: Qualitative Research Seems to Always Involve People—Is That True?
• Chapter 10: What Is the Difference Between a Research Participant and a Research Subject?
• Chapter 11: My Participants Are Really Co-Researchers in My Work—So What Are the Implications for My Project?
• Chapter 12: What Kind of Education or Training Do I Need to Conduct Qualitative Research?
• Chapter 13: What Kind of Time Investment Is Needed for a Qualitative Research Study?
• Chapter 14: Qualitative Research Seems to Be More Expensive to Do Than Other Types of Research—Is That So?
• Chapter 15: What Are the Limitations of Qualitative Research?

Part 2: Ethical Issues in Qualitative Research

• Chapter 16: What Are the Researcher’s Ethical Responsibilities in Qualitative Practice?
• Chapter 17: At What Stage of the Research Do I Need to Get a Formal Ethics Review to Talk to People?
• Chapter 18: What Kinds of Ethics Challenges Do Qualitative Researchers Face, Typically?
• Chapter 19: Ethics Approval Seems to Be More Difficult to Obtain for Qualitative Projects. Is That True?
• Chapter 20: Can I Name My Participants and Their Organization in Publications About My Study?
• Chapter 21: I’m Going to Do Focus Groups and I Know I’ll Need Ethics Approval for Those—But Can I Examine Postings to Social Media Without Seeking Ethics Approval?
• Chapter 22: Can I Show My Colleague Some Transcripts and Let Her Listen to Interview Recordings to Get Advice on My Interpretation of the Data?
• Chapter 23: The Ethics Review Board Requires Me to Submit My Interview Questions—But the Project Is Exploratory and the Questions Will Emerge as the Interview Happens. What Types of Questions Should I Submit for Review?
• Chapter 24: The Ethics Review Board Says I Have to Destroy My Data, but I Think My Analysis Will Take Years. Do I Have to Destroy Everything?
• Chapter 25: I Have Learned Negative Things About People in the Setting I’m Studying. How Do I Deal With This?
• Chapter 26: My Ethics Approval Says That I Have to Let the Board Know if There are “Significant Changes” to My Methodology and/or Method. As My Qualitative Study Is Exploratory and Emergent in Design, How Do I Know When a Significant Change Has Occurred?
• Chapter 27: One of My Participants Told Other People That She Was Involved in My Study, Even Though I Promised I Would Not Identify Her. Is This a Problem?

Part 3: Designing Qualitative Research

• Chapter 28: What Is a Qualitative Research Problem—And How Does This Inform the Development of Research Questions?
• Chapter 29: What Is the Role of a Hypothesis in Qualitative Research?
• Chapter 30: What Is an Exploratory Qualitative Design? If I Do This, Does It Mean That My Research Isn’t Going to Come Up With “Usable” Findings?
• Chapter 31: What Is an Emergent Qualitative Design?
• Chapter 32: What Is the Role of a Theoretical and/or Conceptual Framework in a Qualitative Study?
• Chapter 33: How Extensive Should My Literature Review Be When I’m Designing My Project?
• Chapter 34: What Kinds of Sampling Approaches Are Appropriate for Qualitative Studies?
• Chapter 35: Why Are Sample Sizes So Small in Many Qualitative Studies? Isn’t This a Problem?
• Chapter 36: What Is the Ideal Sample Size for a Qualitative Project?
• Chapter 37: How Do You Recruit Participants for a Qualitative Study?
• Chapter 38: I Really Want to Use Focus Groups, So How Can I Design a Project That Will Use That Method?
• Chapter 39: What Is Triangulation?

Part 4: Ensuring Rigor in Qualitative Research Design

• Chapter 40: I’ve Heard That Quantitative Research Is More Rigorous Than Qualitative Research—Is That True?
• Chapter 41: What Are Effective Strategies for Promoting Trustworthiness?
• Chapter 42: I’ve Heard That Qualitative Studies Suffer From Researcher Bias. How Do I Deal With This Criticism?
• Chapter 43: Can the Findings From Qualitative Research Be Generalized? I’ve Heard They Can Be Transferable, but I Don’t Know the Differences Between These Terms.
• Chapter 44: How Do I Ensure That My Study Will Have an Impact on Other Scholars or Practitioners?
• Chapter 45: What Are Useful and Practical Approaches to Ensure That I Am Gathering Good Data?

Part 5: Methodologies and Methods

• Chapter 46: What Are the Differences Between Qualitative “Design,” “Methodology,” and “Method”?
• Chapter 47: All the Studies I’ve Read Seem to Use Interviews. What Other Methods Can I Use to Make My Study More Interesting?
• Chapter 48: I Don’t Understand the Differences Between Grounded Theory, Phenomenology, Case Study, Ethnography, Narrative Inquiry, and So On. Can I Combine These—Or Choose Not to Use One of These Approaches at All?
• Chapter 49: What Kinds of Research Methods Are Appropriate for Talking With People?
• Chapter 50: What Are the Pros and Cons of Conducting Individual Versus Group Interviews?
• Chapter 51: I Want to Observe What People Are Doing, but I Don’t Want Them to Know That I’m Watching. Can I Do That?
• Chapter 52: How Can I Use Documents in My Qualitative Study?
• Chapter 53: I’ve Heard That There Are Some Interesting Visual Methods That I Can Use—What Are They?
• Chapter 54: In My Discipline We Conduct a Lot of Systematic Reviews of the Literature. Is It Possible to Do a Qualitative Systematic Review?

Part 6: Mixed-Methods Research Involving Qualitative Approaches

• Chapter 55: I’ve Heard That Qualitative Research Is Only Useful as a First, Exploratory Step to Designing a Quantitative Project—Is That True?
• Chapter 56: How Can Qualitative Research Complement a Quantitative Study?
• Chapter 57: Which Do I Do First—The Qualitative Component of the Study or the Quantitative Component?
• Chapter 58: I Have Included Some Open-Ended Questions Alongside the Closed-Response Items on My Survey/Questionnaire. Am I Conducting Qualitative Research?
• Chapter 59: Is It Better to Bring a Qualitative Researcher Onto My Team, or Should I Try to Do the Qualitative Research Myself?
• Chapter 60: I’ve Only Ever Used Quantitative Designs, but I Want to Use Qualitative Approaches Now—What Are the Key Issues I Need to Consider and How Can I Learn More About Them?

Part 7: Collecting Qualitative Data

• Chapter 61: How Do I Conduct a Pilot Study for My Qualitative Research Project?
• Chapter 62: My Colleague Says That We Need to Be Unobtrusive When Gathering Data—What Does That Mean?
• Chapter 63: Can I Hire Someone to Do All of the Data Collection?
• Chapter 64: What Does It Mean to Be “Neutral” When I’m Gathering My Data?
• Chapter 65: What Does “Fieldwork” Involve in a Qualitative Project?
• Chapter 66: What Does It Mean to Gather “Rich Data”?
• Chapter 67: Do I Have to Transcribe All of My Interview Data or Can I Simply Transcribe a Few Quotes When I Need Them?
• Chapter 68: What Are the Pros and Cons of Audio or Video Recording My Participants?
• Chapter 69: Qualitative Research Seems to Involve a Lot of Talking to People. Sounds Easy—So What Issues Should I Expect if I’m Doing Formal or Informal Interviews With Individuals or Groups?
• Chapter 70: Do I Have to Work With My Participants in Person, or Can I Use the Internet (or Other Tools) to Gather Data at a Distance?
• Chapter 71: There Are Many Interviews and Other Potential Sources of Data Online, Including People’s Quotes Posted to Social Media and Websites. Can I Use These in My Qualitative Study?
• Chapter 72: I See That Some Qualitative Studies Use Participant-Generated Photographs, Drawings, and Other Arts-Based Approaches. When Is It Appropriate to Use These Kinds of Methods for Gathering Data?
• Chapter 73: I Have a Lot of Data—Dozens of Digital Data Files, Hundreds of Pages of Printed Transcripts, and Hours of Video-Recordings. How Can I Manage All of This Material?
• Chapter 74: How Do I Know When I’ve Reached Saturation of Themes in My Data?
• Chapter 75: My Colleague Says That There Are Many “Lost Opportunities” in His Dataset. What Does That Mean?
• Chapter 76: I’m Trying to Select the Best Site for Conducting Individual Interviews, So How Do I Choose?

Part 8: Conducting Qualitative Analysis

• Chapter 77: Do I Have to Wait Until My Data Collection Is Done Before I Can Start Analyzing My Data?
• Chapter 78: My Supervisor Says I Should Use an Interpretive Lens for My Analysis—What Does This Mean?
• Chapter 79: What Is the Process for “Coding” My Dataset? Can I BorrowSomeone Else’s Codebook to Get Me Started?
• Chapter 80: What Is the Difference Between “Themes,” “Codes,” and “Categories”?
• Chapter 81: Does the Person Who Gathered the Data Have to Be the Person Who Codes and Analyzes Those Data?
• Chapter 82: I’ve Heard Data Analysis Described as an Iterative Process of Coding. What Does That Mean?
• Chapter 83: How Can I Use a Team to Code Data?
• Chapter 84: Do Qualitative Researchers Count Things in Their Data, or Is This Only Done in Quantitative Research?
• Chapter 85: Will Using a Qualitative Data Analysis Software Package Improve the Quality of My Results?
• Chapter 86: What Is the Best Software Package to Use for Qualitative Data Analysis?
• Chapter 87: Does It Matter if Someone Else Interprets My Results in a Different Way?
• Chapter 88: I Have Anomalous Data. Is This a Problem?
• Chapter 89: I’ve Heard That I Need to Immerse Myself in the Data During Analysis, but I Have a Full-Time Job. How Much Time Do I Need to Devote to This Process?
• Chapter 90: My Analysis Seems to Raise More Questions Than Answers, So What Do I Do About This?

Part 9: Writing Qualitative Research

• Chapter 91: How Do I Present My Findings So That They Reflect Both My Analysis and the Participants’ Voices?
• Chapter 92: I’ve Assigned My Participants Numbers (to Keep Them Anonymous in the Writing) but Now My Supervisor Says I Should Give Them Pseudonyms Instead. Which Approach Is Best?
• Chapter 93: My Supervisor Says I Should “Give Voice” to My Research Participants in My Writing—What Does That Mean?
• Chapter 94: I Have Some Pictures, Audio-Recordings, and Other Multimedia Data, So How Can I Include These in Publications?
• Chapter 95: Journal Articles in My Discipline Are Limited in Length, So How Can I Present My Results in a Succinct Way While Providing Enough Detail to Support My Arguments?
• Chapter 96: I Have a Lot of Data and I’m Struggling to Fit Everything Into One Paper! How Can I Write Up My Results in a Single Research Report?
• Chapter 97: Qualitative Research Reports Are Published in Many Formats and Styles, Ranging From Traditional (i.e., With Results, Discussion, and Conclusions) to Progressive (e.g., Narrative Short Stories, Poetry, Plays). Which Approach Should I Choose for My Own Report?
• Chapter 98: What Kind of Audience Reads Qualitative Research Reports, Typically?
• Chapter 99: My Data Are Just Interview Transcripts and Other Texts, So How Can I Present My Findings in a Visual Poster Presentation?
• Chapter 100: Should I Send Copies of My Publications to My Participants?

Sign in to access this content

Get a 30 day free trial, more like this, sage recommends.

We found other relevant content for you on other Sage platforms.

Have you created a personal profile? Login or create a profile so that you can save clips, playlists and searches

• Sign in/register

Navigating away from this page will delete your results

Please save your results to "My Self-Assessments" in your profile before navigating away from this page.

Sign in to my profile

Sign up for a free trial and experience all Sage Learning Resources have to offer.

You must have a valid academic email address to sign up.

Get off-campus access

• View or download all content my institution has access to.

Sign up for a free trial and experience all Sage Research Methods has to offer.

• view my profile
• view my lists

• Request new password
• Create a new account

Doing Research in the Real World

Student resources, multiple choice quiz.

Take the quiz to test your understanding of the key concepts covered in the chapter. Try testing yourself before you read the chapter to see where your strengths and weaknesses are, then test yourself again once you’ve read the chapter to see how well you’ve understood.

Tip: Click on each link to expand and view the content. Click again to collapse.

PART A: PRINCIPLES AND PLANNING FOR RESEARCH

1. Which of the following should not be a criterion for a good research project?

• Demonstrates the abilities of the researcher
• Is dependent on the completion of other projects
• Demonstrates the integration of different fields of knowledge
• Develops the skills of the researcher

b.  Is dependent on the completion of other projects

2. Which form of reasoning is the process of drawing a specific conclusion from a set of premises?

• Objective reasoning
• Positivistic reasoning
• Inductive reasoning
• Deductive reasoning

d:  Deductive reasoning

3. Research that seeks to examine the findings of a study by using the same design but a different sample is which of the following?

• An exploratory study
• A replication study
• An empirical study
• Hypothesis testing

b:  A replication study

4. A researcher designs an experiment to test how variables interact to influence job-seeking behaviours. The main purpose of the study was:

• Description
• Exploration
• Explanation

d:  Explanation

5. Cyber bullying at work is a growing threat to employee job satisfaction. Researchers want to find out why people do this and how they feel about it. The primary purpose of the study is:

c:  Exploration

6. A theory: 

• Is an accumulated body of knowledge
• Includes inconsequential ideas
• Is independent of research methodology
• Should be viewed uncritically

a:  Is an accumulated body of knowledge

7. Which research method is a bottom-up approach to research?

• Deductive method
• Explanatory method
• Inductive method
• Exploratory method

c:  Inductive method

8. How much confidence should you place in a single research study?

• You should trust research findings after different researchers have replicated the findings
• You should completely trust a single research study
• Neither a nor b
• Both a and b 

a:  You should trust research findings after different researchers have replicated the findings

9. A qualitative research problem statement:

• Specifies the research methods to be utilized
• Specifies a research hypothesis
• Expresses a relationship between variables
• Conveys a sense of emerging design

d:  Conveys a sense of emerging design

10. Which of the following is a good research question?

• To produce a report on student job searching behaviours
• To identify the relationship between self-efficacy and student job searching behaviours
• Students with higher levels of self-efficacy will demonstrate more active job searching behaviours
• Do students with high levels of self-efficacy demonstrate more active job searching behaviours?

d:  Do students with high levels of self-efficacy demonstrate more active job searching behaviours?

11. A review of the literature prior to formulating research questions allows the researcher to :

• Provide an up-to-date understanding of the subject, its significance, and structure
• Guide the development of research questions
• Present the kinds of research methodologies used in previous studies
• All of the above

d:  All of the above

12. Sometimes a comprehensive review of the literature prior to data collection is not recommended by:

• Ethnomethodology
• Grounded theory
• Symbolic interactionism
• Feminist theory

b:  Grounded theory

13. The feasibility of a research study should be considered in light of: 

• Cost and time required to conduct the study
• Access to gatekeepers and respondents
• Potential ethical concerns

14. Research that uses qualitative methods for one phase and quantitative methods for the next phase is known as:

• Action research
• Mixed-method research
• Quantitative research
• Pragmatic research

b:  Mixed-method research

15. Research hypotheses are:

• Formulated prior to a review of the literature
• Statements of predicted relationships between variables
• B but not A
• Both A and B

c:  B but not A

16. Which research approach is based on the epistemological viewpoint of pragmatism? 

• Qualitative research
• Mixed-methods research

c:  Mixed-methods research

17. Adopting ethical principles in research means: 

• Avoiding harm to participants
• The researcher is anonymous
• Deception is only used when necessary
• Selected informants give their consent

a:  Avoiding harm to participants

18. A radical perspective on ethics suggests that: 

• Researchers can do anything they want
• The use of checklists of ethical actions is essential
• The powers of Institutional Review Boards should be strengthened
• Ethics should be based on self-reflexivity

d:  Ethics should be based on self-reflexivity

19. Ethical problems can arise when researching the Internet because:

• Everyone has access to digital media
• Respondents may fake their identities
• Researchers may fake their identities
• Internet research has to be covert

b:  Respondents may fake their identities

20. The Kappa statistic: 

• Is a measure of inter-judge validity
• Compares the level of agreement between two judges against what might have been predicted by chance
• Ranges from 0 to +1
• Is acceptable above a score of 0.5

b:  Compares the level of agreement between two judges against what might have been predicted by chance

PART B: RESEARCH METHODOLOGY  

1. Which research paradigm is most concerned about generalizing its findings? 

a:  Quantitative research

2. A variable that is presumed to cause a change in another variable is called:

• An intervening variable
• A dependent variable
• An independent variable
• A numerical variable

c:  An independent variable

3. A study of teaching professionals posits that their performance-related pay increases their motivation which in turn leads to an increase in their job satisfaction. What kind of variable is ‘motivation”’ in this study? 

• Extraneous 
• Confounding
• Intervening
• Manipulated

c:  Intervening

4. Which correlation is the strongest? 

5. When interpreting a correlation coefficient expressing the relationship between two variables, it is important not to:

• Assume causality
• Measure the values for X and Y independently
• Choose X and Y values that are normally distributed
• Check the direction of the relationship

a:  Assume causality

6. Which of the following can be described as a nominal variable? 

• Annual income
• Annual sales
• Geographical location of a firm

d:  Geographical location of a firm

7. A positive correlation occurs when:

• Two variables remain constant
• Two variables move in the same direction
• One variable goes up and the other goes down
• Two variables move in opposite directions

b:  Two variables move in the same direction

8. The key defining characteristic of experimental research is that:

• The independent variable is manipulated
• Hypotheses are proved
• A positive correlation exists
• Samples are large

a:  The independent variable is manipulated

9. Qualitative research is used in all the following circumstances, EXCEPT:

• It is based on a collection of non-numerical data such as words and pictures
• It often uses small samples
• It uses the inductive method
• It is typically used when a great deal is already known about the topic of interest

d:  It is typically used when a great deal is already known about the topic of interest

10. In an experiment, the group that does not receive the intervention is called:

• The experimental group
• The participant group
• The control group
• The treatment group

c:  The control group

11. Which generally cannot be guaranteed in conducting qualitative studies in the field? 

• Keeping participants from physical and emotional harm
• Gaining informed consent
• Assuring anonymity rather than just confidentiality
• Maintaining consent forms

c:  Assuring anonymity rather than just confidentiality

12. Which of the following is not ethical practice in research with humans? 

• Maintaining participants’ anonymity
• Informing participants that they are free to withdraw at any time
• Requiring participants to continue until the study has been completed

d:  Requiring participants to continue until the study has been completed

13. What do we call data that are used for a new study but which were collected by an earlier researcher for a different set of research questions?

• Secondary data
• Field notes
• Qualitative data
• Primary data

a:  Secondary data

14. When each member of a population has an equal chance of being selected, this is called:

• A snowball sample
• A stratified sample
• A random probability sample
• A non-random sample

c:  A random probability sample

15. Which of the following techniques yields a simple random sample of hospitals?

• Randomly selecting a district and then sampling all hospitals within the district
• Numbering all the elements of a hospital sampling frame and then using a random number generator to pick hospitals from the table
• Listing hospitals by sector and choosing a proportion from within each sector at random
• Choosing volunteer hospitals to participate

b:  Numbering all the elements of a hospital sampling frame and then using a random number generator to pick hospitals from the table

16. Which of the following statements are true?

• The larger the sample size, the larger the confidence interval
• The smaller the sample size, the greater the sampling error
• The more categories being measured, the smaller the sample size
• A confidence level of 95 percent is always sufficient

b:  The smaller the sample size, the greater the sampling error

17. Which of the following will produce the least sampling error?

• A large sample based on convenience sampling 
• A small sample based on random sampling
• A large snowball sample
• A large sample based on random sampling

d:  A large sample based on random sampling

18. When people are readily available, volunteer, or are easily recruited to the sample, this is called:

• Snowball sampling
• Convenience sampling
• Stratified sampling
• Random sampling

b:  Convenience sampling

19. In qualitative research, sampling that involves selecting diverse cases is referred to as:

• Typical-case sampling
• Critical-case sampling
• Intensity sampling
• Maximum variation sampling

d:  Maximum variation sampling

20. A test accurately indicates an employee’s scores on a future criterion (e.g., conscientiousness).  What kind of validity is this?

a:  Predictive

PART C: DATA COLLECTION METHODS  

1. When designing a questionnaire it is important to do each of the following EXCEPT

• Pilot the questionnaire
• Avoid jargon
• Avoid double questions
• Use leading questions

d:  Use leading questions

2. One advantage of using a questionnaire is that:

• Probe questions can be asked
• Respondents can be put at ease
• Interview bias can be avoided
• Response rates are always high

c:  Interview bias can be avoided

3. Which of the following is true of observations?

• It takes less time than interviews
• It is often not possible to determine exactly why people behave as they do
• Covert observation raises fewer ethical concerns than overt

b:  It is often not possible to determine exactly why people behave as they do

4. A researcher secretly becomes an active member of a group in order to observe their behaviour. This researcher is acting as:

• An overt participant observer
• A covert non-participant observer
• A covert participant observer
• None of the above

c:  A covert participant observer

5. All of the following are advantages of structured observation, EXCEPT:

• Results can be replicated at a different time
• The coding schedule might impose a framework on what is being observed
• Data can be collected that participants may not realize is important
• Data do not have to rely on the recall of participants

b:  The coding schedule might impose a framework on what is being observed

6. When conducting an interview, asking questions such as: "What else? or ‘Could you expand on that?’ are all forms of:

• Structured responses
• Category questions

7. Secondary data can include which of the following? 

• Government statistics
• Personal diaries
• Organizational records

8. An ordinal scale is:

• The simplest form of measurement
• A scale with an absolute zero point
• A rank-order scale of measurement
• A scale with equal intervals between ranks

c:  A rank-order scale of measurement

9. Which term measures the extent to which scores from a test can be used to infer or predict performance in some activity? 

• Face validity
• Content reliability
• Criterion-related validity
• Construct validity

c:  Criterion-related validity

10. The ‘reliability’of a measure refers to the researcher asking:

• Does it give consistent results?
• Does it measure what it is supposed to measure?
• Can the results be generalized?
• Does it have face reliability?

a:  Does it give consistent results?

11. Interviewing is the favoured approach EXCEPT when:

• There is a need for highly personalized data
• It is important to ask supplementary questions
• High numbers of respondents are needed
• Respondents have difficulty with written language

c:  High numbers of respondents are needed

12. Validity in interviews is strengthened by the following EXCEPT:

• Building rapport with interviewees
• Multiple questions cover the same theme
• Constructing interview schedules that contain themes drawn from the literature
• Prompting respondents to expand on initial responses

b:  Multiple questions cover the same theme

13. Interview questions should:

• Lead the respondent
• Probe sensitive issues
• Be delivered in a neutral tone
• Test the respondents’ powers of memory

c:  Be delivered in a neutral tone

14. Active listening skills means:

• Asking as many questions as possible
• Avoiding silences
• Keeping to time
• Attentive listening

d:  Attentive listening

15. All the following are strengths of focus groups EXCEPT:

• They allow access to a wide range of participants
• Discussion allows for the validation of ideas and views
• They can generate a collective perspective
• They help maintain confidentiality

d:  They help maintain confidentiality

16. Which of the following is not always true about focus groups?

• The ideal size is normally between 6 and 12 participants
• Moderators should introduce themselves to the group
• Participants should come from diverse backgrounds
• The moderator poses preplanned questions

c:  Participants should come from diverse backgrounds

17. A disadvantage of using secondary data is that:

• The data may have been collected with reference to research questions that are not those of the researcher
• The researcher may bring more detachment in viewing the data than original researchers could muster
• Data have often been collected by teams of experienced researchers
• Secondary data sets are often available and accessible

a:  The data may have been collected with reference to research questions that are not those of the researcher

18. All of the following are sources of secondary data EXCEPT:

• Official statistics
• A television documentary
• The researcher’s research diary
• A company’s annual report

c:  The researcher’s research diary

19. Which of the following is not true about visual methods?

• They are not reliant on respondent recall
• The have low resource requirements
• They do not rely on words to capture what is happening
• They can capture what is happening in real time

b:  The have low resource requirements

20. Avoiding naïve empiricism in the interpretation of visual data means:

• Understanding the context in which they were produced
• Ensuring that visual images such as photographs are accurately taken
• Only using visual images with other data gathering sources
• Planning the capture of visual data carefully

a:  Understanding the context in which they were produced

PART D: ANALYSIS AND REPORT WRITING  

1. Which of the following is incorrect when naming a variable in SPSS?

• Must begin with a letter and not a number
• Must end in a full stop
• Cannot exceed 64 characters
• Cannot include symbols such as ?, & and %

b:  Must end in a full stop

2. Which of the following is not an SPSS Type variable?

3. A graph that uses vertical bars to represent data is called:

• A bar chart
• A pie chart
• A line graph
• A vertical graph

a:  A bar chart

4. The purpose of descriptive statistics is to:

• Summarize the characteristics of a data set
• Draw conclusions from the data

a:  Summarize the characteristics of a data set

5. The measure of the extent to which responses vary from the mean is called:

• The normal distribution
• The standard deviation
• The variance

c:  The standard deviation

6. To compare the performance of a group at time T1 and then at T2, we would use:

• A chi-squared test
• One-way analysis of variance
• Analysis of variance
• A paired t-test

d:  A paired t-test

7. A Type 1 error occurs in a situation where:

• The null hypothesis is accepted when it is in fact true
• The null hypothesis is rejected when it is in fact false
• The null hypothesis is rejected when it is in fact true
• The null hypothesis is accepted when it is in fact false

c:  The null hypothesis is rejected when it is in fact true

8. The significance level

• Is set after a statistical test is conducted
• Is always set at 0.05
• Results in a p -value
• Measures the probability of rejecting a true null hypothesis

d:  Measures the probability of rejecting a true null hypothesis

9. To predict the value of the dependent variable for a new case based on the knowledge of one or more independent variables, we would use

• Regression analysis
• Correlation analysis
• Kolmogorov-Smirnov test

a:  Regression analysis

10. In conducting secondary data analysis, researchers should ask themselves all of the following EXCEPT:

• Who produced the document?
• Is the material genuine?
• How can respondents be re-interviewed?
• Why was the document produced?

c:  How can respondents be re-interviewed?

11. Which of the following are not true of reflexivity?

• It recognizes that the researcher is not a neutral observer
• It has mainly been applied to the analysis of qualitative data
• It is part of a post-positivist tradition
• A danger of adopting a reflexive stance is the researcher can become the focus of the study

c:  It is part of a post-positivist tradition

12. Validity in qualitative research can be strengthened by all of the following EXCEPT:

• Member checking for accuracy and interpretation
• Transcribing interviews to improve accuracy of data
• Exploring rival explanations
• Analysing negative cases

b:  Transcribing interviews to improve accuracy of data

13. Qualitative data analysis programs are useful for each of the following EXCEPT: 

• Manipulation of large amounts of data
• Exploring of the data against new dimensions
• Querying of data
• Generating codes

d:  Generating codes

14. Which part of a research report contains details of how the research was planned and conducted?

• Introduction

b:  Design 

15. Which of the following is a form of research typically conducted by managers and other professionals to address issues in their organizations and/or professional practice?

• Basic research
• Professional research
• Predictive research

a:  Action research

16. Plagiarism can be avoided by:

• Copying the work of others accurately
• Paraphrasing the author’s text in your own words
• Cut and pasting from the Internet
• Quoting directly without revealing the source

b:  Paraphrasing the author’s text in your own words

17. In preparing for a presentation, you should do all of the following EXCEPT:

• Practice the presentation
• Ignore your nerves
• Get to know more about your audience
• Take an advanced look, if possible, at the facilities

b:  Ignore your nerves

18. You can create interest in your presentation by:

• Using bullet points
• Reading from notes
• Maximizing the use of animation effects
• Using metaphors

d:  Using metaphors

19. In preparing for a viva or similar oral examination, it is best if you have:

• Avoided citing the examiner in your thesis
• Made exaggerated claims on the basis of your data
• Published and referenced your own article(s)
• Tried to memorize your work

c:  Published and referenced your own article(s)

20. Grounded theory coding:

• Makes use of a priori concepts from the literature
• Uses open coding, selective coding, then axial coding
• Adopts a deductive stance
• Stops when theoretical saturation has been reached

d:  Stops when theoretical saturation has been reached

• Privacy Policy

Buy Me a Coffee

Home » Qualitative Research – Methods, Analysis Types and Guide

Qualitative Research – Methods, Analysis Types and Guide

Table of Contents

Qualitative Research

Qualitative research is a type of research methodology that focuses on exploring and understanding people’s beliefs, attitudes, behaviors, and experiences through the collection and analysis of non-numerical data. It seeks to answer research questions through the examination of subjective data, such as interviews, focus groups, observations, and textual analysis.

Qualitative research aims to uncover the meaning and significance of social phenomena, and it typically involves a more flexible and iterative approach to data collection and analysis compared to quantitative research. Qualitative research is often used in fields such as sociology, anthropology, psychology, and education.

Qualitative Research Methods

Qualitative Research Methods are as follows:

One-to-One Interview

This method involves conducting an interview with a single participant to gain a detailed understanding of their experiences, attitudes, and beliefs. One-to-one interviews can be conducted in-person, over the phone, or through video conferencing. The interviewer typically uses open-ended questions to encourage the participant to share their thoughts and feelings. One-to-one interviews are useful for gaining detailed insights into individual experiences.

Focus Groups

This method involves bringing together a group of people to discuss a specific topic in a structured setting. The focus group is led by a moderator who guides the discussion and encourages participants to share their thoughts and opinions. Focus groups are useful for generating ideas and insights, exploring social norms and attitudes, and understanding group dynamics.

Ethnographic Studies

This method involves immersing oneself in a culture or community to gain a deep understanding of its norms, beliefs, and practices. Ethnographic studies typically involve long-term fieldwork and observation, as well as interviews and document analysis. Ethnographic studies are useful for understanding the cultural context of social phenomena and for gaining a holistic understanding of complex social processes.

Text Analysis

This method involves analyzing written or spoken language to identify patterns and themes. Text analysis can be quantitative or qualitative. Qualitative text analysis involves close reading and interpretation of texts to identify recurring themes, concepts, and patterns. Text analysis is useful for understanding media messages, public discourse, and cultural trends.

This method involves an in-depth examination of a single person, group, or event to gain an understanding of complex phenomena. Case studies typically involve a combination of data collection methods, such as interviews, observations, and document analysis, to provide a comprehensive understanding of the case. Case studies are useful for exploring unique or rare cases, and for generating hypotheses for further research.

Process of Observation

This method involves systematically observing and recording behaviors and interactions in natural settings. The observer may take notes, use audio or video recordings, or use other methods to document what they see. Process of observation is useful for understanding social interactions, cultural practices, and the context in which behaviors occur.

Record Keeping

This method involves keeping detailed records of observations, interviews, and other data collected during the research process. Record keeping is essential for ensuring the accuracy and reliability of the data, and for providing a basis for analysis and interpretation.

This method involves collecting data from a large sample of participants through a structured questionnaire. Surveys can be conducted in person, over the phone, through mail, or online. Surveys are useful for collecting data on attitudes, beliefs, and behaviors, and for identifying patterns and trends in a population.

Qualitative data analysis is a process of turning unstructured data into meaningful insights. It involves extracting and organizing information from sources like interviews, focus groups, and surveys. The goal is to understand people’s attitudes, behaviors, and motivations

Qualitative Research Analysis Methods

Qualitative Research analysis methods involve a systematic approach to interpreting and making sense of the data collected in qualitative research. Here are some common qualitative data analysis methods:

Thematic Analysis

This method involves identifying patterns or themes in the data that are relevant to the research question. The researcher reviews the data, identifies keywords or phrases, and groups them into categories or themes. Thematic analysis is useful for identifying patterns across multiple data sources and for generating new insights into the research topic.

Content Analysis

This method involves analyzing the content of written or spoken language to identify key themes or concepts. Content analysis can be quantitative or qualitative. Qualitative content analysis involves close reading and interpretation of texts to identify recurring themes, concepts, and patterns. Content analysis is useful for identifying patterns in media messages, public discourse, and cultural trends.

Discourse Analysis

This method involves analyzing language to understand how it constructs meaning and shapes social interactions. Discourse analysis can involve a variety of methods, such as conversation analysis, critical discourse analysis, and narrative analysis. Discourse analysis is useful for understanding how language shapes social interactions, cultural norms, and power relationships.

Grounded Theory Analysis

This method involves developing a theory or explanation based on the data collected. Grounded theory analysis starts with the data and uses an iterative process of coding and analysis to identify patterns and themes in the data. The theory or explanation that emerges is grounded in the data, rather than preconceived hypotheses. Grounded theory analysis is useful for understanding complex social phenomena and for generating new theoretical insights.

Narrative Analysis

This method involves analyzing the stories or narratives that participants share to gain insights into their experiences, attitudes, and beliefs. Narrative analysis can involve a variety of methods, such as structural analysis, thematic analysis, and discourse analysis. Narrative analysis is useful for understanding how individuals construct their identities, make sense of their experiences, and communicate their values and beliefs.

Phenomenological Analysis

This method involves analyzing how individuals make sense of their experiences and the meanings they attach to them. Phenomenological analysis typically involves in-depth interviews with participants to explore their experiences in detail. Phenomenological analysis is useful for understanding subjective experiences and for developing a rich understanding of human consciousness.

Comparative Analysis

This method involves comparing and contrasting data across different cases or groups to identify similarities and differences. Comparative analysis can be used to identify patterns or themes that are common across multiple cases, as well as to identify unique or distinctive features of individual cases. Comparative analysis is useful for understanding how social phenomena vary across different contexts and groups.

Applications of Qualitative Research

Qualitative research has many applications across different fields and industries. Here are some examples of how qualitative research is used:

• Market Research: Qualitative research is often used in market research to understand consumer attitudes, behaviors, and preferences. Researchers conduct focus groups and one-on-one interviews with consumers to gather insights into their experiences and perceptions of products and services.
• Health Care: Qualitative research is used in health care to explore patient experiences and perspectives on health and illness. Researchers conduct in-depth interviews with patients and their families to gather information on their experiences with different health care providers and treatments.
• Education: Qualitative research is used in education to understand student experiences and to develop effective teaching strategies. Researchers conduct classroom observations and interviews with students and teachers to gather insights into classroom dynamics and instructional practices.
• Social Work : Qualitative research is used in social work to explore social problems and to develop interventions to address them. Researchers conduct in-depth interviews with individuals and families to understand their experiences with poverty, discrimination, and other social problems.
• Anthropology : Qualitative research is used in anthropology to understand different cultures and societies. Researchers conduct ethnographic studies and observe and interview members of different cultural groups to gain insights into their beliefs, practices, and social structures.
• Psychology : Qualitative research is used in psychology to understand human behavior and mental processes. Researchers conduct in-depth interviews with individuals to explore their thoughts, feelings, and experiences.
• Public Policy : Qualitative research is used in public policy to explore public attitudes and to inform policy decisions. Researchers conduct focus groups and one-on-one interviews with members of the public to gather insights into their perspectives on different policy issues.

How to Conduct Qualitative Research

Here are some general steps for conducting qualitative research:

• Identify your research question: Qualitative research starts with a research question or set of questions that you want to explore. This question should be focused and specific, but also broad enough to allow for exploration and discovery.
• Select your research design: There are different types of qualitative research designs, including ethnography, case study, grounded theory, and phenomenology. You should select a design that aligns with your research question and that will allow you to gather the data you need to answer your research question.
• Recruit participants: Once you have your research question and design, you need to recruit participants. The number of participants you need will depend on your research design and the scope of your research. You can recruit participants through advertisements, social media, or through personal networks.
• Collect data: There are different methods for collecting qualitative data, including interviews, focus groups, observation, and document analysis. You should select the method or methods that align with your research design and that will allow you to gather the data you need to answer your research question.
• Analyze data: Once you have collected your data, you need to analyze it. This involves reviewing your data, identifying patterns and themes, and developing codes to organize your data. You can use different software programs to help you analyze your data, or you can do it manually.
• Interpret data: Once you have analyzed your data, you need to interpret it. This involves making sense of the patterns and themes you have identified, and developing insights and conclusions that answer your research question. You should be guided by your research question and use your data to support your conclusions.
• Communicate results: Once you have interpreted your data, you need to communicate your results. This can be done through academic papers, presentations, or reports. You should be clear and concise in your communication, and use examples and quotes from your data to support your findings.

Examples of Qualitative Research

Here are some real-time examples of qualitative research:

• Customer Feedback: A company may conduct qualitative research to understand the feedback and experiences of its customers. This may involve conducting focus groups or one-on-one interviews with customers to gather insights into their attitudes, behaviors, and preferences.
• Healthcare : A healthcare provider may conduct qualitative research to explore patient experiences and perspectives on health and illness. This may involve conducting in-depth interviews with patients and their families to gather information on their experiences with different health care providers and treatments.
• Education : An educational institution may conduct qualitative research to understand student experiences and to develop effective teaching strategies. This may involve conducting classroom observations and interviews with students and teachers to gather insights into classroom dynamics and instructional practices.
• Social Work: A social worker may conduct qualitative research to explore social problems and to develop interventions to address them. This may involve conducting in-depth interviews with individuals and families to understand their experiences with poverty, discrimination, and other social problems.
• Anthropology : An anthropologist may conduct qualitative research to understand different cultures and societies. This may involve conducting ethnographic studies and observing and interviewing members of different cultural groups to gain insights into their beliefs, practices, and social structures.
• Psychology : A psychologist may conduct qualitative research to understand human behavior and mental processes. This may involve conducting in-depth interviews with individuals to explore their thoughts, feelings, and experiences.
• Public Policy: A government agency or non-profit organization may conduct qualitative research to explore public attitudes and to inform policy decisions. This may involve conducting focus groups and one-on-one interviews with members of the public to gather insights into their perspectives on different policy issues.

Purpose of Qualitative Research

The purpose of qualitative research is to explore and understand the subjective experiences, behaviors, and perspectives of individuals or groups in a particular context. Unlike quantitative research, which focuses on numerical data and statistical analysis, qualitative research aims to provide in-depth, descriptive information that can help researchers develop insights and theories about complex social phenomena.

Qualitative research can serve multiple purposes, including:

• Exploring new or emerging phenomena : Qualitative research can be useful for exploring new or emerging phenomena, such as new technologies or social trends. This type of research can help researchers develop a deeper understanding of these phenomena and identify potential areas for further study.
• Understanding complex social phenomena : Qualitative research can be useful for exploring complex social phenomena, such as cultural beliefs, social norms, or political processes. This type of research can help researchers develop a more nuanced understanding of these phenomena and identify factors that may influence them.
• Generating new theories or hypotheses: Qualitative research can be useful for generating new theories or hypotheses about social phenomena. By gathering rich, detailed data about individuals’ experiences and perspectives, researchers can develop insights that may challenge existing theories or lead to new lines of inquiry.
• Providing context for quantitative data: Qualitative research can be useful for providing context for quantitative data. By gathering qualitative data alongside quantitative data, researchers can develop a more complete understanding of complex social phenomena and identify potential explanations for quantitative findings.

When to use Qualitative Research

Here are some situations where qualitative research may be appropriate:

• Exploring a new area: If little is known about a particular topic, qualitative research can help to identify key issues, generate hypotheses, and develop new theories.
• Understanding complex phenomena: Qualitative research can be used to investigate complex social, cultural, or organizational phenomena that are difficult to measure quantitatively.
• Investigating subjective experiences: Qualitative research is particularly useful for investigating the subjective experiences of individuals or groups, such as their attitudes, beliefs, values, or emotions.
• Conducting formative research: Qualitative research can be used in the early stages of a research project to develop research questions, identify potential research participants, and refine research methods.
• Evaluating interventions or programs: Qualitative research can be used to evaluate the effectiveness of interventions or programs by collecting data on participants’ experiences, attitudes, and behaviors.

Characteristics of Qualitative Research

Qualitative research is characterized by several key features, including:

• Focus on subjective experience: Qualitative research is concerned with understanding the subjective experiences, beliefs, and perspectives of individuals or groups in a particular context. Researchers aim to explore the meanings that people attach to their experiences and to understand the social and cultural factors that shape these meanings.
• Use of open-ended questions: Qualitative research relies on open-ended questions that allow participants to provide detailed, in-depth responses. Researchers seek to elicit rich, descriptive data that can provide insights into participants’ experiences and perspectives.
• Sampling-based on purpose and diversity: Qualitative research often involves purposive sampling, in which participants are selected based on specific criteria related to the research question. Researchers may also seek to include participants with diverse experiences and perspectives to capture a range of viewpoints.
• Data collection through multiple methods: Qualitative research typically involves the use of multiple data collection methods, such as in-depth interviews, focus groups, and observation. This allows researchers to gather rich, detailed data from multiple sources, which can provide a more complete picture of participants’ experiences and perspectives.
• Inductive data analysis: Qualitative research relies on inductive data analysis, in which researchers develop theories and insights based on the data rather than testing pre-existing hypotheses. Researchers use coding and thematic analysis to identify patterns and themes in the data and to develop theories and explanations based on these patterns.
• Emphasis on researcher reflexivity: Qualitative research recognizes the importance of the researcher’s role in shaping the research process and outcomes. Researchers are encouraged to reflect on their own biases and assumptions and to be transparent about their role in the research process.

Advantages of Qualitative Research

Qualitative research offers several advantages over other research methods, including:

• Depth and detail: Qualitative research allows researchers to gather rich, detailed data that provides a deeper understanding of complex social phenomena. Through in-depth interviews, focus groups, and observation, researchers can gather detailed information about participants’ experiences and perspectives that may be missed by other research methods.
• Flexibility : Qualitative research is a flexible approach that allows researchers to adapt their methods to the research question and context. Researchers can adjust their research methods in real-time to gather more information or explore unexpected findings.
• Contextual understanding: Qualitative research is well-suited to exploring the social and cultural context in which individuals or groups are situated. Researchers can gather information about cultural norms, social structures, and historical events that may influence participants’ experiences and perspectives.
• Participant perspective : Qualitative research prioritizes the perspective of participants, allowing researchers to explore subjective experiences and understand the meanings that participants attach to their experiences.
• Theory development: Qualitative research can contribute to the development of new theories and insights about complex social phenomena. By gathering rich, detailed data and using inductive data analysis, researchers can develop new theories and explanations that may challenge existing understandings.
• Validity : Qualitative research can offer high validity by using multiple data collection methods, purposive and diverse sampling, and researcher reflexivity. This can help ensure that findings are credible and trustworthy.

Limitations of Qualitative Research

Qualitative research also has some limitations, including:

• Subjectivity : Qualitative research relies on the subjective interpretation of researchers, which can introduce bias into the research process. The researcher’s perspective, beliefs, and experiences can influence the way data is collected, analyzed, and interpreted.
• Limited generalizability: Qualitative research typically involves small, purposive samples that may not be representative of larger populations. This limits the generalizability of findings to other contexts or populations.
• Time-consuming: Qualitative research can be a time-consuming process, requiring significant resources for data collection, analysis, and interpretation.
• Resource-intensive: Qualitative research may require more resources than other research methods, including specialized training for researchers, specialized software for data analysis, and transcription services.
• Limited reliability: Qualitative research may be less reliable than quantitative research, as it relies on the subjective interpretation of researchers. This can make it difficult to replicate findings or compare results across different studies.
• Ethics and confidentiality: Qualitative research involves collecting sensitive information from participants, which raises ethical concerns about confidentiality and informed consent. Researchers must take care to protect the privacy and confidentiality of participants and obtain informed consent.

Also see Research Methods

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Questionnaire – Definition, Types, and Examples

Case Study – Methods, Examples and Guide

Observational Research – Methods and Guide

Quantitative Research – Methods, Types and...

Qualitative Research Methods

Explanatory Research – Types, Methods, Guide

Qualitative Research Questions: Gain Powerful Insights + 25 Examples

We review the basics of qualitative research questions, including their key components, how to craft them effectively, & 25 example questions.

Einstein was many things—a physicist, a philosopher, and, undoubtedly, a mastermind. He also had an incredible way with words. His quote, "Everything that can be counted does not necessarily count; everything that counts cannot necessarily be counted," is particularly poignant when it comes to research. 

Some inquiries call for a quantitative approach, for counting and measuring data in order to arrive at general conclusions. Other investigations, like qualitative research, rely on deep exploration and understanding of individual cases in order to develop a greater understanding of the whole. That’s what we’re going to focus on today.

Qualitative research questions focus on the "how" and "why" of things, rather than the "what". They ask about people's experiences and perceptions , and can be used to explore a wide range of topics.

The following article will discuss the basics of qualitative research questions, including their key components, and how to craft them effectively. You'll also find 25 examples of effective qualitative research questions you can use as inspiration for your own studies.

Let’s get started!

What are qualitative research questions, and when are they used?

When researchers set out to conduct a study on a certain topic, their research is chiefly directed by an overarching question . This question provides focus for the study and helps determine what kind of data will be collected.

By starting with a question, we gain parameters and objectives for our line of research. What are we studying? For what purpose? How will we know when we’ve achieved our goals?

Of course, some of these questions can be described as quantitative in nature. When a research question is quantitative, it usually seeks to measure or calculate something in a systematic way.

For example:

• How many people in our town use the library?
• What is the average income of families in our city?
• How much does the average person weigh?

Other research questions, however—and the ones we will be focusing on in this article—are qualitative in nature. Qualitative research questions are open-ended and seek to explore a given topic in-depth.

According to the Australian & New Zealand Journal of Psychiatry , “Qualitative research aims to address questions concerned with developing an understanding of the meaning and experience dimensions of humans’ lives and social worlds.”

This type of research can be used to gain a better understanding of people’s thoughts, feelings and experiences by “addressing questions beyond ‘what works’, towards ‘what works for whom when, how and why, and focusing on intervention improvement rather than accreditation,” states one paper in Neurological Research and Practice .

Qualitative questions often produce rich data that can help researchers develop hypotheses for further quantitative study.

• What are people’s thoughts on the new library?
• How does it feel to be a first-generation student at our school?
• How do people feel about the changes taking place in our town?

As stated by a paper in Human Reproduction , “...‘qualitative’ methods are used to answer questions about experience, meaning, and perspective, most often from the standpoint of the participant. These data are usually not amenable to counting or measuring.”

Both quantitative and qualitative questions have their uses; in fact, they often complement each other. A well-designed research study will include a mix of both types of questions in order to gain a fuller understanding of the topic at hand.

If you would like to recruit unlimited participants for qualitative research for free and only pay for the interview you conduct, try using Respondent  today. 

Crafting qualitative research questions for powerful insights

Now that we have a basic understanding of what qualitative research questions are and when they are used, let’s take a look at how you can begin crafting your own.

According to a study in the International Journal of Qualitative Studies in Education, there is a certain process researchers should follow when crafting their questions, which we’ll explore in more depth.

1. Beginning the process 

Start with a point of interest or curiosity, and pose a draft question or ‘self-question’. What do you want to know about the topic at hand? What is your specific curiosity? You may find it helpful to begin by writing several questions.

For example, if you’re interested in understanding how your customer base feels about a recent change to your product, you might ask: 

• What made you decide to try the new product?
• How do you feel about the change?
• What do you think of the new design/functionality?
• What benefits do you see in the change?

2. Create one overarching, guiding question 

At this point, narrow down the draft questions into one specific question. “Sometimes, these broader research questions are not stated as questions, but rather as goals for the study.”

As an example of this, you might narrow down these three questions: 

into the following question: 

• What are our customers’ thoughts on the recent change to our product?

3. Theoretical framing 

As you read the relevant literature and apply theory to your research, the question should be altered to achieve better outcomes. Experts agree that pursuing a qualitative line of inquiry should open up the possibility for questioning your original theories and altering the conceptual framework with which the research began.

If we continue with the current example, it’s possible you may uncover new data that informs your research and changes your question. For instance, you may discover that customers’ feelings about the change are not just a reaction to the change itself, but also to how it was implemented. In this case, your question would need to reflect this new information: 

• How did customers react to the process of the change, as well as the change itself?

4. Ethical considerations 

A study in the International Journal of Qualitative Studies in Education stresses that ethics are “a central issue when a researcher proposes to study the lives of others, especially marginalized populations.” Consider how your question or inquiry will affect the people it relates to—their lives and their safety. Shape your question to avoid physical, emotional, or mental upset for the focus group.

In analyzing your question from this perspective, if you feel that it may cause harm, you should consider changing the question or ending your research project. Perhaps you’ve discovered that your question encourages harmful or invasive questioning, in which case you should reformulate it.

5. Writing the question 

The actual process of writing the question comes only after considering the above points. The purpose of crafting your research questions is to delve into what your study is specifically about” Remember that qualitative research questions are not trying to find the cause of an effect, but rather to explore the effect itself.

Your questions should be clear, concise, and understandable to those outside of your field. In addition, they should generate rich data. The questions you choose will also depend on the type of research you are conducting: 

• If you’re doing a phenomenological study, your questions might be open-ended, in order to allow participants to share their experiences in their own words.
• If you’re doing a grounded-theory study, your questions might be focused on generating a list of categories or themes.
• If you’re doing ethnography, your questions might be about understanding the culture you’re studying.

Whenyou have well-written questions, it is much easier to develop your research design and collect data that accurately reflects your inquiry.

In writing your questions, it may help you to refer to this simple flowchart process for constructing questions:

Download Free E-Book 

25 examples of expertly crafted qualitative research questions

It's easy enough to cover the theory of writing a qualitative research question, but sometimes it's best if you can see the process in practice. In this section, we'll list 25 examples of B2B and B2C-related qualitative questions.

Let's begin with five questions. We'll show you the question, explain why it's considered qualitative, and then give you an example of how it can be used in research.

1. What is the customer's perception of our company's brand?

Qualitative research questions are often open-ended and invite respondents to share their thoughts and feelings on a subject. This question is qualitative because it seeks customer feedback on the company's brand. 

This question can be used in research to understand how customers feel about the company's branding, what they like and don't like about it, and whether they would recommend it to others.

2. Why do customers buy our product?

This question is also qualitative because it seeks to understand the customer's motivations for purchasing a product. It can be used in research to identify the reasons  customers buy a certain product, what needs or desires the product fulfills for them, and how they feel about the purchase after using the product.

3. How do our customers interact with our products?

Again, this question is qualitative because it seeks to understand customer behavior. In this case, it can be used in research to see how customers use the product, how they interact with it, and what emotions or thoughts the product evokes in them.

4. What are our customers' biggest frustrations with our products?

By seeking to understand customer frustrations, this question is qualitative and can provide valuable insights. It can be used in research to help identify areas in which the company needs to make improvements with its products.

5. How do our customers feel about our customer service?

Rather than asking why customers like or dislike something, this question asks how they feel. This qualitative question can provide insights into customer satisfaction or dissatisfaction with a company. 

This type of question can be used in research to understand what customers think of the company's customer service and whether they feel it meets their needs.

20 more examples to refer to when writing your question

Now that you’re aware of what makes certain questions qualitative, let's move into 20 more examples of qualitative research questions:

• How do your customers react when updates are made to your app interface?
• How do customers feel when they complete their purchase through your ecommerce site?
• What are your customers' main frustrations with your service?
• How do people feel about the quality of your products compared to those of your competitors?
• What motivates customers to refer their friends and family members to your product or service?
• What are the main benefits your customers receive from using your product or service?
• How do people feel when they finish a purchase on your website?
• What are the main motivations behind customer loyalty to your brand?
• How does your app make people feel emotionally?
• For younger generations using your app, how does it make them feel about themselves?
• What reputation do people associate with your brand?
• How inclusive do people find your app?
• In what ways are your customers' experiences unique to them?
• What are the main areas of improvement your customers would like to see in your product or service?
• How do people feel about their interactions with your tech team?
• What are the top five reasons people use your online marketplace?
• How does using your app make people feel in terms of connectedness?
• What emotions do people experience when they're using your product or service?
• Aside from the features of your product, what else about it attracts customers?
• How does your company culture make people feel?

As you can see, these kinds of questions are completely open-ended. In a way, they allow the research and discoveries made along the way to direct the research. The questions are merely a starting point from which to explore.

This video offers tips on how to write good qualitative research questions, produced by Qualitative Research Expert, Kimberly Baker.

Wrap-up: crafting your own qualitative research questions.

Over the course of this article, we've explored what qualitative research questions are, why they matter, and how they should be written. Hopefully you now have a clear understanding of how to craft your own.

Remember, qualitative research questions should always be designed to explore a certain experience or phenomena in-depth, in order to generate powerful insights. As you write your questions, be sure to keep the following in mind:

• Are you being inclusive of all relevant perspectives?
• Are your questions specific enough to generate clear answers?
• Will your questions allow for an in-depth exploration of the topic at hand?
• Do the questions reflect your research goals and objectives?

If you can answer "yes" to all of the questions above, and you've followed the tips for writing qualitative research questions we shared in this article, then you're well on your way to crafting powerful queries that will yield valuable insights.

Download Free E-Book

Asking the right questions in the right way is the key to research success. That’s true for not just the discussion guide but for every step of a research project. Following are 100+ questions that will take you from defining your research objective through  screening and participant discussions.

Fill out the form below to access free e-book! 

Recommend Resources:

• How to Recruit Participants for Qualitative Research
• The Best UX Research Tools of 2022
• 10 Smart Tips for Conducting Better User Interviews
• 50 Powerful Questions You Should Ask In Your Next User Interview
• How To Find Participants For User Research: 13 Ways To Make It Happen
• UX Diary Study: 5 Essential Tips For Conducing Better Studies
• User Testing Recruitment: 10 Smart Tips To Find Participants Fast
• Qualitative Research Questions: Gain Powerful Insights + 25
• How To Successfully Recruit Participants for A Study (2022 Edition)
• How To Properly Recruit Focus Group Participants (2022 Edition)
• The Best Unmoderated Usability Testing Tools of 2022

50 Powerful User Interview Questions You Should Consider Asking

We researched the best user interview questions you can use for your qualitative research studies. Use these 50 sample questions for your next...

Understanding Why High-Quality Research Needs High-Quality Participants

Why are high-quality participants essential to your research? Read here to find out who they are, why you need them, and how to find them.

How To ​​Unleash Your Extra Income Potential With Respondent

The number one question we get from new participants is “how can I get invited to participate in more projects.” In this article, we’ll discuss a few...

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Qualitative vs. Quantitative Research | Differences, Examples & Methods

Qualitative vs. Quantitative Research | Differences, Examples & Methods

Published on April 12, 2019 by Raimo Streefkerk . Revised on June 22, 2023.

When collecting and analyzing data, quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings. Both are important for gaining different kinds of knowledge.

Common quantitative methods include experiments, observations recorded as numbers, and surveys with closed-ended questions.

Quantitative research is at risk for research biases including information bias , omitted variable bias , sampling bias , or selection bias . Qualitative research Qualitative research is expressed in words . It is used to understand concepts, thoughts or experiences. This type of research enables you to gather in-depth insights on topics that are not well understood.

Common qualitative methods include interviews with open-ended questions, observations described in words, and literature reviews that explore concepts and theories.

Table of contents

The differences between quantitative and qualitative research, data collection methods, when to use qualitative vs. quantitative research, how to analyze qualitative and quantitative data, other interesting articles, frequently asked questions about qualitative and quantitative research.

Quantitative and qualitative research use different research methods to collect and analyze data, and they allow you to answer different kinds of research questions.

Quantitative and qualitative data can be collected using various methods. It is important to use a data collection method that will help answer your research question(s).

Many data collection methods can be either qualitative or quantitative. For example, in surveys, observational studies or case studies , your data can be represented as numbers (e.g., using rating scales or counting frequencies) or as words (e.g., with open-ended questions or descriptions of what you observe).

However, some methods are more commonly used in one type or the other.

Quantitative data collection methods

• Surveys :  List of closed or multiple choice questions that is distributed to a sample (online, in person, or over the phone).
• Experiments : Situation in which different types of variables are controlled and manipulated to establish cause-and-effect relationships.
• Observations : Observing subjects in a natural environment where variables can’t be controlled.

Qualitative data collection methods

• Interviews : Asking open-ended questions verbally to respondents.
• Focus groups : Discussion among a group of people about a topic to gather opinions that can be used for further research.
• Ethnography : Participating in a community or organization for an extended period of time to closely observe culture and behavior.
• Literature review : Survey of published works by other authors.

A rule of thumb for deciding whether to use qualitative or quantitative data is:

• Use quantitative research if you want to confirm or test something (a theory or hypothesis )
• Use qualitative research if you want to understand something (concepts, thoughts, experiences)

For most research topics you can choose a qualitative, quantitative or mixed methods approach . Which type you choose depends on, among other things, whether you’re taking an inductive vs. deductive research approach ; your research question(s) ; whether you’re doing experimental , correlational , or descriptive research ; and practical considerations such as time, money, availability of data, and access to respondents.

Quantitative research approach

You survey 300 students at your university and ask them questions such as: “on a scale from 1-5, how satisfied are your with your professors?”

You can perform statistical analysis on the data and draw conclusions such as: “on average students rated their professors 4.4”.

Qualitative research approach

You conduct in-depth interviews with 15 students and ask them open-ended questions such as: “How satisfied are you with your studies?”, “What is the most positive aspect of your study program?” and “What can be done to improve the study program?”

Based on the answers you get you can ask follow-up questions to clarify things. You transcribe all interviews using transcription software and try to find commonalities and patterns.

Mixed methods approach

You conduct interviews to find out how satisfied students are with their studies. Through open-ended questions you learn things you never thought about before and gain new insights. Later, you use a survey to test these insights on a larger scale.

It’s also possible to start with a survey to find out the overall trends, followed by interviews to better understand the reasons behind the trends.

Qualitative or quantitative data by itself can’t prove or demonstrate anything, but has to be analyzed to show its meaning in relation to the research questions. The method of analysis differs for each type of data.

Analyzing quantitative data

Quantitative data is based on numbers. Simple math or more advanced statistical analysis is used to discover commonalities or patterns in the data. The results are often reported in graphs and tables.

Applications such as Excel, SPSS, or R can be used to calculate things like:

• Average scores ( means )
• The number of times a particular answer was given
• The correlation or causation between two or more variables
• The reliability and validity of the results

Analyzing qualitative data

Qualitative data is more difficult to analyze than quantitative data. It consists of text, images or videos instead of numbers.

Some common approaches to analyzing qualitative data include:

• Qualitative content analysis : Tracking the occurrence, position and meaning of words or phrases
• Thematic analysis : Closely examining the data to identify the main themes and patterns
• Discourse analysis : Studying how communication works in social contexts

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Chi square goodness of fit test
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Inclusion and exclusion criteria

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to systematically measure variables and test hypotheses . Qualitative methods allow you to explore concepts and experiences in more detail.

In mixed methods research , you use both qualitative and quantitative data collection and analysis methods to answer your research question .

The research methods you use depend on the type of data you need to answer your research question .

• If you want to measure something or test a hypothesis , use quantitative methods . If you want to explore ideas, thoughts and meanings, use qualitative methods .
• If you want to analyze a large amount of readily-available data, use secondary data. If you want data specific to your purposes with control over how it is generated, collect primary data.
• If you want to establish cause-and-effect relationships between variables , use experimental methods. If you want to understand the characteristics of a research subject, use descriptive methods.

Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organizations.

There are various approaches to qualitative data analysis , but they all share five steps in common:

• Prepare and organize your data.
• Review and explore your data.
• Develop a data coding system.
• Assign codes to the data.
• Identify recurring themes.

The specifics of each step depend on the focus of the analysis. Some common approaches include textual analysis , thematic analysis , and discourse analysis .

A research project is an academic, scientific, or professional undertaking to answer a research question . Research projects can take many forms, such as qualitative or quantitative , descriptive , longitudinal , experimental , or correlational . What kind of research approach you choose will depend on your topic.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Streefkerk, R. (2023, June 22). Qualitative vs. Quantitative Research | Differences, Examples & Methods. Scribbr. Retrieved March 29, 2024, from https://www.scribbr.com/methodology/qualitative-quantitative-research/

Is this article helpful?

Raimo Streefkerk

Other students also liked, what is quantitative research | definition, uses & methods, what is qualitative research | methods & examples, mixed methods research | definition, guide & examples, unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

• Register or Log In
• 0) { document.location='/search/'+document.getElementById('quicksearch').value.trim().toLowerCase(); }">

'Research Methods' Practice Quiz Chapter 7

Quiz content, are you sure, select your country.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Neurol Res Pract

How to use and assess qualitative research methods

Loraine busetto.

1 Department of Neurology, Heidelberg University Hospital, Im Neuenheimer Feld 400, 69120 Heidelberg, Germany

Wolfgang Wick

2 Clinical Cooperation Unit Neuro-Oncology, German Cancer Research Center, Heidelberg, Germany

Christoph Gumbinger

Associated data.

Not applicable.

This paper aims to provide an overview of the use and assessment of qualitative research methods in the health sciences. Qualitative research can be defined as the study of the nature of phenomena and is especially appropriate for answering questions of why something is (not) observed, assessing complex multi-component interventions, and focussing on intervention improvement. The most common methods of data collection are document study, (non-) participant observations, semi-structured interviews and focus groups. For data analysis, field-notes and audio-recordings are transcribed into protocols and transcripts, and coded using qualitative data management software. Criteria such as checklists, reflexivity, sampling strategies, piloting, co-coding, member-checking and stakeholder involvement can be used to enhance and assess the quality of the research conducted. Using qualitative in addition to quantitative designs will equip us with better tools to address a greater range of research problems, and to fill in blind spots in current neurological research and practice.

The aim of this paper is to provide an overview of qualitative research methods, including hands-on information on how they can be used, reported and assessed. This article is intended for beginning qualitative researchers in the health sciences as well as experienced quantitative researchers who wish to broaden their understanding of qualitative research.

What is qualitative research?

Qualitative research is defined as “the study of the nature of phenomena”, including “their quality, different manifestations, the context in which they appear or the perspectives from which they can be perceived” , but excluding “their range, frequency and place in an objectively determined chain of cause and effect” [ 1 ]. This formal definition can be complemented with a more pragmatic rule of thumb: qualitative research generally includes data in form of words rather than numbers [ 2 ].

Why conduct qualitative research?

Because some research questions cannot be answered using (only) quantitative methods. For example, one Australian study addressed the issue of why patients from Aboriginal communities often present late or not at all to specialist services offered by tertiary care hospitals. Using qualitative interviews with patients and staff, it found one of the most significant access barriers to be transportation problems, including some towns and communities simply not having a bus service to the hospital [ 3 ]. A quantitative study could have measured the number of patients over time or even looked at possible explanatory factors – but only those previously known or suspected to be of relevance. To discover reasons for observed patterns, especially the invisible or surprising ones, qualitative designs are needed.

While qualitative research is common in other fields, it is still relatively underrepresented in health services research. The latter field is more traditionally rooted in the evidence-based-medicine paradigm, as seen in " research that involves testing the effectiveness of various strategies to achieve changes in clinical practice, preferably applying randomised controlled trial study designs (...) " [ 4 ]. This focus on quantitative research and specifically randomised controlled trials (RCT) is visible in the idea of a hierarchy of research evidence which assumes that some research designs are objectively better than others, and that choosing a "lesser" design is only acceptable when the better ones are not practically or ethically feasible [ 5 , 6 ]. Others, however, argue that an objective hierarchy does not exist, and that, instead, the research design and methods should be chosen to fit the specific research question at hand – "questions before methods" [ 2 , 7 – 9 ]. This means that even when an RCT is possible, some research problems require a different design that is better suited to addressing them. Arguing in JAMA, Berwick uses the example of rapid response teams in hospitals, which he describes as " a complex, multicomponent intervention – essentially a process of social change" susceptible to a range of different context factors including leadership or organisation history. According to him, "[in] such complex terrain, the RCT is an impoverished way to learn. Critics who use it as a truth standard in this context are incorrect" [ 8 ] . Instead of limiting oneself to RCTs, Berwick recommends embracing a wider range of methods , including qualitative ones, which for "these specific applications, (...) are not compromises in learning how to improve; they are superior" [ 8 ].

Research problems that can be approached particularly well using qualitative methods include assessing complex multi-component interventions or systems (of change), addressing questions beyond “what works”, towards “what works for whom when, how and why”, and focussing on intervention improvement rather than accreditation [ 7 , 9 – 12 ]. Using qualitative methods can also help shed light on the “softer” side of medical treatment. For example, while quantitative trials can measure the costs and benefits of neuro-oncological treatment in terms of survival rates or adverse effects, qualitative research can help provide a better understanding of patient or caregiver stress, visibility of illness or out-of-pocket expenses.

How to conduct qualitative research?

Given that qualitative research is characterised by flexibility, openness and responsivity to context, the steps of data collection and analysis are not as separate and consecutive as they tend to be in quantitative research [ 13 , 14 ]. As Fossey puts it : “sampling, data collection, analysis and interpretation are related to each other in a cyclical (iterative) manner, rather than following one after another in a stepwise approach” [ 15 ]. The researcher can make educated decisions with regard to the choice of method, how they are implemented, and to which and how many units they are applied [ 13 ]. As shown in Fig.  1 , this can involve several back-and-forth steps between data collection and analysis where new insights and experiences can lead to adaption and expansion of the original plan. Some insights may also necessitate a revision of the research question and/or the research design as a whole. The process ends when saturation is achieved, i.e. when no relevant new information can be found (see also below: sampling and saturation). For reasons of transparency, it is essential for all decisions as well as the underlying reasoning to be well-documented.

Iterative research process

While it is not always explicitly addressed, qualitative methods reflect a different underlying research paradigm than quantitative research (e.g. constructivism or interpretivism as opposed to positivism). The choice of methods can be based on the respective underlying substantive theory or theoretical framework used by the researcher [ 2 ].

Data collection

The methods of qualitative data collection most commonly used in health research are document study, observations, semi-structured interviews and focus groups [ 1 , 14 , 16 , 17 ].

Document study

Document study (also called document analysis) refers to the review by the researcher of written materials [ 14 ]. These can include personal and non-personal documents such as archives, annual reports, guidelines, policy documents, diaries or letters.

Observations

Observations are particularly useful to gain insights into a certain setting and actual behaviour – as opposed to reported behaviour or opinions [ 13 ]. Qualitative observations can be either participant or non-participant in nature. In participant observations, the observer is part of the observed setting, for example a nurse working in an intensive care unit [ 18 ]. In non-participant observations, the observer is “on the outside looking in”, i.e. present in but not part of the situation, trying not to influence the setting by their presence. Observations can be planned (e.g. for 3 h during the day or night shift) or ad hoc (e.g. as soon as a stroke patient arrives at the emergency room). During the observation, the observer takes notes on everything or certain pre-determined parts of what is happening around them, for example focusing on physician-patient interactions or communication between different professional groups. Written notes can be taken during or after the observations, depending on feasibility (which is usually lower during participant observations) and acceptability (e.g. when the observer is perceived to be judging the observed). Afterwards, these field notes are transcribed into observation protocols. If more than one observer was involved, field notes are taken independently, but notes can be consolidated into one protocol after discussions. Advantages of conducting observations include minimising the distance between the researcher and the researched, the potential discovery of topics that the researcher did not realise were relevant and gaining deeper insights into the real-world dimensions of the research problem at hand [ 18 ].

Semi-structured interviews

Hijmans & Kuyper describe qualitative interviews as “an exchange with an informal character, a conversation with a goal” [ 19 ]. Interviews are used to gain insights into a person’s subjective experiences, opinions and motivations – as opposed to facts or behaviours [ 13 ]. Interviews can be distinguished by the degree to which they are structured (i.e. a questionnaire), open (e.g. free conversation or autobiographical interviews) or semi-structured [ 2 , 13 ]. Semi-structured interviews are characterized by open-ended questions and the use of an interview guide (or topic guide/list) in which the broad areas of interest, sometimes including sub-questions, are defined [ 19 ]. The pre-defined topics in the interview guide can be derived from the literature, previous research or a preliminary method of data collection, e.g. document study or observations. The topic list is usually adapted and improved at the start of the data collection process as the interviewer learns more about the field [ 20 ]. Across interviews the focus on the different (blocks of) questions may differ and some questions may be skipped altogether (e.g. if the interviewee is not able or willing to answer the questions or for concerns about the total length of the interview) [ 20 ]. Qualitative interviews are usually not conducted in written format as it impedes on the interactive component of the method [ 20 ]. In comparison to written surveys, qualitative interviews have the advantage of being interactive and allowing for unexpected topics to emerge and to be taken up by the researcher. This can also help overcome a provider or researcher-centred bias often found in written surveys, which by nature, can only measure what is already known or expected to be of relevance to the researcher. Interviews can be audio- or video-taped; but sometimes it is only feasible or acceptable for the interviewer to take written notes [ 14 , 16 , 20 ].

Focus groups

Focus groups are group interviews to explore participants’ expertise and experiences, including explorations of how and why people behave in certain ways [ 1 ]. Focus groups usually consist of 6–8 people and are led by an experienced moderator following a topic guide or “script” [ 21 ]. They can involve an observer who takes note of the non-verbal aspects of the situation, possibly using an observation guide [ 21 ]. Depending on researchers’ and participants’ preferences, the discussions can be audio- or video-taped and transcribed afterwards [ 21 ]. Focus groups are useful for bringing together homogeneous (to a lesser extent heterogeneous) groups of participants with relevant expertise and experience on a given topic on which they can share detailed information [ 21 ]. Focus groups are a relatively easy, fast and inexpensive method to gain access to information on interactions in a given group, i.e. “the sharing and comparing” among participants [ 21 ]. Disadvantages include less control over the process and a lesser extent to which each individual may participate. Moreover, focus group moderators need experience, as do those tasked with the analysis of the resulting data. Focus groups can be less appropriate for discussing sensitive topics that participants might be reluctant to disclose in a group setting [ 13 ]. Moreover, attention must be paid to the emergence of “groupthink” as well as possible power dynamics within the group, e.g. when patients are awed or intimidated by health professionals.

Choosing the “right” method

As explained above, the school of thought underlying qualitative research assumes no objective hierarchy of evidence and methods. This means that each choice of single or combined methods has to be based on the research question that needs to be answered and a critical assessment with regard to whether or to what extent the chosen method can accomplish this – i.e. the “fit” between question and method [ 14 ]. It is necessary for these decisions to be documented when they are being made, and to be critically discussed when reporting methods and results.

Let us assume that our research aim is to examine the (clinical) processes around acute endovascular treatment (EVT), from the patient’s arrival at the emergency room to recanalization, with the aim to identify possible causes for delay and/or other causes for sub-optimal treatment outcome. As a first step, we could conduct a document study of the relevant standard operating procedures (SOPs) for this phase of care – are they up-to-date and in line with current guidelines? Do they contain any mistakes, irregularities or uncertainties that could cause delays or other problems? Regardless of the answers to these questions, the results have to be interpreted based on what they are: a written outline of what care processes in this hospital should look like. If we want to know what they actually look like in practice, we can conduct observations of the processes described in the SOPs. These results can (and should) be analysed in themselves, but also in comparison to the results of the document analysis, especially as regards relevant discrepancies. Do the SOPs outline specific tests for which no equipment can be observed or tasks to be performed by specialized nurses who are not present during the observation? It might also be possible that the written SOP is outdated, but the actual care provided is in line with current best practice. In order to find out why these discrepancies exist, it can be useful to conduct interviews. Are the physicians simply not aware of the SOPs (because their existence is limited to the hospital’s intranet) or do they actively disagree with them or does the infrastructure make it impossible to provide the care as described? Another rationale for adding interviews is that some situations (or all of their possible variations for different patient groups or the day, night or weekend shift) cannot practically or ethically be observed. In this case, it is possible to ask those involved to report on their actions – being aware that this is not the same as the actual observation. A senior physician’s or hospital manager’s description of certain situations might differ from a nurse’s or junior physician’s one, maybe because they intentionally misrepresent facts or maybe because different aspects of the process are visible or important to them. In some cases, it can also be relevant to consider to whom the interviewee is disclosing this information – someone they trust, someone they are otherwise not connected to, or someone they suspect or are aware of being in a potentially “dangerous” power relationship to them. Lastly, a focus group could be conducted with representatives of the relevant professional groups to explore how and why exactly they provide care around EVT. The discussion might reveal discrepancies (between SOPs and actual care or between different physicians) and motivations to the researchers as well as to the focus group members that they might not have been aware of themselves. For the focus group to deliver relevant information, attention has to be paid to its composition and conduct, for example, to make sure that all participants feel safe to disclose sensitive or potentially problematic information or that the discussion is not dominated by (senior) physicians only. The resulting combination of data collection methods is shown in Fig.  2 .

Possible combination of data collection methods

Attributions for icons: “Book” by Serhii Smirnov, “Interview” by Adrien Coquet, FR, “Magnifying Glass” by anggun, ID, “Business communication” by Vectors Market; all from the Noun Project

The combination of multiple data source as described for this example can be referred to as “triangulation”, in which multiple measurements are carried out from different angles to achieve a more comprehensive understanding of the phenomenon under study [ 22 , 23 ].

Data analysis

To analyse the data collected through observations, interviews and focus groups these need to be transcribed into protocols and transcripts (see Fig.  3 ). Interviews and focus groups can be transcribed verbatim , with or without annotations for behaviour (e.g. laughing, crying, pausing) and with or without phonetic transcription of dialects and filler words, depending on what is expected or known to be relevant for the analysis. In the next step, the protocols and transcripts are coded , that is, marked (or tagged, labelled) with one or more short descriptors of the content of a sentence or paragraph [ 2 , 15 , 23 ]. Jansen describes coding as “connecting the raw data with “theoretical” terms” [ 20 ]. In a more practical sense, coding makes raw data sortable. This makes it possible to extract and examine all segments describing, say, a tele-neurology consultation from multiple data sources (e.g. SOPs, emergency room observations, staff and patient interview). In a process of synthesis and abstraction, the codes are then grouped, summarised and/or categorised [ 15 , 20 ]. The end product of the coding or analysis process is a descriptive theory of the behavioural pattern under investigation [ 20 ]. The coding process is performed using qualitative data management software, the most common ones being InVivo, MaxQDA and Atlas.ti. It should be noted that these are data management tools which support the analysis performed by the researcher(s) [ 14 ].

From data collection to data analysis

Attributions for icons: see Fig. ​ Fig.2, 2 , also “Speech to text” by Trevor Dsouza, “Field Notes” by Mike O’Brien, US, “Voice Record” by ProSymbols, US, “Inspection” by Made, AU, and “Cloud” by Graphic Tigers; all from the Noun Project

How to report qualitative research?

Protocols of qualitative research can be published separately and in advance of the study results. However, the aim is not the same as in RCT protocols, i.e. to pre-define and set in stone the research questions and primary or secondary endpoints. Rather, it is a way to describe the research methods in detail, which might not be possible in the results paper given journals’ word limits. Qualitative research papers are usually longer than their quantitative counterparts to allow for deep understanding and so-called “thick description”. In the methods section, the focus is on transparency of the methods used, including why, how and by whom they were implemented in the specific study setting, so as to enable a discussion of whether and how this may have influenced data collection, analysis and interpretation. The results section usually starts with a paragraph outlining the main findings, followed by more detailed descriptions of, for example, the commonalities, discrepancies or exceptions per category [ 20 ]. Here it is important to support main findings by relevant quotations, which may add information, context, emphasis or real-life examples [ 20 , 23 ]. It is subject to debate in the field whether it is relevant to state the exact number or percentage of respondents supporting a certain statement (e.g. “Five interviewees expressed negative feelings towards XYZ”) [ 21 ].

How to combine qualitative with quantitative research?

Qualitative methods can be combined with other methods in multi- or mixed methods designs, which “[employ] two or more different methods [ …] within the same study or research program rather than confining the research to one single method” [ 24 ]. Reasons for combining methods can be diverse, including triangulation for corroboration of findings, complementarity for illustration and clarification of results, expansion to extend the breadth and range of the study, explanation of (unexpected) results generated with one method with the help of another, or offsetting the weakness of one method with the strength of another [ 1 , 17 , 24 – 26 ]. The resulting designs can be classified according to when, why and how the different quantitative and/or qualitative data strands are combined. The three most common types of mixed method designs are the convergent parallel design , the explanatory sequential design and the exploratory sequential design. The designs with examples are shown in Fig.  4 .

Three common mixed methods designs

In the convergent parallel design, a qualitative study is conducted in parallel to and independently of a quantitative study, and the results of both studies are compared and combined at the stage of interpretation of results. Using the above example of EVT provision, this could entail setting up a quantitative EVT registry to measure process times and patient outcomes in parallel to conducting the qualitative research outlined above, and then comparing results. Amongst other things, this would make it possible to assess whether interview respondents’ subjective impressions of patients receiving good care match modified Rankin Scores at follow-up, or whether observed delays in care provision are exceptions or the rule when compared to door-to-needle times as documented in the registry. In the explanatory sequential design, a quantitative study is carried out first, followed by a qualitative study to help explain the results from the quantitative study. This would be an appropriate design if the registry alone had revealed relevant delays in door-to-needle times and the qualitative study would be used to understand where and why these occurred, and how they could be improved. In the exploratory design, the qualitative study is carried out first and its results help informing and building the quantitative study in the next step [ 26 ]. If the qualitative study around EVT provision had shown a high level of dissatisfaction among the staff members involved, a quantitative questionnaire investigating staff satisfaction could be set up in the next step, informed by the qualitative study on which topics dissatisfaction had been expressed. Amongst other things, the questionnaire design would make it possible to widen the reach of the research to more respondents from different (types of) hospitals, regions, countries or settings, and to conduct sub-group analyses for different professional groups.

How to assess qualitative research?

A variety of assessment criteria and lists have been developed for qualitative research, ranging in their focus and comprehensiveness [ 14 , 17 , 27 ]. However, none of these has been elevated to the “gold standard” in the field. In the following, we therefore focus on a set of commonly used assessment criteria that, from a practical standpoint, a researcher can look for when assessing a qualitative research report or paper.

Assessors should check the authors’ use of and adherence to the relevant reporting checklists (e.g. Standards for Reporting Qualitative Research (SRQR)) to make sure all items that are relevant for this type of research are addressed [ 23 , 28 ]. Discussions of quantitative measures in addition to or instead of these qualitative measures can be a sign of lower quality of the research (paper). Providing and adhering to a checklist for qualitative research contributes to an important quality criterion for qualitative research, namely transparency [ 15 , 17 , 23 ].

Reflexivity

While methodological transparency and complete reporting is relevant for all types of research, some additional criteria must be taken into account for qualitative research. This includes what is called reflexivity, i.e. sensitivity to the relationship between the researcher and the researched, including how contact was established and maintained, or the background and experience of the researcher(s) involved in data collection and analysis. Depending on the research question and population to be researched this can be limited to professional experience, but it may also include gender, age or ethnicity [ 17 , 27 ]. These details are relevant because in qualitative research, as opposed to quantitative research, the researcher as a person cannot be isolated from the research process [ 23 ]. It may influence the conversation when an interviewed patient speaks to an interviewer who is a physician, or when an interviewee is asked to discuss a gynaecological procedure with a male interviewer, and therefore the reader must be made aware of these details [ 19 ].

Sampling and saturation

The aim of qualitative sampling is for all variants of the objects of observation that are deemed relevant for the study to be present in the sample “ to see the issue and its meanings from as many angles as possible” [ 1 , 16 , 19 , 20 , 27 ] , and to ensure “information-richness [ 15 ]. An iterative sampling approach is advised, in which data collection (e.g. five interviews) is followed by data analysis, followed by more data collection to find variants that are lacking in the current sample. This process continues until no new (relevant) information can be found and further sampling becomes redundant – which is called saturation [ 1 , 15 ] . In other words: qualitative data collection finds its end point not a priori , but when the research team determines that saturation has been reached [ 29 , 30 ].

This is also the reason why most qualitative studies use deliberate instead of random sampling strategies. This is generally referred to as “ purposive sampling” , in which researchers pre-define which types of participants or cases they need to include so as to cover all variations that are expected to be of relevance, based on the literature, previous experience or theory (i.e. theoretical sampling) [ 14 , 20 ]. Other types of purposive sampling include (but are not limited to) maximum variation sampling, critical case sampling or extreme or deviant case sampling [ 2 ]. In the above EVT example, a purposive sample could include all relevant professional groups and/or all relevant stakeholders (patients, relatives) and/or all relevant times of observation (day, night and weekend shift).

Assessors of qualitative research should check whether the considerations underlying the sampling strategy were sound and whether or how researchers tried to adapt and improve their strategies in stepwise or cyclical approaches between data collection and analysis to achieve saturation [ 14 ].

Good qualitative research is iterative in nature, i.e. it goes back and forth between data collection and analysis, revising and improving the approach where necessary. One example of this are pilot interviews, where different aspects of the interview (especially the interview guide, but also, for example, the site of the interview or whether the interview can be audio-recorded) are tested with a small number of respondents, evaluated and revised [ 19 ]. In doing so, the interviewer learns which wording or types of questions work best, or which is the best length of an interview with patients who have trouble concentrating for an extended time. Of course, the same reasoning applies to observations or focus groups which can also be piloted.

Ideally, coding should be performed by at least two researchers, especially at the beginning of the coding process when a common approach must be defined, including the establishment of a useful coding list (or tree), and when a common meaning of individual codes must be established [ 23 ]. An initial sub-set or all transcripts can be coded independently by the coders and then compared and consolidated after regular discussions in the research team. This is to make sure that codes are applied consistently to the research data.

Member checking

Member checking, also called respondent validation , refers to the practice of checking back with study respondents to see if the research is in line with their views [ 14 , 27 ]. This can happen after data collection or analysis or when first results are available [ 23 ]. For example, interviewees can be provided with (summaries of) their transcripts and asked whether they believe this to be a complete representation of their views or whether they would like to clarify or elaborate on their responses [ 17 ]. Respondents’ feedback on these issues then becomes part of the data collection and analysis [ 27 ].

Stakeholder involvement

In those niches where qualitative approaches have been able to evolve and grow, a new trend has seen the inclusion of patients and their representatives not only as study participants (i.e. “members”, see above) but as consultants to and active participants in the broader research process [ 31 – 33 ]. The underlying assumption is that patients and other stakeholders hold unique perspectives and experiences that add value beyond their own single story, making the research more relevant and beneficial to researchers, study participants and (future) patients alike [ 34 , 35 ]. Using the example of patients on or nearing dialysis, a recent scoping review found that 80% of clinical research did not address the top 10 research priorities identified by patients and caregivers [ 32 , 36 ]. In this sense, the involvement of the relevant stakeholders, especially patients and relatives, is increasingly being seen as a quality indicator in and of itself.

How not to assess qualitative research

The above overview does not include certain items that are routine in assessments of quantitative research. What follows is a non-exhaustive, non-representative, experience-based list of the quantitative criteria often applied to the assessment of qualitative research, as well as an explanation of the limited usefulness of these endeavours.

Protocol adherence

Given the openness and flexibility of qualitative research, it should not be assessed by how well it adheres to pre-determined and fixed strategies – in other words: its rigidity. Instead, the assessor should look for signs of adaptation and refinement based on lessons learned from earlier steps in the research process.

Sample size

For the reasons explained above, qualitative research does not require specific sample sizes, nor does it require that the sample size be determined a priori [ 1 , 14 , 27 , 37 – 39 ]. Sample size can only be a useful quality indicator when related to the research purpose, the chosen methodology and the composition of the sample, i.e. who was included and why.

Randomisation

While some authors argue that randomisation can be used in qualitative research, this is not commonly the case, as neither its feasibility nor its necessity or usefulness has been convincingly established for qualitative research [ 13 , 27 ]. Relevant disadvantages include the negative impact of a too large sample size as well as the possibility (or probability) of selecting “ quiet, uncooperative or inarticulate individuals ” [ 17 ]. Qualitative studies do not use control groups, either.

Interrater reliability, variability and other “objectivity checks”

The concept of “interrater reliability” is sometimes used in qualitative research to assess to which extent the coding approach overlaps between the two co-coders. However, it is not clear what this measure tells us about the quality of the analysis [ 23 ]. This means that these scores can be included in qualitative research reports, preferably with some additional information on what the score means for the analysis, but it is not a requirement. Relatedly, it is not relevant for the quality or “objectivity” of qualitative research to separate those who recruited the study participants and collected and analysed the data. Experiences even show that it might be better to have the same person or team perform all of these tasks [ 20 ]. First, when researchers introduce themselves during recruitment this can enhance trust when the interview takes place days or weeks later with the same researcher. Second, when the audio-recording is transcribed for analysis, the researcher conducting the interviews will usually remember the interviewee and the specific interview situation during data analysis. This might be helpful in providing additional context information for interpretation of data, e.g. on whether something might have been meant as a joke [ 18 ].

Not being quantitative research

Being qualitative research instead of quantitative research should not be used as an assessment criterion if it is used irrespectively of the research problem at hand. Similarly, qualitative research should not be required to be combined with quantitative research per se – unless mixed methods research is judged as inherently better than single-method research. In this case, the same criterion should be applied for quantitative studies without a qualitative component.

The main take-away points of this paper are summarised in Table ​ Table1. 1 . We aimed to show that, if conducted well, qualitative research can answer specific research questions that cannot to be adequately answered using (only) quantitative designs. Seeing qualitative and quantitative methods as equal will help us become more aware and critical of the “fit” between the research problem and our chosen methods: I can conduct an RCT to determine the reasons for transportation delays of acute stroke patients – but should I? It also provides us with a greater range of tools to tackle a greater range of research problems more appropriately and successfully, filling in the blind spots on one half of the methodological spectrum to better address the whole complexity of neurological research and practice.

Take-away-points

Acknowledgements

Abbreviations, authors’ contributions.

LB drafted the manuscript; WW and CG revised the manuscript; all authors approved the final versions.

no external funding.

Availability of data and materials

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Have an account?

Suggestions for you See more

Scientific Method

9th -  12th  , observations and inferences, 6th -  8th  , 11.6k plays.

Qualitative Research

80 questions

Introducing new   Paper mode

No student devices needed.   Know more

Triangulation is a validation approach based on the search for the __________ of results obtained by multiple sources and perspectives.

convergence

A researcher who utilizes methods triangulation __________.

mixes research methods with non overlapping strengths and weaknesses

examines multiple theories in order to explain study results

uses multiple methods of data collection to capture a single phenomenon

corroborates the observations of multiple observers

Researchers conducting a qualitative study focus on exploring the world as it occurs __________ in order to capture participants’ perspectives.

artificially

Data collected in qualitative research are analyzed and interpreted and then written using ________.

statistical

passive-voice

third-person

Qualitative research uses the _________ or _________ part of an inclusive scientific method.

Qualitative research focuses on studying particular people, groups, schools, and places.

Qualitative research is just the opinion of the researcher and is based only on idiosyncratic and anecdotal information.

Qualitative research uses an inductive mode of science and often focuses on exploration and discovery.

_______ research usually focuses on the relationships among variables, while ______ attempts to understand concrete human reality in its local context, with all of its complexity.

Answer the first blank only

Quantitative

Qualitative

Which of the following is an example or characteristic of using a “holistic” approach in qualitative research?

Transcribing a distinctive communication pattern of a community member

focusing only on the data available on a single day within a study

Documenting the ethnic background of a study participant

Describing the established cultural norms of a given population

The __________________ of qualitative research contend that knowledge is socially constructed and relativistic.

Epistemology

View of Thought & Behavior

Research Focus

Perspectives on ______________ in qualitative studies are fluid and highly contextualized in qualitative research, providing researchers with many situational perspectives.

The ____________ of qualitative students involves a “wide-angle” lens to capture the entirety, of a phenomenon.

In qualitative research __________ emphasizes individuals’ personal realities and truths rather than objective or universal ones.

Another term for criterion-based selection is __________.

response rate sampling

purposive sampling

typical-case sampling

random sampling

A type of sampling in which the researcher specifies the characteristics of the population of interest and locates individuals with those characteristics is called?

quota sampling

snowball sampling

convenience sampling

__________is a small group of research participants discussing issues in one setting at the same time.

an interview

a focus group

an observation

a questionnaire

In qualitative research, “designs” refers ______?

research methods

research objectives

research studies

Phenomenology-

Describing individuals’ experiences of a particular event, such as the death of a loved one

Addressing research questions through in-depth analysis of a single (small group of people) instance(s) or individuals

Inductively generating a theory explain a process

Describing lives/ stories to add to understanding

Ethnography-

Inductively generating a theory explaining a process

Describing cultural characteristics of a group of people

Narrative Inquiry-

Describing individuals’ experiences of an event, such as the death of a loved one

Describing individuals’ experiences of a event, such as the death of a loved one

Grounded Theory-

A study that intends to describe the shared beliefs and practices of individuals living in a community in the foothills of the Appalachian Mountains would be an example of which type of qualitative research?

phenomenology

ethnography

narrative inquiry

grounded theory

A researcher interviews 20–30 women who are in abusive relationships and from this research she develops a theory to explain the dynamics of abusive relationships. This would likely be an example of which type of qualitative research?

A researcher interviews college students to learn more about their experiences of learning from bad instructors. This would be an example of which type of qualitative research?

absence divergence

The researcher assumes that each case is special and unique. The first level of analysis is being true to, respecting, and capturing the details of the individual cases being studied; cross-case analysis follows from—and depends on—the quality of individual case studies.

unique case orientation

inductive analysis and creative synthesis

holistic perspective

context sensitivity

voice, perspective, and reflexivity

The researcher seeks immersion in the details and specifics of the data to discover important patterns, themes, and interrelationships. Begins by exploring, then confirming; is guided by analytical principles rather than rules. Study ends with a creative synthesis.

The whole phenomenon under study is understood as a complex system that is more than the sum of its parts. The focus is on complex interdependencies and system dynamics that cannot meaningfully be reduced to a few discrete variables and linear, cause-effect relationships.

The researcher places findings in a social, historical, and temporal context and is careful about, even dubious of, the possibility or meaningfulness of generalizations across time and space. Emphasizes instead careful comparative case analyses and extrapolating patterns for possible transferability to and adaptation in new settings.

The qualitative analyst owns and is reflective about her or his own voice and perspective; a credible voice conveys authenticity and trustworthiness. Complete objectivity being impossible and pure subjectivity undermining credibility, the researcher’s focus is on balance—understanding and depicting the world authentically in all its complexity while being self-analytical, politically aware, and reflexive in consciousness.

Qualitative researchers often conduct data collection and analysis ___________ during the study process.

in repeated cycles over time

at a single point in time

Data collection and analysis in qualitative studies are often _________.

triangulated and thematic

statistical and time-limited

Refers to the use of multiple forms of data to capture a single phenomenon. These data provide the opportunity for cross-checking or researcher interpretations.

data triangulation

investigator triangulation

methods triangulation

theory triangulation

Involves the use of multiple observers to record and describe the research participants’ behavior and context. These observers assess the degree to which their observations are corroborated in order to enhance the credibility and defensibility of the results.

investigation triangulation

Refers to the use of a mix of methods for organizing the study and collecting data with an objective of combining different methods that have non overlapping weaknesses and strengths.

Involves the use of a variety of theories and perspectives to explain the phenomena under study (including prior theories and/or other researchers). These contrasting perspectives can provide researchers with insights and help in the development of a more cogent explanation that fits their data.

Use a wide-angle and “deep-angle” lens, examining the breadth and depth of phenomena to learn anything and everything that might be important.

quantitative research

qualitative research

mixed methods research

action research

Which of the following is not a common objective of qualitative research?

Laboratory experimentation

Naturalistic observation

Unobtrusive observation

Holistic descriptions

Which method of research allows qualitative studies' explanations to develop throughout the process?

A specific culture’s language differentiates one concept (e.g., snow) into many varied types not recognized in outside populations. This is known as ______________.

demographic area

mistranslation

epidemiology research

linguistic relativity

Ontology and epistemology are examples of ___________ within the context of research.

philosophical assumptions

theoretical frameworks

analysis techniques

design strategies

In qualitative data collection, which term refers to participants self-reporting instruments?

questionnaires

observations

secondary data

focus groups

After selecting a research topic, what is the next major step in the process of qualitative study?

design the study

collect data

determine research questions

generate findings

Positivism is a belief that all true knowledge must be based on ____.

observation

Postmodernism emphasizes the primacy of individuality, fragmentation, constant change, and _________.

Case study narrative reports include a _______.

discussion of themes issues, and implications

description of invariant structures

discussion of evolving stories and relationships

description of cultural theme contexts

Which theoretical framework utilizes coding to build explanatory models during the data analysis phase?

narrative Inquiry

An established “strength” of qualitative research occurs when data is collected in which kind of setting?

small population pool

naturalistic

unrepeatable

researcher-biased

Qualitative researchers try to understand multiple layers of reality in research settings. Which of the following is NOT an example of one of these layers?

relationships between participants

types of people in a group

results from outside studies

established cultural norms

In qualitative research, the researchers can also be known as the _________ of data collection.

manipulator

The qualitative analysis strategy focusing on complex interdependencies and system dynamics is __________.

inductive analysis

reflexivity

Qualitative studies can often suffer from which limitation?

insight into participants construct interpretation

access to local contexts

cross-case comparisons

lower credibility with some authorities

According to the module, ________ is not considered a key component of qualitative research.

lab results are

special jargons are

quality criteria are

The theoretical framework of “Grounded Theory” originates from....

Which arrow represents the inductive method of qualitative research?

The exploratory arrow

The confirmatory arrow

The arrow from Theory to Hypotheses, Predictions

The arrow from Hypotheses, Predictions to Observations, Data

Qualitative researchers use a ________ approach?

Which major research approach seeks to understand people’s subjective and shared experiences through systematic observation?

Mixed Methods

Describe whether each of the following apply to the ontology, epistemology, view of thought and behavior, or research focus of qualitative research.

subjective, mental, personal, constructed

epistemology

view of though and behavior

research focus

relativism, individual/group justification, varying standards, social construction

view of thought and behavior

personal, social, contextual/situational, fluid, unpredictable

wide-angle and “deep-angle” lens, examining breadth & depth of phenomena, Not interested in generalizing beyond individuals/groups studied

Which three views of human thought and behavior are characteristic of qualitative research?

Choose three answers.

It is fluid

it is predictable

it is personal

it occurs in regular patterns

it is highly situational

Which two characteristics of data collection and analysis are common to the theoretical frameworks of qualitative research?

Choose two answers.

context sensitive

triangulated

time-limited

Which of the following are theoretical frameworks of qualitative research?

experimental

nonexperimental

Which type of qualitative data analysis places a high priority on being true to, respecting, and capturing the details of individual situations?

What is a main strength of qualitative research?

generates results that are relatively independent of the research

leads to a complex understanding about events or phenomenon

produces integrated knowledge that informs theory and practice

empowers practitioners to contribute to knowledge base

Classify each research method as belonging to either the quantitative or qualitative research.

Experimental

qualitative

quantitative

Nonexperimental

Ethnography

Match each type of research report with the corresponding type of qualitative research. Answer options may be used more than once or not at all.

Extensive description of physical and social settings aimed at holistic understanding.

Phenomenology

Narrative Inquiry

Grounded Theory

Account that includes patterns, connections, and insights uncovered.

Rich narrative that lets readers experience phenomenon through eyes of participants.

Holistic narrative that triangulates data and places study into meaningful context.

Methodological descriptions followed by proposed idea built during the study.

Answer the second blank only

Explore all questions with a free account

Continue with email

Continue with phone

• Create A Quiz
• Relationship
• Personality
• Harry Potter
• Online Exam
• Entertainment
• Training Maker
• Survey Maker
• Brain Games
• ProProfs.com

Research Method Quizzes, Questions & Answers

Top trending quizzes.

Popular Topics

Recent quizzes.

83 Qualitative Research Questions & Examples

Qualitative research questions help you understand consumer sentiment. They’re strategically designed to show organizations how and why people feel the way they do about a brand, product, or service. It looks beyond the numbers and is one of the most telling types of market research a company can do.

The UK Data Service describes this perfectly, saying, “The value of qualitative research is that it gives a voice to the lived experience .”

Read on to see seven use cases and 83 qualitative research questions, with the added bonus of examples that show how to get similar insights faster with Similarweb Digital Research Intelligence.

What is a qualitative research question?

A qualitative research question explores a topic in-depth, aiming to better understand the subject through interviews, observations, and other non-numerical data. Qualitative research questions are open-ended, helping to uncover a target audience’s opinions, beliefs, and motivations.

How to choose qualitative research questions?

Choosing the right qualitative research questions can be incremental to the success of your research and the findings you uncover. Here’s my six-step process for choosing the best qualitative research questions.

• Start by understanding the purpose of your research. What do you want to learn? What outcome are you hoping to achieve?
• Consider who you are researching. What are their experiences, attitudes, and beliefs? How can you best capture these in your research questions ?
• Keep your questions open-ended . Qualitative research questions should not be too narrow or too broad. Aim to ask specific questions to provide meaningful answers but broad enough to allow for exploration.
• Balance your research questions. You don’t want all of your questions to be the same type. Aim to mix up your questions to get a variety of answers.
• Ensure your research questions are ethical and free from bias. Always have a second (and third) person check for unconscious bias.
• Consider the language you use. Your questions should be written in a way that is clear and easy to understand. Avoid using jargon , acronyms, or overly technical language.

Types of qualitative research questions

For a question to be considered qualitative, it usually needs to be open-ended. However, as I’ll explain, there can sometimes be a slight cross-over between quantitative and qualitative research questions.

Open-ended questions

These allow for a wide range of responses and can be formatted with multiple-choice answers or a free-text box to collect additional details. The next two types of qualitative questions are considered open questions, but each has its own style and purpose.

• Probing questions are used to delve deeper into a respondent’s thoughts, such as “Can you tell me more about why you feel that way?”
• Comparative questions ask people to compare two or more items, such as “Which product do you prefer and why?” These qualitative questions are highly useful for understanding brand awareness , competitive analysis , and more.

Closed-ended questions

These ask respondents to choose from a predetermined set of responses, such as “On a scale of 1-5, how satisfied are you with the new product?” While they’re traditionally quantitative, adding a free text box that asks for extra comments into why a specific rating was chosen will provide qualitative insights alongside their respective quantitative research question responses.

• Ranking questions get people to rank items in order of preference, such as “Please rank these products in terms of quality.” They’re advantageous in many scenarios, like product development, competitive analysis, and brand awareness.
• Likert scale questions ask people to rate items on a scale, such as “On a scale of 1-5, how satisfied are you with the new product?” Ideal for placement on websites and emails to gather quick, snappy feedback.

Qualitative research question examples

There are many applications of qualitative research and lots of ways you can put your findings to work for the success of your business. Here’s a summary of the most common use cases for qualitative questions and examples to ask.

Qualitative questions for identifying customer needs and motivations

These types of questions help you find out why customers choose products or services and what they are looking for when making a purchase.

• What factors do you consider when deciding to buy a product?
• What would make you choose one product or service over another?
• What are the most important elements of a product that you would buy?
• What features do you look for when purchasing a product?
• What qualities do you look for in a company’s products?
• Do you prefer localized or global brands when making a purchase?
• How do you determine the value of a product?
• What do you think is the most important factor when choosing a product?
• How do you decide if a product or service is worth the money?
• Do you have any specific expectations when purchasing a product?
• Do you prefer to purchase products or services online or in person?
• What kind of customer service do you expect when buying a product?
• How do you decide when it is time to switch to a different product?
• Where do you research products before you decide to buy?
• What do you think is the most important customer value when making a purchase?

Qualitative research questions to enhance customer experience

Use these questions to reveal insights into how customers interact with a company’s products or services and how those experiences can be improved.

• What aspects of our product or service do customers find most valuable?
• How do customers perceive our customer service?
• What factors are most important to customers when purchasing?
• What do customers think of our brand?
• What do customers think of our current marketing efforts?
• How do customers feel about the features and benefits of our product?
• How do customers feel about the price of our product or service?
• How could we improve the customer experience?
• What do customers think of our website or app?
• What do customers think of our customer support?
• What could we do to make our product or service easier to use?
• What do customers think of our competitors?
• What is your preferred way to access our site?
• How do customers feel about our delivery/shipping times?
• What do customers think of our loyalty programs?

Qualitative research question example for customer experience

• 🙋‍♀️ Question: What is your preferred way to access our site?
• 🤓 Insight sought: How mobile-dominant are consumers? Should you invest more in mobile optimization or mobile marketing?
• 🤯 Challenges with traditional qualitative research methods: While using this type of question is ideal if you have a large database to survey when placed on a site or sent to a limited customer list, it only gives you a point-in-time perspective from a limited group of people.
• 💡 A new approach: You can get better, broader insights quicker with Similarweb Digital Research Intelligence. To fully inform your research, you need to know preferences at the industry or market level.
• ⏰ Time to insight: 30 seconds
• ✅ How it’s done: Similarweb offers multiple ways to answer this question without going through a lengthy qualitative research process. 

First, I’m going to do a website market analysis of the banking credit and lending market in the finance sector to get a clearer picture of industry benchmarks.

Here, I can view device preferences across any industry or market instantly. It shows me the device distribution for any country across any period. This clearly answers the question of how mobile dominate my target audience is , with 59.79% opting to access site via a desktop vs. 40.21% via mobile

I then use the trends section to show me the exact split between mobile and web traffic for each key player in my space. Let’s say I’m about to embark on a competitive campaign that targets customers of Chase and Bank of America ; I can see both their audiences are highly desktop dominant compared with others in their space .

Qualitative question examples for developing new products or services

Research questions like this can help you understand customer pain points and give you insights to develop products that meet those needs.

• What is the primary reason you would choose to purchase a product from our company?
• How do you currently use products or services that are similar to ours?
• Is there anything that could be improved with products currently on the market?
• What features would you like to see added to our products?
• How do you prefer to contact a customer service team?
• What do you think sets our company apart from our competitors?
• What other product or service offerings would like to see us offer?
• What type of information would help you make decisions about buying a product?
• What type of advertising methods are most effective in getting your attention?
• What is the biggest deterrent to purchasing products from us?

Qualitative research question example for service development

• 🙋‍♀️ Question: What type of advertising methods are most effective in getting your attention?
• 🤓 Insight sought: The marketing channels and/or content that performs best with a target audience .
• 🤯 Challenges with traditional qualitative research methods: When using qualitative research surveys to answer questions like this, the sample size is limited, and bias could be at play.
• 💡 A better approach: The most authentic insights come from viewing real actions and results that take place in the digital world. No questions or answers are needed to uncover this intel, and the information you seek is readily available in less than a minute.
• ⏰ Time to insight: 5 minutes
• ✅ How it’s done: There are a few ways to approach this. You can either take an industry-wide perspective or hone in on specific competitors to unpack their individual successes. Here, I’ll quickly show a snapshot with a whole market perspective.

Using the market analysis element of Similarweb Digital Research Intelligence, I select my industry or market, which I’ve kept as banking and credit. A quick click into marketing channels shows me which channels drive the highest traffic in my market. Taking direct traffic out of the equation, for now, I can see that referrals and organic traffic are the two highest-performing channels in this market.

Similarweb allows me to view the specific referral partners and pages across these channels. 

Looking closely at referrals in this market, I’ve chosen chase.com and its five closest rivals . I select referrals in the channel traffic element of marketing channels. I see that Capital One is a clear winner, gaining almost 25 million visits due to referral partnerships.

Next, I get to see exactly who is referring traffic to Capital One and the total traffic share for each referrer. I can see the growth as a percentage and how that has changed, along with an engagement score that rates the average engagement level of that audience segment. This is particularly useful when deciding on which new referral partnerships to pursue.  

Once I’ve identified the channels and campaigns that yield the best results, I can then use Similarweb to dive into the various ad creatives and content that have the greatest impact.

These ads are just a few of those listed in the creatives section from my competitive website analysis of Capital One. You can filter this list by the specific campaign, publishers, and ad networks to view those that matter to you most. You can also discover video ad creatives in the same place too.

In just five minutes ⏰ 

• I’ve captured audience loyalty statistics across my market
• Spotted the most competitive players
• Identified the marketing channels my audience is most responsive to
• I know which content and campaigns are driving the highest traffic volume
• I’ve created a target list for new referral partners and have been able to prioritize this based on results and engagement figures from my rivals
• I can see the types of creatives that my target audience is responding to, giving me ideas for ways to generate effective copy for future campaigns

Qualitative questions to determine pricing strategies

Companies need to make sure pricing stays relevant and competitive. Use these questions to determine customer perceptions on pricing and develop pricing strategies to maximize profits and reduce churn.

• How do you feel about our pricing structure?
• How does our pricing compare to other similar products?
• What value do you feel you get from our pricing?
• How could we make our pricing more attractive?
• What would be an ideal price for our product?
• Which features of our product that you would like to see priced differently?
• What discounts or deals would you like to see us offer?
• How do you feel about the amount you have to pay for our product?

Get Faster Answers to Qualitative Research Questions with Similarweb Today

Qualitative research question example for determining pricing strategies.

• 🙋‍♀️ Question: What discounts or deals would you like to see us offer?
• 🤓 Insight sought: The promotions or campaigns that resonate with your target audience.
• 🤯 Challenges with traditional qualitative research methods: Consumers don’t always recall the types of ads or campaigns they respond to. Over time, their needs and habits change. Your sample size is limited to those you ask, leaving a huge pool of unknowns at play.
• 💡 A better approach: While qualitative insights are good to know, you get the most accurate picture of the highest-performing promotion and campaigns by looking at data collected directly from the web. These analytics are real-world, real-time, and based on the collective actions of many, instead of the limited survey group you approach. By getting a complete picture across an entire market, your decisions are better informed and more aligned with current market trends and behaviors.
• ✅ How it’s done: Similarweb’s Popular Pages feature shows the content, products, campaigns, and pages with the highest growth for any website. So, if you’re trying to unpack the successes of others in your space and find out what content resonates with a target audience, there’s a far quicker way to get answers to these questions with Similarweb.

Here, I’m using Capital One as an example site. I can see trending pages on their site showing the largest increase in page views. Other filters include campaign, best-performing, and new–each of which shows you page URLs, share of traffic , and growth as a percentage. This page is particularly useful for staying on top of trending topics , campaigns, and new content being pushed out in a market by key competitors.

Qualitative research questions for product development teams

It’s vital to stay in touch with changing consumer needs. These questions can also be used for new product or service development, but this time, it’s from the perspective of a product manager or development team. 

• What are customers’ primary needs and wants for this product?
• What do customers think of our current product offerings?
• What is the most important feature or benefit of our product?
• How can we improve our product to meet customers’ needs better?
• What do customers like or dislike about our competitors’ products?
• What do customers look for when deciding between our product and a competitor’s?
• How have customer needs and wants for this product changed over time?
• What motivates customers to purchase this product?
• What is the most important thing customers want from this product?
• What features or benefits are most important when selecting a product?
• What do customers perceive to be our product’s pros and cons?
• What would make customers switch from a competitor’s product to ours?
• How do customers perceive our product in comparison to similar products?
• What do customers think of our pricing and value proposition?
• What do customers think of our product’s design, usability, and aesthetics?

Qualitative questions examples to understand customer segments

Market segmentation seeks to create groups of consumers with shared characteristics. Use these questions to learn more about different customer segments and how to target them with tailored messaging.

• What motivates customers to make a purchase?
• How do customers perceive our brand in comparison to our competitors?
• How do customers feel about our product quality?
• How do customers define quality in our products?
• What factors influence customers’ purchasing decisions ?
• What are the most important aspects of customer service?
• What do customers think of our customer service?
• What do customers think of our pricing?
• How do customers rate our product offerings?
• How do customers prefer to make purchases (online, in-store, etc.)?

Qualitative research question example for understanding customer segments

• 🙋‍♀️ Question: Which social media channels are you most active on?
• 🤓 Insight sought: Formulate a social media strategy . Specifically, the social media channels most likely to succeed with a target audience.
• 🤯 Challenges with traditional qualitative research methods: Qualitative research question responses are limited to those you ask, giving you a limited sample size. Questions like this are usually at risk of some bias, and this may not be reflective of real-world actions.
• 💡 A better approach: Get a complete picture of social media preferences for an entire market or specific audience belonging to rival firms. Insights are available in real-time, and are based on the actions of many, not a select group of participants. Data is readily available, easy to understand, and expandable at a moment’s notice.
• ✅ How it’s done: Using Similarweb’s website analysis feature, you can get a clear breakdown of social media stats for your audience using the marketing channels element. It shows the percentage of visits from each channel to your site, respective growth, and specific referral pages by each platform. All data is expandable, meaning you can select any platform, period, and region to drill down and get more accurate intel, instantly.

This example shows me Bank of America’s social media distribution, with YouTube , Linkedin , and Facebook taking the top three spots, and accounting for almost 80% of traffic being driven from social media.

When doing any type of market research, it’s important to benchmark performance against industry averages and perform a social media competitive analysis to verify rival performance across the same channels.

Qualitative questions to inform competitive analysis

Organizations must assess market sentiment toward other players to compete and beat rival firms. Whether you want to increase market share , challenge industry leaders , or reduce churn, understanding how people view you vs. the competition is key.

• What is the overall perception of our competitors’ product offerings in the market?
• What attributes do our competitors prioritize in their customer experience?
• What strategies do our competitors use to differentiate their products from ours?
• How do our competitors position their products in relation to ours?
• How do our competitors’ pricing models compare to ours?
• What do consumers think of our competitors’ product quality?
• What do consumers think of our competitors’ customer service?
• What are the key drivers of purchase decisions in our market?
• What is the impact of our competitors’ marketing campaigns on our market share ? 10. How do our competitors leverage social media to promote their products?

Qualitative research question example for competitive analysis

• 🙋‍♀️ Question: What other companies do you shop with for x?
• 🤓 Insight sought: W ho are your competitors? Which of your rival’s sites do your customers visit? How loyal are consumers in your market?
• 🤯 Challenges with traditional qualitative research methods:  Sample size is limited, and customers could be unwilling to reveal which competitors they shop with, or how often they around. Where finances are involved, people can act with reluctance or bias, and be unwilling to reveal other suppliers they do business with.
• 💡 A better approach: Get a complete picture of your audience’s loyalty, see who else they shop with, and how many other sites they visit in your competitive group. Find out the size of the untapped opportunity and which players are doing a better job at attracting unique visitors – without having to ask people to reveal their preferences.
• ✅ How it’s done: Similarweb website analysis shows you the competitive sites your audience visits, giving you access to data that shows cross-visitation habits, audience loyalty, and untapped potential in a matter of minutes.

Using the audience interests element of Similarweb website analysis, you can view the cross-browsing behaviors of a website’s audience instantly. You can see a matrix that shows the percentage of visitors on a target site and any rival site they may have visited.

With the Similarweb audience overlap feature, view the cross-visitation habits of an audience across specific websites. In this example, I chose chase.com and its four closest competitors to review. For each intersection, you see the number of unique visitors and the overall proportion of each site’s audience it represents. It also shows the volume of unreached potential visitors.

Here, you can see a direct comparison of the audience loyalty represented in a bar graph. It shows a breakdown of each site’s audience based on how many other sites they have visited. Those sites with the highest loyalty show fewer additional sites visited.

From the perspective of chase.com, I can see 47% of their visitors do not visit rival sites. 33% of their audience visited 1 or more sites in this group, 14% visited 2 or more sites, 4% visited 3 or more sites, and just 0.8% viewed all sites in this comparison. 

How to answer qualitative research questions with Similarweb

Similarweb Digital Research Intelligence drastically improves market research efficiency and time to insight. Both of these can impact the bottom line and the pace at which organizations can adapt and flex when markets shift, and rivals change tactics.

Outdated practices, while still useful, take time . And with a quicker, more efficient way to garner similar insights, opting for the fast lane puts you at a competitive advantage.

With a birds-eye view of the actions and behaviors of companies and consumers across a market , you can answer certain research questions without the need to plan, do, and review extensive qualitative market research .

Wrapping up

Qualitative research methods have been around for centuries. From designing the questions to finding the best distribution channels, collecting and analyzing findings takes time to get the insights you need. Similarweb Digital Research Intelligence drastically improves efficiency and time to insight. Both of which impact the bottom line and the pace at which organizations can adapt and flex when markets shift.

Similarweb’s suite of digital intelligence solutions offers unbiased, accurate, honest insights you can trust for analyzing any industry, market, or audience.

• Methodologies used for data collection are robust, transparent, and trustworthy.
• Clear presentation of data via an easy-to-use, intuitive platform.
• It updates dynamically–giving you the freshest data about an industry or market.
• Data is available via an API – so you can plug into platforms like Tableau or PowerBI to streamline your analyses.
• Filter and refine results according to your needs.

Are quantitative or qualitative research questions best?

Both have their place and purpose in market research. Qualitative research questions seek to provide details, whereas quantitative market research gives you numerical statistics that are easier and quicker to analyze. You get more flexibility with qualitative questions, and they’re non-directional.

What are the advantages of qualitative research?

Qualitative research is advantageous because it allows researchers to better understand their subject matter by exploring people’s attitudes, behaviors, and motivations in a particular context. It also allows researchers to uncover new insights that may not have been discovered with quantitative research methods.

What are some of the challenges of qualitative research?

Qualitative research can be time-consuming and costly, typically involving in-depth interviews and focus groups. Additionally, there are challenges associated with the reliability and validity of the collected data, as there is no universal standard for interpreting the results.

Related Posts

From AI to Buy: The Role of Artificial Intelligence in Retail

How to Conduct a Social Media Competitor Analysis: 5 Quick Steps

Industry Research: The Data-Backed Approach

Wondering what similarweb can do for you.

Here are two ways you can get started with Similarweb today!

2024 Workshops

• 2024 Homepage
• Registration
• Featured Speakers
• Invited Programs
• Teaching Sessions
• Student Programming
• Flash Talks
• Code of Conduct
• Reserve Your Room
• Plan Your Trip
• APS 2024 FAQs
• Program Committee
• Exhibits & Sponsorships
• Press Information

The APS Annual Convention includes these extended educational sessions that offer attendees the opportunity to learn research methods and techniques from prominent psychological scientists.

Workshops are open to Convention registrants only and require additional registration fees. Workshops can be added when you register for the APS Convention. If you’ve finalized your registration, please follow these instructions  to add a workshop .

Registration Fees:

THURSDAY, MAY 23

Introduction to Structural Equation Modeling in the Psychological Sciences

Presenter: Tim Hayes, Florida International University

Thursday, May 23, 9:00 AM – 12:50 PM  

Structural Equation Modeling (SEM) combines common factor analysis with multiple regression to allow researchers to assess true score relations among constructs of theoretical interest. This workshop presents an overview of the logic, implementation, and interpretation of SEMs. Topics covered include: path analysis, confirmatory factor analysis, and structural regression analysis. 

Prerequisite :

• A standard graduate course in linear regression analysis
• Software packages: lavaan (R) and Mplus.
• Bring a laptop (fully charged).

Measurement, Not Schmeasurement

Presenter: Jessica Flake, McGill University

We assume that psychological measures produce meaningful numbers: higher satisfaction scores indeed represent more satisfaction. Measurement is a fundamental part of psychological research, and our scores require thorough and transparent evaluation of their validity. This workshop will cover how to evaluate and refine scales using psychometric methods and open science practices.

• An introductory and intermediate courses in statistics with knowledge of regression.

Understanding Bayesian: An Introduction to Key Concepts

Presenter: Brian Leventhal, Jame Madison University

This workshop introduces steps of Bayesian analysis:

• specifying a prior
• using a likelihood
• forming a posterior
• making inferences

Attendees will be able to articulate major considerations of a Bayesian analysis, contrast Bayesian and Frequentist approaches, and identify components of Bayesian research.

Prerequisite:

• Participants should have an understanding of basic descriptive and inferential statistics (e.g., hypothesis testing, p-values, confidence intervals).
• Familiarity with linear regression and maximum likelihood estimation will be beneficial, but not required. 

Data Visualization in R for Researchers Who Do Not Use R

Presenters: Emily Nordmann, University of Glasgow and Wilhelmiina Toivo, University of Glasgow

Thursday, May 23, 1:00 PM – 3:50 PM  

In this session we’ll cover why you should use R for data visualization followed by an introduction to boxplots, histograms, violin-plots, density plots, bar charts, scatterplots, and more complex layered plots using ggplot. No prior knowledge of R is required.

• Please work through the workshop prep at  https://emilynordmann.github.io/aps-dataviz/index.html . This will include installing R and the necessary packages to run the workshop, as well as introducing some basic terminology to help support the session.

Dynamic Structural Equation Modeling (DSEM) in Mplus

Presenter: Ellen Hamaker, Utrecth University

Thursday, May 23, 1:00 PM – 4:50 PM           

This workshop provides a first introduction—both theoretically and practically—into dynamic structural equation modeling (DSEM). DSEM combines time series analysis, multilevel modeling, and structural equation modeling, and allows for a flexible and elegant approach to intensive longitudinal data such as obtained with daily diaries, experience sampling, and ambulatory assessments.

• Bring a laptop (fully charged) with Mplus installed or use the demo version of Mplus to participate in the practical portion.

Power Analysis in Structural Equation Modeling (SEM)

Presenter: Y. Andre Wang, University of Toronto

Thursday, May 23, 1:00 PM – 4:50 PM    

This workshop will introduce attendees to power analysis in SEM and offer a hands-on tutorial. Attendees will learn how to connect power considerations (e.g., to detect specific target effects, to assess model fit) to their research goals, and how to conduct power analysis on their own models using R or pwrSEM, a point-and-click web application.

• Bring a laptop (fully charged) with the following software installed: R, R Studio.
• Working knowledge of R is not necessary but preferred.
• Familiarity with structural equation modelling is recommended.

FRIDAY, MAY 24

Introduction to Generalized Linear Models in the Psychological Sciences

Presenter : Connor McCabe, University of Washington

Friday, May 24, 10:30 AM – 11:50 AM

This workshop provides a comprehensive introduction to generalized linear models (GLMs), a popular approach for modeling binary and count dependent variables in psychological science. It will provide accessible coverage of both theory and application, including hands-on demonstrations for data analysis and visualization in R and state-of-the-science methods for moderation analysis.

• Bring a laptop (fully charged) with R and Rstudio. (optional)
• A working knowledge of linear regression principles and at least some familiarity with the R statistical software language.

Experience Sampling Methods and Implementation

Presenter: Sabrina Thai, Brock University

Friday, May 24, 10:30 AM – 12:20 PM

Learn how to create your own experience-sampling smartphone app using ExperienceSampler, and how to integrate ExperienceSampler with existing survey software. We will also discuss issues related to conducting experience sampling studies: design decisions, best practices, data organization, and data analysis. 

Prerequisite :  

• Bring your laptops (fully charged) to follow along with the workshop slides.

A Beginner’s Guide to Qualitative Research

Presenter : Jaclyn Siegel, The University of Chicago

Friday, May 24, 10:30 AM – 12:20 PM

This beginner-friendly workshop will lead researchers through some of the basics of conducting qualitative research (e.g., different analytic methods, positionality), including practical elements of qualitative work (e.g., sample sizes). The workshop will end with a deep dive into thematic analysis, a popular approach to analyzing qualitative data in psychology. 

Prerequisite : none

Creating Computationally Reproducible Manuscripts

Presenter : Jason Geller, Princeton University

Friday, May 24, 1:00 PM – 2:20 PM

Open Science practices, which emphasize transparency, reproducibility, and accessibility, are indeed becoming increasingly important in the psychological community. In this workshop, you will learn to set up a reproducible workflow to create a publication-ready manuscript that combines data, R or Python code, text, and references.  

Prerequisites :

• R/Rstudio 
• Basic knowledge of R or Python

Introduction to Multilevel Modeling

Presenter : Jason Rights, The University of British Columbia

Friday, May 24, 1:00 PM – 2:50 PM

Multilevel modeling (MLM) is widely used in psychology and other fields to analyze nested data structures (e.g., students nested within schools or repeated measures nested within individuals). This workshop will provide a brief introduction to MLM, including both theoretical foundations as well as tools for practical application.

• Knowledge of regression modeling
• Bring a laptop (fully charged) with R & Rstudio installed

SATURDAY, MAY 25

Writing for a Popular Audience to Disseminate Your Work and Broaden Your Impact

Presenter : Andrew Devendorf, University of South Florida

Saturday, May 25, 10:30 AM – 11:50 AM

Although psychologists can benefit society by sharing their research with the public, most lack training on translating ideas into an accessible, engaging, and meaningful package. This workshop will overview the process of writing for, and pitching to, the popular press. Strategies and challenges related to scientific communication will be discussed.    

Data Storytelling Training

Presenter : Lisa Cantrell, Stories of Science

Saturday, May 25, 1:00 PM – 2:20 PM

How do you make a research presentation compelling? One of the biggest secrets in science communication is this: the same narrative strategies that Hollywood uses for creating compelling movies are those that we should be using to talk about our research findings.  In this workshop, participants experience demos of research presentations told with and without storytelling components and then discuss how a story format pushes the audience’s thinking forward about the research. 

Participants will have the opportunity to practice sharing their data story in small groups and by the end of the session, participants will have a draft of their own research data story and a method for turning their future studies into data stories for presentations at conferences, job talks, and speaking engagements.

The Art of the Elevator Pitch

Presenter: Tammy Spence, Stories of Science

Saturday, May 25, 2:30 PM – 3:50 PM

The elevator pitch. We talk about it often. We say it is important for networking and sharing our research. And yet when it comes to actually doing it, we find ourselves using jargon, spending too much time on details that aren’t important, or not giving the right context for our audience. The elevator pitch is, at its core, a story that should be compelling, focused, and clear. 

This workshop is intended to guide participants step by step through building their own compelling, clear, and focused elevator pitch—specifically around their research—and then honing it to 60 seconds. By the end of the session, participants will have a draft of a 60-second elevator pitch and 1-2 short anecdotes that they can use to illustrate their research.

Privacy Overview

This paper is in the following e-collection/theme issue:

Published on 28.3.2024 in Vol 26 (2024)

Augmenting K-Means Clustering With Qualitative Data to Discover the Engagement Patterns of Older Adults With Multimorbidity When Using Digital Health Technologies: Proof-of-Concept Trial

Authors of this article:

Original Paper

• Yiyang Sheng 1 , MSc   ; 
• Raymond Bond 2 , PhD   ; 
• Rajesh Jaiswal 3 , PhD   ; 
• John Dinsmore 4 , PhD   ; 
• Julie Doyle 1 , PhD  

1 NetwellCASALA, Dundalk Institution of Technology, Dundalk, Ireland

2 School of Computing, Ulster University, Jordanstown, United Kingdom

3 School of Enterprise Computing and Digital Transformation, Technological University Dublin, Dublin, Ireland

4 Trinity Centre for Practice and Healthcare Innovation, School of Nursing and Midwifery, Trinity College Dublin, Dublin, Ireland

Corresponding Author:

Yiyang Sheng, MSc

NetwellCASALA

Dundalk Institution of Technology

Dublin Road, PJ Carrolls Building, Dundalk Institute of Technology

Co.Louth, Ireland

Dundalk, A91 K584

Phone: 353 894308214

Email: [email protected]

Background: Multiple chronic conditions (multimorbidity) are becoming more prevalent among aging populations. Digital health technologies have the potential to assist in the self-management of multimorbidity, improving the awareness and monitoring of health and well-being, supporting a better understanding of the disease, and encouraging behavior change.

Objective: The aim of this study was to analyze how 60 older adults (mean age 74, SD 6.4; range 65-92 years) with multimorbidity engaged with digital symptom and well-being monitoring when using a digital health platform over a period of approximately 12 months.

Methods: Principal component analysis and clustering analysis were used to group participants based on their levels of engagement, and the data analysis focused on characteristics (eg, age, sex, and chronic health conditions), engagement outcomes, and symptom outcomes of the different clusters that were discovered.

Results: Three clusters were identified: the typical user group, the least engaged user group, and the highly engaged user group. Our findings show that age, sex, and the types of chronic health conditions do not influence engagement. The 3 primary factors influencing engagement were whether the same device was used to submit different health and well-being parameters, the number of manual operations required to take a reading, and the daily routine of the participants. The findings also indicate that higher levels of engagement may improve the participants’ outcomes (eg, reduce symptom exacerbation and increase physical activity).

Conclusions: The findings indicate potential factors that influence older adult engagement with digital health technologies for home-based multimorbidity self-management. The least engaged user groups showed decreased health and well-being outcomes related to multimorbidity self-management. Addressing the factors highlighted in this study in the design and implementation of home-based digital health technologies may improve symptom management and physical activity outcomes for older adults self-managing multimorbidity.

Introduction

According to the United Nations, the number of people aged ≥65 years is growing faster than all other age groups [ 1 ]. The worldwide population of people aged ≥65 years will increase from approximately 550 million in 2000 to 973 million in 2030 [ 2 ]. Furthermore, by 2050, approximately 16% of the world’s population will be aged >65 years, whereas 426 million people will be aged >80 years [ 1 ]. Living longer is a great benefit to today’s society. However, this comes with several challenges. Aging can be associated with many health problems, including multimorbidity (ie, the presence of ≥2 chronic conditions) [ 3 ]. The prevalence rate of multimorbidity among older adults is estimated to be between 55% and 98%, and the factors associated with multimorbidity are older age, female sex, and low socioeconomic status [ 4 ]. In the United States, almost 75% of older adults have multimorbidity [ 5 ], and it was estimated that 50 million people in the European Union were living with multimorbidity in 2015 [ 6 ]. Likewise, the prevalence rate of multimorbidity is 69.3% among older adults in China [ 5 ].

Home-based self-management for chronic health conditions involves actions and behaviors that protect and promote good health care practices comprising the management of physical, emotional, and social care [ 7 ]. Engaging in self-management can help older adults understand and manage their health conditions, prevent illness, and promote wellness [ 7 , 8 ]. However, self-management for older adults with multimorbidity is a long-term, complex, and challenging mission [ 9 , 10 ]. There are numerous self-care tasks to engage in, which can be very complicated, especially for people with multiple chronic health conditions. Furthermore, the severity of the disease can negatively impact a person’s ability to engage in self-management [ 10 ].

Digital home-based health technologies have the potential to support better engagement with self-management interventions, such as the monitoring of symptom and well-being parameters as well as medication adherence [ 10 , 11 ]. Such technologies can help older adults understand their disease or diseases, respond to changes, and communicate with health care providers [ 12 - 14 ]. Furthermore, digital health technologies can be tailored to individual motivations and personal needs [ 13 ], which can improve sustained use [ 15 ] and result in people feeling supported [ 16 ]. Digital self-management can also create better opportunities for adoption and adherence in the long term compared with paper booklet self-management [ 16 ]. Moreover, digital health technologies, such as small wearable monitoring devices, can increase the frequency of symptom monitoring for patients with minimal stress compared with symptom monitoring with manual notifications [ 17 ].

A large body of research implements data mining and machine learning algorithms using data acquired from home-based health care data sets. Data mining techniques, such as data visualization, clustering, classification, and prediction, to name a few, can help researchers understand users, behaviors, and health care phenomena by identifying novel, interesting patterns. These techniques can also be used to build predictive models [ 18 - 21 ]. In addition, data mining techniques can help in designing health care management systems and tracking the state of a person’s chronic disease, resulting in appropriate interventions and a reduction in hospital admissions [ 18 , 22 ]. Vast amounts of data can be generated when users interact with digital health technologies, which provides an opportunity to understand chronic illnesses as well as elucidate how users engage with digital health technologies in the real world. Armstrong et al [ 23 ] used the k-means algorithm to identify previously unknown patterns of clinical characteristics in home care rehabilitation services. The authors used k-means cluster analysis to analyze data from 150,253 clients and discovered new insights into the clients’ characteristics and their needs, which led to more appropriate rehabilitation services for home care clients. Madigan and Curet [ 22 ] used classification and regression trees to investigate a home-based health care data set that comprised 580 patients who had 3 specific conditions: chronic obstructive pulmonary disease (COPD), heart failure (HF), and hip replacement. They found that data mining methods identified the dependencies and interactions that influence the results, thereby improving the accuracy of risk adjustment methods and establishing practical benchmarks [ 22 ]. Other research [ 24 ] has developed a flow diagram of a proposed platform by using machine learning methods to analyze multiple health care data sets, including medical images as well as diagnostic and voice records. The authors believe that the system could help people in resource-limited areas, which have lower ratios of physicians and hospitals, to diagnose diseases such as breast cancer, heart disease (HD), diabetes, and liver disease at a lower cost and in less time than local hospitals. In the study, the accuracy of disease detection was >95% [ 24 ].

There are many different approaches to clustering analysis of health care data sets, such as k-means, density-based spatial clustering of applications with noise, agglomerative hierarchical clustering, self-organizing maps, partitioning around medoids algorithm, hybrid hierarchical clustering, and so on [ 25 - 28 ]. K-means clustering is 1 of the most commonly used clustering or unsupervised machine learning algorithms [ 19 , 29 ], and it is relatively easy to implement and relatively fast [ 30 - 32 ]. In addition, k-means has been used in research studies related to chronic health conditions such as diabetes [ 33 ], COPD [ 34 , 35 ], and HF [ 36 ]; for example, a cloud-based framework with k-means clustering technique has been used for the diagnosis of diabetes and was found to be more efficient and suitable for handling extensive data sets in cloud computing platforms than hierarchical clustering [ 32 ]. Violán et al [ 37 ] analyzed data from 408,994 patients aged 45 to 64 years with multimorbidity using k-means clustering to ascertain multimorbidity patterns. The authors stratified the k-means clustering analysis by sex, and 6 multimorbidity patterns were found for each sex. They also suggest that clusters identified by multimorbidity patterns obtained using nonhierarchical clustering analysis (eg, k-means and k-medoids) are more consistent with clinical practice [ 37 ].

The majority of data mining studies on chronic health conditions focus on the diseases themselves and their symptoms; there is less exploration of the patterns of engagement of persons with multimorbidity with digital health technologies. However, data mining and machine learning are excellent ways to understand users’ engagement patterns with digital health technologies. A study by McCauley et al [ 38 ] compared clustering analysis of the user interaction event log data from a reminiscence mobile app that was designed for people living with dementia. In addition to performing quantitative user interaction log analysis, the authors also gathered data on the qualitative experience of users. The study showed the benefits of using data mining to analyze the user log data with complementary qualitative data analysis [ 38 ]. This is a research challenge where both quantitative and qualitative methods can be combined to fully understand users; for example, the quantitative analysis of the user event data can tell us about use patterns, the preferred times of day to use the app, the feature use, and so on, but qualitative data (eg, user interviews) are necessary to understand why these use patterns exist.

The aim of this study was to analyze how older adults with multimorbidity engage with digital symptom and health monitoring over a period of approximately 12 months using a digital health platform. In this study, user log data of engagement with digital health technology and user interview qualitative data were examined to explore the patterns of engagement. K-means clustering was used to analyze the user log data. The study had four research questions: (1) How do clusters differ in terms of participant characteristics such as age, sex, and health conditions? (2) How do clusters differ in terms of patterns of engagement, such as the number of days a week participants take readings (eg, weight and blood pressure [BP])? (3) How do engagement rates with the different devices correlate with each other (determined by analyzing the weekly submissions of every parameter and the interviews of participants)? and (4) How do engagement rates affect participants’ health condition symptoms, such as BP, blood glucose (BG) level, weight, peripheral oxygen saturation (SpO 2 ) level, and physical activity (PA)?

The study was a proof-of-concept trial with an action research design and mixed methods approach. Action research is a period of investigation that “describes, interprets, and explains social situations while executing a change intervention aimed at improvement and involvement” [ 39 ]. An action research approach supports the generation of solutions to practical problems while using methods to understand the contexts of care as well as the needs and experiences of participants.

Recruitment and Sample

Although 120 participants consented to take part across Ireland and Belgium, this paper reports on data from 60 Irish older adults with multiple chronic health conditions (≥2 of the following: COPD, HF, HD, and diabetes). Participants were recruited through purposive sampling and from multiple sources, including through health care organizations (general practitioner clinics and specialist clinics), relevant older adult networks, chronic disease support groups, social media, and local newspaper advertising. Recruitment strategies included the use of study flyers and advertisements as well as giving talks and platform demonstrations.

Sources of Data

The data set was collected during the Integrated Technology Systems for Proactive Patient Centred Care (ProACT) project proof-of-concept trial. As the trial was a proof-of-concept of a novel digital health platform, the main goal was to understand how the platform worked or did not work, rather than whether it worked. Thus, to determine sample size, a pragmatic approach was taken in line with two important factors: (1) Is the sample size large enough to provide a reliable analysis of the ecosystem? and (2) Is the sample size small enough to be financially feasible? The literature suggests that overall sample size in proof-of-concept digital health trials is low. A review of 1030 studies on technical interventions for management of chronic disease that focused on HF (436 studies), stroke (422 studies), and COPD (172 studies) suggested that robust sample sizes were 17 for COPD, 19 for HF, and 21 for stroke [ 40 ]. Full details on the study protocol can be found in the study by Dinsmore et al [ 41 ].

Participants used a suite of sensor devices (ie, BP monitors, weight scales, glucometers, pulse oximeters, and activity watches) and a tablet app to monitor their health conditions and well-being. All participants received a smartwatch to measure PA levels and sleep, a BP monitor to measure BP and pulse rate, and a weight scale. A BG meter was provided to participants with diabetes, and a pulse oximeter was provided to those with COPD to measure SpO 2 levels. In addition, all participants received an iPad with a custom-designed app, the ProACT CareApp, that allowed users to view their data, provide self-report (SR) data on symptoms that could not be easily captured through a sensor (eg, breathlessness and edema) and well-being (eg, mood and satisfaction with social life), receive targeted education based on their current health status, set PA goals, and share their data with others. The ProACT platform was designed and developed following an extensive user-centered design process. This involved interviews, focus groups, co-design sessions (hands-on design activities with participants), and usability testing before the platform’s deployment in the trial. A total of 58 people with multimorbidity and 106 care network participants, including informal carers, formal carers, and health care professionals, took part in this process. Findings from the user-centered design process have been published elsewhere [ 42 , 43 ]. More detailed information about the full ProACT platform and the CareApp used by participants can be found in the study by Doyle et al [ 44 ].

The study took place between April 1, 2018, and June 30, 2019. Participants in the trial typically participated for 12 months, although some stayed on for 14 months and others for 9 months (in the case of those who entered the trial later). One of the trial objectives was to understand real-world engagement. Therefore, participants were asked to take readings with the devices and provide SR data in the ProACT CareApp whenever they wished (not necessarily daily). As part of the trial, participants were assisted by technical help desk staff who responded to questions about the technology, and home visits were conducted as needed to resolve issues. In addition, a clinical triage service monitored the participants’ readings and contacted them in instances of abnormal parameter values (eg, high BP and low SpO 2 levels) [ 45 ]. Participants also received a monthly check-in telephone call from 1 of the triage nurses.

Table 1 outlines the types of health and well-being metrics that were collected, as well as the collection method and the number of participants who collected that type of data. The health and well-being metrics were determined from the interviews and focus groups held with health care professionals during the design of the ProACT platform to determine the most important symptom and well-being parameters to monitor across the health conditions of interest [ 42 ]. Off-the-shelf digital devices manufactured by 2 providers, Withings and iHealth, were used during the trial. Data from these providers were extracted into a custom platform called Context-Aware Broker and Inference Engine–Subject Information Management System (CABIE-SIMS), which includes a data aggregator for storing health and well-being data. All devices require the user to interact with them in some way. However, some devices needed more interaction than others (eg, taking a BG reading involved several steps, but PA and sleep only required participants to open the activity watch app to sync the relevant data). The activity watch was supposed to synchronize automatically without user interaction. However, inconsistencies with syncing meant that users were advised to open the Withings app to sync their data. The CABIE-SIMS platform would display the readings in near real time, apart from PA data, which were collected at regular intervals throughout the day, whereas sleep data were gathered every morning. Table 1 lists the types of data that were collected and the number of participants who collected them. In addition, semistructured interviews were conducted with all participants at 4 time points throughout the trial to understand their experience of using the ProACT platform. Although a full qualitative thematic analysis was outside the scope of this study and was reported on elsewhere [ 44 ], interview transcripts for participants of interest to the analysis presented in this paper were reviewed as part of this study to provide an enhanced understanding of the results.

a SpO 2 : peripheral oxygen saturation.

b HF: heart failure.

c ProACT: Integrated Technology Systems for Proactive Patient Centred Care.

d CABIE-SIMS: Context-Aware Broker and Inference Engine–Subject Information Management System.

e COPD: chronic obstructive pulmonary disease.

Data Analysis Methods

The original data set in the CABIE-SIMS platform was formatted using the JSON format. As a first step, a JSON-to-CSV file converter was used to make the data set more accessible for data analysis. The main focus was on dealing with duplicate data and missing data during the data cleaning phase. Data duplication might occur when a user uploads their SpO 2 reading 3 times in 2 minutes as a result of mispressing the button. In such cases, only 1 record was added to the cleaned data file. As for missing data, the data set file comprised “N/A” (not available) values for all missing data.

The cleaned data set was preprocessed using Microsoft Excel, the R programming language (R Foundation for Statistical Computing), and RStudio (Posit Software, PBC). The preprocessed data set included participants’ details (ID, sex, age, and chronic health conditions) and the number of days of weekly submissions of every parameter (BP, pulse rate, SpO 2 level, BG level, weight, PA, SR data, and sleep). All analyses (including correlation analysis, principal component analysis [PCA], k-means clustering, 2-tailed t test, and 1-way ANOVA) were implemented in the R programming language and RStudio.

After performing Shapiro-Wilk normality tests on the data submitted each week, we found that the data were not normally distributed. Therefore, Spearman correlation was used to check the correlation among the parameters. Correlation analysis and PCA were used to determine which portions of the data would be included in the k-means clustering. Correlation analysis determined which characteristics or parameters should be selected, and PCA determined the number of dimensions that should be selected as features for clustering. In the clustering process, the weekly submission of each parameter was considered as an independent variable for the discovery of participant clusters, and the outcome of the clustering was a categorical taxonomy that was used to label the 3 discovered clusters. Similarly, the Shapiro-Wilk test was conducted to check the normality of the variables in each group. It was found that most of the variables in each group were normally distributed, and only the weight data submission records of cluster 3, the PA data submission records of cluster 2, the SR data submission records of cluster 3, and the sleep data submission records of cluster 1 were not normally distributed. Therefore, the 2-tailed t test and 1-way ANOVA were used to compare different groups of variables. The 2-tailed t test was used to compare 2 groups of variables, whereas 1-way ANOVA was used to compare ≥2 groups of variables. P values >.05 indicated that there were no statistically significant differences among the groups of variables [ 46 ].

As for the qualitative data from the interviews, we performed keyword searches after a review of the entire interview; for example, when the data analysis was related to BP and weight monitoring, a search with the keywords “blood pressure,” “weight,” or “scale” was performed to identify relevant information. In addition, when the aim was to understand the impact of digital health care technology, we focused on specific questions in the second interview, such as “Has it had any impact on the management of your health?”

Ethical Considerations

Ethics approval was received from 3 ethics committees: the Health Service Executive North East Area Research Ethics Committee, the School of Health and Science Research Ethics Committee at Dundalk Institute of Technology, and the Faculty of Health Sciences Research Ethics Committee at Trinity College Dublin. All procedures were in line with the European Union’s General Data Protection Regulation for research projects, with the platform and trial methods and procedures undergoing data protection impact assessments. Written informed consent was obtained on an individual basis from participants in accordance with legal and ethics guidelines after a careful explanation of the study and the provision of patient information and informed consent forms in plain language. All participants were informed of their right to withdraw from the study at any time without having to provide a reason. Participants were not compensated for their time. Data stored within the CABIE-SIMS platform were identifiable because they were shared (with the participant’s consent) with the clinical triage teams and health care professionals. This was clearly outlined in the participant information leaflet and consent form. However, the data set that was extracted for the purpose of the analysis presented in this paper was pseudonymized.

Participants

A total of 60 older adults were enrolled in the study. The average age of participants was 74 (SD 6.4; range 65-92) years; 60% (36) were male individuals, and 40% (24/60) were female individuals. The most common combination of health conditions was diabetes and HD (30/60, 50%), which was followed by COPD and HD (16/60, 27%); HF and HD (7/60, 12%); diabetes and COPD (3/60, 5%); diabetes and HF (1/60, 2%); COPD and HF (1/60, 2%); HF, HD, and COPD (1/60, 2%); and COPD, HD, and diabetes (1/60, 2%). Of the 60 participants, 11 (18%) had HF, 55 (92%) had HD, 22 (37%) had COPD, and 31 (52%) had diabetes. Over the course of the trial, of the 60 participants, 8 (13%) withdrew, and 3 (5%) died. However, this study included data from all participants in the beginning, as long as the participant had at least 1 piece of data. Hence, of the 60 participants, we included 56 (93%) in our analysis, whereas 4 (7%) were excluded because no data were recorded.

Correlation of Submission Parameters

To help determine which distinct use characteristics or parameters (such as the weekly frequency of BP data submissions) should be selected as features for clustering, the correlations among the parameters were calculated. Figure 1 shows the correlation matrix for all parameter weekly submissions (days). In this study, a moderate correlation (correlation coefficient between 0.3 to 0.7 and −0.7 to −0.3) [ 47 , 48 ] was chosen as the standard for selecting parameters. First, every participant received a BP monitor to measure BP, and pulse rate was collected as part of the BP measurement. Moreover, the correlation coefficient between BP and pulse rate was 0.93, a strong correlation. In this case, BP was selected for clustering rather than pulse rate. As for the other parameters, the correlations between BP and weight (0.51), PA (0.55), SR data (0.41), and sleep (0.55) were moderate, whereas the correlations between BP and SpO 2 level (0.05) and BG (0.24) were weak. In addition, the correlations between SpO 2 level and weight (−0.25), PA (0.16), SR data (0.29), and sleep (−0.24) were weak. Therefore, SpO 2 level was not selected for clustering. Likewise, the correlations between BG and weight (0.19), PA (0.2), SR data (−0.06), and sleep (0.25) were weak. Therefore, BG was not selected for clustering. Thus, BP, weight, PA, SR data, and sleep were selected for clustering.

PCA and Clustering

The fundamental question for k-means clustering is this: how many clusters (k) should be discovered? To determine the optimum number of clusters, we further investigated the data through visualization offered by PCA. As can be seen from Figure 2 , the first 2 principal components (PCs) explain 73.6% of the variation, which is an acceptably large percentage. However, after a check of individual contributions, we found that there were 3 participants—P038, P016, and P015—who contributed substantially to PC1 and PC2. After a check of the original data set, we found that P038 submitted symptom parameters only on 1 day, and P016 submitted symptom parameters only on 2 days. Conversely, P015 submitted parameters almost every day during the trial. Therefore, P038 and P016 were omitted from clustering.

After removing the outliers (P038 and P016), we found that the first 2 PCs explain 70.5% of the variation ( Figure 3 ), which is an acceptably large percentage.

The clusters were projected into 2 dimensions as shown in Figure 4 . Each subpart in Figure 4 shows a different number of clusters (k). When k=2, the data are obviously separated into 2 big clusters. Similarly, when k=3, the clusters are still separated very well into 3 clusters. When k=4, the clusters are well separated, but compared with the subpart with 3 clusters, 2 clusters are similar, whereas cluster 1, which only has 3 participants, is a relatively small cluster. When k=5, there is some overlap between cluster 1 and cluster 2. Likewise, Figure 5 shows the optimal number of clusters using the elbow method. In view of this, we determined that 3 clusters of participants separate the data set best. The 3 clusters can be labeled as the least engaged user group (cluster 1), the highly engaged user group (cluster 2), and the typical user group (cluster 3).

In the remainder of this section, we report on the examination of the clusters with respect to participant characteristics and the weekly submissions (days) of different parameters in a visual manner to reveal potential correlations and insights. Finally, we report on the examination of the correlations among all parameters by PCA.

Participant Characteristics

As seen in Figure 6 , the distribution of age within the 3 clusters is similar, with the P value of the 1-way ANOVA being .93, because all participants in this trial were older adults. However, the median age in the cluster 3 box plot is slightly higher than the median ages in the box plots of the other 2 clusters, and the average age of cluster 2 participants (74.1 years) is lower than that of cluster 1 (74.6 years) and cluster 3 (74.8 years; Table 2 ) participants. As Table 2 shows, 6 (26%) of the 23 female participants are in cluster 1 compared with 7 (23%) of the 31 male participants. However, the male participants in cluster 2 (10/31, 32%) and cluster 3 (14/31, 45%) represent higher proportions of total male participants compared with female participants in cluster 2 (7/23, 30%) and cluster 3 (10/23, 43%). Figure 7 shows the proportion of the 4 chronic health conditions within the 3 clusters. Cluster 1 has the largest proportion of participants with COPD and the smallest proportion of participants with diabetes. Moreover, cluster 3 has the smallest proportion of participants with HF (3/24, 13%; Table 2 ).

a COPD: chronic obstructive pulmonary disease.

Participant Engagement Outcomes

Cluster 2 has the longest average enrollment time at 352 days compared with cluster 3 at 335 days and cluster 1 at 330 days. As seen in Figure 8 , the overall distribution of the BP data weekly submissions is different, with the P value of the 1-way ANOVA being 8.4 × 10 −9 . The frequency of BP data weekly submissions (days) of cluster 2 exceeds the frequencies of cluster 1 and cluster 3, which means that participants in cluster 2 have a higher frequency of BP data submissions than those in the other 2 clusters. The median and maximum of cluster 3 are higher than those of cluster 1, but the minimum of cluster 3 is lower than that of cluster 1. Likewise, as seen in Table 3 , the mean and SD of cluster 1 (mean 2.5, SD 1.4) are smaller than those of cluster 3 (mean 2.9, SD 2.9).

As Figure 9 shows, the overall distribution of the weekly submissions of weight data is different, with the P value of the 1-way ANOVA being 1.4 × 10 −13 , because the participants in cluster 2 submitted weight parameters more frequently than those in cluster 1 and cluster 3. In addition, similar to the BP data submissions, the median of cluster 3 is higher than that of cluster 1. As seen in Figure 9 , there are 3 outliers in cluster 2. The top outlier is P015, who submitted a weight reading almost every day. During the trial, this participant mentioned many times in the interviews that his goal was to lose weight and that he used the scale to check his progress:

I’ve set out to reduce my weight. The doctor has been saying to me you know there’s where you are and you should be over here. So, I’ve been using the weighing thing just to clock, to track reduction of weight. [P015]

The other 2 outliers are P051 and P053, both of whom mentioned taking their weight measurements as part of their daily routine:

Once I get up in the morning the first thing is I weigh myself. That is, the day starts off with the weight, right. [P053]

Although their frequency of weekly weight data submissions is lower than that of all other participants in cluster 2, it is still higher than that of most of the participants in the other 2 clusters.

In Table 3 , it can be observed that the average frequency of weekly submissions of PA and sleep data for every cluster is higher than the frequencies of other variables, and the SDs are relatively low. This is likely because participants only needed to open the Withings app once a day to ensure the syncing of data. However, the overall distributions of PA and sleep data submissions are different in Figure 10 and Figure 11 , with the P values of the 1-way ANOVA being 1.1 × 10 −9 and 3.7 × 10 −10 , respectively. Moreover, as Figure 10 and Figure 11 show, there are still some outliers who have a low frequency of submissions, and the box plot of cluster 1 is lower than the box plots of cluster 2 and cluster 3 in both figures. The reasons for the low frequency of submissions can mostly be explained by (1) technical issues, including internet connection issues, devices not syncing, and devices needing to be paired again; (2) participants forgetting to put the watch back on after taking it off; and (3) participants stopping using the devices (eg, some participants do not like wearing the watch while sleeping or when they go on holiday):

I was without my watch there for the last month or 3 or 4 weeks [owing to technical issues], and I missed it very badly because everything I look at the watch to tell the time, I was looking at my steps. [P042]

I don’t wear it, I told them I wouldn’t wear the watch at night, I don’t like it. [P030]

Unlike in the case of other variables, the submission of SR data through the ProACT CareApp required participants to reflect on each question and their status before selecting the appropriate answer. Participants had different questions to answer based on their health conditions; for example, participants with HF and COPD were asked to answer symptom-related questions, whereas those with diabetes were not. All participants were presented with general well-being and mood questions. Therefore, for some participants, self-reporting could possibly take more time than using the health monitoring devices. As shown in Table 3 , the frequency of average weekly submissions of SR data within the 3 clusters is relatively small and the SDs are large, which means that the frequency of SR data submissions is lower than that of other variables. Furthermore, there were approximately 5 questions asked daily about general well-being, and some participants would skip the questions if they thought the question was unnecessary or not relevant:

Researcher: And do you answer your daily questions? P027: Yeah, once a week.

Researcher: Once a week, okay. P027: But they’re the same.

As Figure 12 shows, the distribution of SR data submissions is different, with the P value of the 1-way ANOVA being .001. In Figure 12 , the median of cluster 2 is higher than the medians of the other 2 clusters, and compared with other variables, but unlike other parameters, cluster 2 also has some participants who had very low SR data submission rates (close to 0). SR data is the only parameter where cluster 1 has a higher median than cluster 3.

a Lowest submission rate across the clusters.

b Highest submission rate across the clusters.

The Correlation Among the Weekly Submissions of Different Parameters

As seen in Figure 13 , the arrows of BP and weight point to the same side of the plot, which shows a strong correlation. Likewise, PA and sleep also have a strong correlation. As noted previously, the strong correlation between PA and sleep is because the same device collected these 2 measurements, and participants only needed to sync the data once a day. By contrast, BP and weight were collected by 2 different devices but are strongly correlated. During interviews, many participants mentioned that their daily routine with the ProACT platform involved taking both BP and weight readings:

Usually in the morning when I get out of the bed, first, I go into the bathroom, wash my hands and come back, then weigh myself, do my blood pressure, do my bloods. [P008]

I now have a routine that I let the system read my watch first thing, then I do my blood pressure thing and then I do the weight. [P015]

As I said, it’s keeping me in line with my, when I dip my finger, my weight, my blood pressure. [P040]

I use it in the morning and at night for putting in the details of blood pressure in the morning and then the blood glucose at night. Yes, there’s nothing else, is there? Oh, every morning the [weight] scales. [P058]

By contrast, as shown in Figure 13 , SR data have a weak correlation with other parameters, for reasons noted earlier.

Parameter Variation Over Time

Analysis was conducted to determine any differences among the clusters in terms of symptom and well-being parameter changes over the course of the trial. Table 4 provides a description of each cluster in this regard. As Figure 14 shows, the box plot of cluster 2 is comparatively short in every time period of the trial, and the medians of cluster 2 and cluster 3 are more stable than the median of cluster 1. In addition, the median of cluster 1 is increasing over time, whereas the medians of cluster 2 and cluster 3 are decreasing and within the normal systolic BP of older adults [ 49 ] ( Figure 14 ). As can be seen in Table 5 , cluster 2 has a P value of .51 for systolic BP and a P value of .52 for diastolic BP, which are higher than the P values of cluster 1 ( P =.19 and P =.16, respectively) and cluster 3 ( P =.27 and P =.35, respectively). Therefore, participants in cluster 2, as highly engaged users, have more stable B P values than those in the other 2 clusters. By contrast, participants in cluster 1, as the least engaged users, have the most unstable B P values.

As seen in Figure 15 , the median of cluster 2 is relatively higher than the medians of the other 2 clusters. The median of cluster 3 is increasing over time. In the second and third time periods of the trial, the box plot of cluster 1 is comparatively short. Normal SpO 2 levels are between 95% and 100%, but older adults may have SpO 2 levels closer to 95% [ 50 ]. In addition, for patients with COPD, SpO 2 levels range between 88% and 92% [ 51 ]. In this case, there is not much difference in terms of SpO 2 levels, and most of the SpO 2 levels are between 90% and 95% in this study. However, the SpO 2 levels of cluster 1 and cluster 2 were maintained at a relatively high level during the trial. As for cluster 3, the SpO 2 levels were comparatively low but relatively the same as those in the other 2 clusters in the later period of the trial. Therefore, the SpO 2 levels of cluster 3 ( P =.25) are relatively unstable compared with those of cluster 1 ( P =.66) and cluster 2 ( P =.59). As such, there is little correlation between SpO 2 levels and engagement with digital health monitoring.

In relation to BG, Figure 16 shows that the box plot of cluster 2 is relatively lower than the box plots of the other 2 clusters in the second and third time periods. Moreover, the medians of cluster 2 and cluster 3 are lower than those of cluster 1 in the second and third time periods. The BG levels in cluster 2 and cluster 3 decreased at later periods of the trial compared with the beginning of the trial, but those in cluster 1 increased. Cluster 3 ( P =.25), as the typical user group, had more significant change than cluster 1 ( P =.50) and cluster 2 ( P =.41). Overall, participants with a higher engagement rate had better BG control.

In relation to weight, Figure 17 shows that the box plot of cluster 2 is lower than the box plots of the other 2 clusters and comparatively short. As Table 5 shows, the P value of cluster 2 weight data is .72, which is higher than the P values of cluster 1 (.47) and cluster 3 (.61). Therefore, participants in cluster 2 had a relatively stable weight during the trial. In addition, as seen in Figure 17 , the median weight of cluster 1 participants is decreasing, whereas that of cluster 3 participants is increasing. It is well known that there are many factors that can influence body weight, such as PA, diet, environmental factors, and so on. [ 52 ]. In this case, engagement with digital health and well-being monitoring may help control weight but the impact is not significant.

As Table 5 shows, the P value of cluster 2 PA (.049) is lower than .05, which means that there are significant differences among the 3 time slots in cluster 2. However, the median of cluster 2 PA, as seen in Figure 18 , is still higher than the medians of the other 2 clusters. In cluster 2, approximately 50% of daily PA (steps) consists of >2500 steps. Overall, participants with a higher engagement rate also had a higher level of PA.

a BP: blood pressure.

b BG: blood glucose.

c SR: self-report.

d PA: physical activity.

b SpO 2 : peripheral oxygen saturation.

c BG: blood glucose.

Principal Findings

Digital health technologies hold great promise to help older adults with multimorbidity to improve health management and health outcomes. However, such benefits can only be realized if users engage with the technology. The aim of this study was to explore the engagement patterns of older adults with multimorbidity with digital self-management by using data mining to analyze users’ weekly submission data. Three clusters were identified: cluster 1 (the least engaged user group), cluster 2 (the highly engaged user group), and cluster 3 (the typical user group). The subsequent analysis focused on how the clusters differ in terms of participant characteristics, patterns of engagement, and stabilization of health condition symptoms and well-being parameters over time, as well as how engagement rates with the different devices correlate with each other.

The key findings from the study are as follows:

• There is no significant difference in participants’ characteristics among the clusters in general. The highly engaged group had the lowest average age ( Table 4 ), and there was no significant difference with regard to sex and health conditions among these clusters. The least engaged user group had fewer male participants and participants with diabetes.
• There are 3 main factors influencing the correlations among the submission rates of different parameters. The first concerns whether the same device was used to submit the parameters, the second concerns the number of manual operations required to submit the parameter, and the third concerns the daily routine of the participants.
• Increased engagement with devices may improve the participants’ health and well-being outcomes (eg, symptoms and PA levels). However, the difference between the highly engaged user group and the typical user group was relatively minimal compared with the difference between the highly engaged user group and the least engaged user group.

Each of these findings is discussed in further detail in the following subsections.

Although the findings presented in this paper focus on engagement based on the ProACT trial participants’ use data, the interviews that were carried out as part of the trial identified additional potential factors of engagement. As reported in the study by Doyle et al [ 44 ], participants spoke about how they used the data to support their self-management (eg, taking action based on their data) and experienced various benefits, including increased knowledge of their health conditions and well-being, symptom optimization, reductions in weight, increased PA, and increased confidence to participate in certain activities as a result of health improvements. The peace of mind and encouragement provided by the clinical triage service as well as the technical support available were also identified during the interviews as potential factors positively impacting engagement [ 44 ]. In addition, the platform was found to be usable, and it imposed minimal burden on participants ( Table 1 ). These findings supplement the quantitative findings presented in this paper.

Age, Sex, Health Condition Types, and Engagement

In this study, the difference in engagement with health care technologies between the sex was not significant. Of the 23 female participants, 6 (26%) were part of the least engaged user group compared with 7 (23%) of the 31 male participants. Moreover, there were lower proportions of female participants in the highly engaged user group (7/23, 30%) and typical user group (10/23, 43%) compared with male participants (10/31, 32% and 14/31, 45%, respectively). Other research has found that engagement with mobile health technology for BP monitoring was independent of sex [ 53 ]. However, there are also some studies that show that female participants are more likely to engage with digital mental health care interventions [ 54 , 55 ]. Therefore, sex cannot be considered as a separate criterion when comparing engagement with health care technologies, and it was not found to have significant impact on engagement in this study. Regarding age, many studies have shown that younger people are more likely to use health care technologies than older adults [ 56 , 57 ]. Although all participants in our study are older adults, the highly engaged user group is the youngest group. However, there was no significant difference in age among the clusters, with some of the oldest users being part of cluster 3, the typical user cluster. Similarly, the health conditions of a participant did not significantly impact their level of engagement. Other research [ 53 ] found that participants who were highly engaged with health monitoring had higher rates of hypertension, chronic kidney disease, and hypercholesterolemia than those with lower engagement levels. Our findings indicate that the highly engaged user group had a higher proportion of participants with diabetes, and the least engaged user group had a higher proportion of participants with COPD. Further research is needed to understand why there might be differences in engagement depending on health conditions. In our study, participants with COPD also self-reported on certain symptoms, such as breathlessness, chest tightness, and sputum amount and color. Although engagement with specific questions was not explored, participants in cluster 1, the least engaged user group, self-reported more frequently than those in cluster 3, the typical user group. Our findings also indicate that participants monitoring BG level and BP experienced better symptom stabilization over time than those monitoring SpO 2 level. It has been noted that the expected benefits of technology (eg, increased safety and usefulness) and need for technology (eg, subjective health status and perception of need) are 2 important factors that can influence the acceptance and use of technology by older adults [ 58 ]. It is also well understood that engaging in monitoring BG level can help people with diabetes to better self-manage and make decisions about diet, exercise, and medication [ 59 ].

Factors Influencing Engagement

Many research studies use P values to show the level of similarity or difference among clusters [ 60 - 63 ]. For most of the engagement outcomes in this study, all clusters significantly differed, with 1-way ANOVA P <.001, with the exception being SR data ( P =.001). In addition, the 2-tailed t test P values showed that cluster 2 was significantly different from cluster 1 and cluster 3 in BP and weight data submission rates, whereas cluster 1 was significantly different from cluster 2 and cluster 3 in PA and sleep data submission rates. As for SR data submission rates, all 3 two-tailed t tests had P values >.001, meaning that there were no significant differences between any 2 of these clusters. Therefore, all 5 parameters used for clustering were separated into 3 groups based on the correlations of submission rates: 1 for BP and weight, 1 for PA and sleep, and 1 for SR data. PA and sleep data submission rates have a strong correlation because participants used the same device to record daily PA and sleeping conditions. SR data submission rates have a weak correlation with other parameters’ submission rates. Our previous research found that user retention in terms of submitting SR data was poorer than user retention in terms of using digital health devices, possibly because more manual operations are involved in the submission of SR data than other parameters or because the same questions were asked regularly, as noted by P027 in the Participant Engagement Outcomes subsection [ 64 ].

Other research that analyzed engagement with a diabetes support app found that user engagement was lower when more manual data entry was required [ 65 ]. In contrast to the other 2 groups of parameters, BP and weight data are collected using different devices. Whereas measuring BP requires using a BP monitor and manually synchronizing the data, measuring weight simply requires standing on the weight scale, and the data are automatically synchronized. Therefore, the manual operations involved in submitting BP and weight data are slightly different. However, the results showed a strong correlation between BP and weight because many participants preferred to measure both BP and weight together and incorporate taking these measurements into their daily routines. Research has indicated that if the use of a health care device becomes a regular routine, then participants will use it without consciously thinking about it [ 66 ]. Likewise, Yuan et al [ 67 ] note that integrating health apps into people’s daily activities and forming regular habits can increase people’s willingness to continue using the apps. However, participants using health care technology for long periods of time might become less receptive to exploring the system compared with using it based on the established methods to which they are accustomed [ 68 ]. In this study, many participants bundled their BP measurement with their weight measurement during their morning routine. Therefore, the engagement rates of interacting with these 2 devices were enhanced by each other. Future work could explore how to integrate additional measurements, such as monitoring SpO 2 level as well as self-reporting into this routine (eg, through prompting the user to submit these parameters while they are engaging with monitoring other parameters, such as BP and weight).

Relationship Between Engagement and Health and Well-Being Outcomes

Our third finding indicates that higher levels of engagement with digital health monitoring may result in better outcomes, such as symptom stabilization and increased PA levels. Milani et al [ 69 ] found that digital health care interventions can help people achieve BP control and improve hypertension control compared with usual care. In their study, users in the digital intervention group took an average of 4.2 readings a week. Compared with our study, this rate is lower than that of cluster 2 (5.7), the highly engaged user group, but higher than cluster 1 (2.5) and cluster 3 (2.9) rates. In our study, participants with a higher engagement rate experienced more stable BP, and for the majority of these participants (34/41, 83%), levels were maintained within the recommended thresholds of 140/90 mm Hg [ 70 ]. Many studies have shown that as engagement in digital diabetes interventions increases, patients will experience greater reductions in BG level compared with those with lower engagement [ 71 , 72 ]. However, in our study, BG levels in both the highly engaged user group (cluster 2) and the least engaged user group (cluster 1) increased in the later stages of the trial. Only the BG levels of the typical user group (cluster 3) decreased over time, which could be because the cluster 3 participants performed more PA in the later stages of the trial than during other time periods, as Figure 18 shows. Cluster 2, the highly engaged user group, maintained a relatively high level of PA during the trial period, although it continued to decline throughout the trial. Other research shows that more PA can also lead to better weight control and management [ 73 , 74 ], which could be 1 of the reasons why cluster 2 participants maintained their weight.

Limitations

There are some limitations to the research presented in this paper. First, although the sample size (n=60) was relatively large for a digital health study, the sample sizes for some parameters were small because not all participants monitored all parameters. Second, the participants were clustered based on weekly submissions of parameters only. If more features were included in clustering, such as submission intervals, participants could be grouped differently. It should also be pointed out that correlation is not a causality with respect to analyzing engagement rates with outcomes.

Conclusions

This study presents findings after the clustering of a data set that was generated from a longitudinal study of older adults using a digital health technology platform (ProACT) to self-manage multiple chronic health conditions. The highly engaged user group cluster (includes 17/54, 31% of users) had the lowest average age and highest frequency of submissions for every parameter. Engagement with digital health care technologies may also influence health and well-being outcomes (eg, symptoms and PA levels). The least engaged user group in our study had relatively poorer outcomes. However, the difference between the outcomes of the highly engaged user group and those of the typical user group is relatively small. There are 3 possible reasons for the correlations between the submission rates of parameters and devices. First, if 2 parameters are collected by the same device, they usually have a strong correlation, and users will engage with both equally. Second, the devices that involve fewer steps and parameters with less manual data entry will have a weak correlation with those devices that require more manual operations and data entry. Finally, participants’ daily routines also influence the correlations among devices; for example, in this study, many participants had developed a daily routine to weigh themselves after measuring their BP, which led to a strong correlation between BP and weight data submission rates. Future work should explore how to integrate the monitoring of additional parameters into a user’s routine and whether additional characteristics, such as the severity of disease or technical proficiency, impact engagement.

Acknowledgments

This work was part funded by the Integrated Technology Systems for Proactive Patient Centred Care (ProACT) project and has received funding from the European Union (EU)–funded Horizon 2020 research and innovation program (689996). This work was part funded by the EU’s INTERREG VA program, managed by the Special EU Programs Body through the Eastern Corridor Medical Engineering Centre (ECME) project. This work was part funded by the Scaling European Citizen Driven Transferable and Transformative Digital Health (SEURO) project and has received funding from the EU-funded Horizon 2020 research and innovation program (945449). This work was part funded by the COVID-19 Relief for Researchers Scheme set up by Ireland’s Higher Education Authority. The authors would like to sincerely thank all the participants of this research for their valuable time.

Conflicts of Interest

None declared.

• Ageing. United Nations. 2020. URL: https://www.un.org/en/global-issues/ageing [accessed 2022-01-13]
• Centers for Disease Control and Prevention (CDC). Trends in aging--United States and worldwide. MMWR Morb Mortal Wkly Rep. Feb 14, 2003;52(6):101-104. [ FREE Full text ] [ Medline ]
• Valderas JM, Starfield B, Sibbald B, Salisbury C, Roland M. Defining comorbidity: implications for understanding health and health services. Ann Fam Med. Jul 13, 2009;7(4):357-363. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Marengoni A, Angleman S, Melis R, Mangialasche F, Karp A, Garmen A, et al. Aging with multimorbidity: a systematic review of the literature. Ageing Res Rev. Sep 2011;10(4):430-439. [ CrossRef ] [ Medline ]
• Zhang L, Ma L, Sun F, Tang Z, Chan P. A multicenter study of multimorbidity in older adult inpatients in China. J Nutr Health Aging. Mar 2020;24(3):269-276. [ FREE Full text ] [ CrossRef ] [ Medline ]
• van der Heide I, Snoeijs S, Melchiorre MG, Quattrini S, Boerma W, Schellevis F, et al. Innovating care for people with multiple chronic conditions in Europe. Innovating Care for people with Multiple Chronic Conditions in Europe (ICARE4EU). 2015. URL: http:/​/www.​icare4eu.org/​pdf/​Innovating-care-for-people-with-multiple-chronic-conditions-in-Europe.​pdf [accessed 2024-01-29]
• Bartlett SJ, Lambert SD, McCusker J, Yaffe M, de Raad M, Belzile E, et al. Self-management across chronic diseases: targeting education and support needs. Patient Educ Couns. Feb 2020;103(2):398-404. [ CrossRef ] [ Medline ]
• Anekwe TD, Rahkovsky I. Self-management: a comprehensive approach to management of chronic conditions. Am J Public Health. Dec 2018;108(S6):S430-S436. [ CrossRef ]
• Barlow J, Wright C, Sheasby J, Turner A, Hainsworth J. Self-management approaches for people with chronic conditions: a review. Patient Educ Couns. 2002;48(2):177-187. [ CrossRef ] [ Medline ]
• Setiawan IM, Zhou L, Alfikri Z, Saptono A, Fairman AD, Dicianno BE, et al. An adaptive mobile health system to support self-management for persons with chronic conditions and disabilities: usability and feasibility studie. JMIR Form Res. Apr 25, 2019;3(2):e12982. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Alanzi T. mHealth for diabetes self-management in the Kingdom of Saudi Arabia: barriers and solutions. J Multidiscip Healthc. 2018;11:535-546. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Nunes F, Verdezoto N, Fitzpatrick G, Kyng M, Grönvall E, Storni C. Self-care technologies in HCI. ACM Trans Comput Hum Interact. Dec 14, 2015;22(6):1-45. [ CrossRef ]
• Klasnja P, Kendall L, Pratt W, Blondon K. Long-term engagement with health-management technology: a dynamic process in diabetes. AMIA Annu Symp Proc. 2015;2015:756-765. [ FREE Full text ] [ Medline ]
• Talboom-Kamp EP, Verdijk NA, Harmans LM, Numans ME, Chavannes NH. An eHealth platform to manage chronic disease in primary care: an innovative approach. Interact J Med Res. Feb 09, 2016;5(1):e5. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tighe SA, Ball K, Kensing F, Kayser L, Rawstorn JC, Maddison R. Toward a digital platform for the self-management of noncommunicable disease: systematic review of platform-like interventions. J Med Internet Res. Oct 28, 2020;22(10):e16774. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Pettersson B, Wiklund M, Janols R, Lindgren H, Lundin-Olsson L, Skelton DA, et al. 'Managing pieces of a personal puzzle' - older people's experiences of self-management falls prevention exercise guided by a digital program or a booklet. BMC Geriatr. Feb 18, 2019;19(1):43. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kario K. Management of hypertension in the digital era: mall wearable monitoring devices for remote blood pressure monitoring. Hypertension. Sep 2020;76(3):640-650. [ CrossRef ]
• Koh HC, Tan G. Data mining applications in healthcare. J Healthc Inf Manag. 2005;19(2):64-72. [ Medline ]
• Alsayat A, El-Sayed H. Efficient genetic k-means clustering for health care knowledge discovery. In: Proceedings of the 14th International Conference on Software Engineering Research, Management and Applications. 2016. Presented at: SERA '16; June 8-10, 2016;45-52; Towson, MD. URL: https://ieeexplore.ieee.org/document/7516127 [ CrossRef ]
• Katsis Y, Balac N, Chapman D, Kapoor M, Block J, Griswold WG, et al. Big data techniques for public health: a case study. In: Proceedings of the 2017 IEEE/ACM International Conference on Connected Health: Applications, Systems and Engineering Technologies. 2017. Presented at: CHASE '17; July 17-19, 2017;222-231; Philadelphia, PA. URL: https://ieeexplore.ieee.org/document/8010636 [ CrossRef ]
• Elbattah M, Molloy O. Data-driven patient segmentation using k-means clustering: the case of hip fracture care in Ireland. In: Proceedings of the 2017 Australasian Computer Science Week Multiconference. 2017. Presented at: ACSW '17; January 30- February 3, 2017;1-8; Geelong, Australia. URL: https://dl.acm.org/doi/10.1145/3014812.3014874 [ CrossRef ]
• Madigan EA, Curet OL. A data mining approach in home healthcare: outcomes and service use. BMC Health Serv Res. Feb 24, 2006;6(1):18. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Armstrong JJ, Zhu M, Hirdes JP, Stolee P. K-means cluster analysis of rehabilitation service users in the home health care system of Ontario: examining the heterogeneity of a complex geriatric population. Arch Phys Med Rehabil. Dec 2012;93(12):2198-2205. [ CrossRef ] [ Medline ]
• Islam MS, Liu D, Wang K, Zhou P, Yu L, Wu D. A case study of healthcare platform using big data analytics and machine learning. In: Proceedings of the 2019 3rd High Performance Computing and Cluster Technologies Conference. 2019. Presented at: HPCCT '19; June 22-24, 2019;139-146; Guangzhou, China. URL: https://dl.acm.org/doi/10.1145/3341069.3342980 [ CrossRef ]
• Delias P, Doumpos M, Grigoroudis E, Manolitzas P, Matsatsinis N. Supporting healthcare management decisions via robust clustering of event logs. Knowl Based Syst. Aug 2015;84:203-213. [ CrossRef ]
• Lefèvre T, Rondet C, Parizot I, Chauvin P. Applying multivariate clustering techniques to health data: the 4 types of healthcare utilization in the Paris metropolitan area. PLoS One. Dec 15, 2014;9(12):e115064. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ahmad P, Qamar S, Qasim Afser Rizvi S. Techniques of data mining in healthcare: a review. Int J Comput Appl. Jun 18, 2015;120(15):38-50. [ CrossRef ]
• Mahoto NA, Shaikh FK, Ansari AQ. Exploitation of clustering techniques in transactional healthcare data. Mehran Univ Res J Eng Technol. 2014;33(1):77-92.
• Zahi S, Achchab B. Clustering of the population benefiting from health insurance using k-means. In: Proceedings of the 4th International Conference on Smart City Applications. 2019. Presented at: SCA '19; October 2-4, 2019;1-6; Casablanca, Morocco. URL: https://dl.acm.org/doi/abs/10.1145/3368756.3369103 [ CrossRef ]
• Jain AK. Data clustering: 50 years beyond k-means. Pattern Recognit Lett. Jun 2010;31(8):651-666. [ CrossRef ]
• Silitonga P. Clustering of patient disease data by using k-means clustering. Int J Comput Sci Inf Sec. 2017;15(7):219-221. [ FREE Full text ]
• Shakeel PM, Baskar S, Dhulipala VR, Jaber MM. Cloud based framework for diagnosis of diabetes mellitus using k-means clustering. Health Inf Sci Syst. Dec 24, 2018;6(1):16. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Berry E, Davies M, Dempster M. Illness perception clusters and relationship quality are associated with diabetes distress in adults with type 2 diabetes. Psychol Health Med. Oct 19, 2017;22(9):1118-1126. [ CrossRef ] [ Medline ]
• Harrison S, Robertson N, Graham C, Williams J, Steiner M, Morgan M, et al. Can we identify patients with different illness schema following an acute exacerbation of COPD: a cluster analysis. Respir Med. Feb 2014;108(2):319-328. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lopes AC, Xavier RF, Ac Pereira AC, Stelmach R, Fernandes FL, Harrison SL, et al. Identifying COPD patients at risk for worse symptoms, HRQoL, and self-efficacy: a cluster analysis. Chronic Illn. Jun 17, 2019;15(2):138-148. [ CrossRef ] [ Medline ]
• Cikes M, Sanchez-Martinez S, Claggett B, Duchateau N, Piella G, Butakoff C, et al. Machine learning-based phenogrouping in heart failure to identify responders to cardiac resynchronization therapy. Eur J Heart Fail. Jan 17, 2019;21(1):74-85. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Violán C, Roso-Llorach A, Foguet-Boreu Q, Guisado-Clavero M, Pons-Vigués M, Pujol-Ribera E, et al. Multimorbidity patterns with K-means nonhierarchical cluster analysis. BMC Fam Pract. Jul 03, 2018;19(1):108. [ FREE Full text ] [ CrossRef ] [ Medline ]
• McCauley CO, Bond RB, Ryan A, Mulvenna MD, Laird L, Gibson A, et al. Evaluating user engagement with a reminiscence app using cross-comparative analysis of user event logs and qualitative data. Cyberpsychol Behav Soc Netw. Aug 2019;22(8):543-551. [ CrossRef ] [ Medline ]
• Waterman H, Tillen D, Dickson R, de Koning K. Action research: a systematic review and guidance for assessment. Health Technol Assess. 2001;5(23):iii-157. [ FREE Full text ] [ Medline ]
• Bashshur RL, Shannon GW, Smith BR, Alverson DC, Antoniotti N, Barsan WG, et al. The empirical foundations of telemedicine interventions for chronic disease management. Telemed J E Health. Sep 2014;20(9):769-800. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Dinsmore J, Hannigan C, Smith S, Murphy E, Kuiper JM, O'Byrne E, et al. A digital health platform for integrated and proactive patient-centered multimorbidity self-management and care (ProACT): protocol for an action research proof-of-concept trial. JMIR Res Protoc. Dec 15, 2021;10(12):e22125. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Doyle J, Murphy E, Kuiper J, Smith S, Hannigan C, Jacobs A, et al. Managing multimorbidity: identifying design requirements for a digital self-management tool to support older adults with multiple chronic conditions. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 2019. Presented at: CHI '19; May 4-9, 2019;1-14; Glasgow, Scotland. URL: https://dl.acm.org/doi/10.1145/3290605.3300629 [ CrossRef ]
• Doyle J, Murphy E, Hannigan C, Smith S, Bettencourt-Silva J, Dinsmore J. Designing digital goal support systems for multimorbidity self-management: insights from older adults and their care network. In: Proceedings of the 12th EAI International Conference on Pervasive Computing Technologies for Healthcare. 2018. Presented at: PervasiveHealth '18; May 21-24, 2018;168-177; New York, NY. URL: https://dl.acm.org/doi/10.1145/3240925.3240982 [ CrossRef ]
• Doyle J, Murphy E, Gavin S, Pascale A, Deparis S, Tommasi P, et al. A digital platform to support self-management of multiple chronic conditions (ProACT): findings in relation to engagement during a one-year proof-of-concept trial. J Med Internet Res. Dec 15, 2021;23(12):e22672. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Doyle J, McAleer P, van Leeuwen C, Smith S, Murphy E, Sillevis Smitt M, et al. The role of phone-based triage nurses in supporting older adults with multimorbidity to digitally self-manage - findings from the ProACT proof-of-concept study. Digit Health. Oct 09, 2022;8:20552076221131140. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ross A, Willson VL. One-way anova. In: Ross A, Willson VL, editors. Basic and Advanced Statistical Tests: Writing Results Sections and Creating Tables and Figures. Cham, Switzerland. Springer; 2017;21-24.
• Dancey CP, Reidy J. Statistics without Maths for Psychology. Upper Saddle River, NJ. Prentice Hall; 2007.
• Akoglu H. User's guide to correlation coefficients. Turk J Emerg Med. Sep 2018;18(3):91-93. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Master AM, Dublin LI, Marks HH. The normal blood pressure range and its clinical implications. J Am Med Assoc. Aug 26, 1950;143(17):1464-1470. [ CrossRef ] [ Medline ]
• Cunha JP. What is a good oxygen rate by age? eMedicineHealth. URL: https://www.emedicinehealth.com/what_is_a_good_ oxygen_rate_by_age/article_em.htm [accessed 2024-01-29]
• Echevarria C, Steer J, Wason J, Bourke S. Oxygen therapy and inpatient mortality in COPD exacerbation. Emerg Med J. Mar 26, 2021;38(3):170-177. [ CrossRef ] [ Medline ]
• Atkinson Jr RL, Butterfield G, Dietz W, Fernstrom J, Frank A, Hansen B. Weight Management: State of the Science and Opportunities for Military Programs. Washington, DC. National Academies Press; 2003.
• Kaplan AL, Cohen ER, Zimlichman E. Improving patient engagement in self-measured blood pressure monitoring using a mobile health technology. Health Inf Sci Syst. Dec 07, 2017;5(1):4. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mikolasek M, Witt CM, Barth J. Adherence to a mindfulness and relaxation self-care app for cancer patients: mixed-methods feasibility study. JMIR Mhealth Uhealth. Dec 06, 2018;6(12):e11271. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Harjumaa M, Halttu K, Koistinen K, Oinas-Kukkonen H. User experience of mobile coaching for stress-management to tackle prevalent health complaints. In: Proceedings of the 6th Scandinavian Conference on Information Systems. 2015. Presented at: SCIS '15; August 9-12, 2015; Oulu, Finland. URL: https:/​/cris.​vtt.fi/​en/​publications/​user-experience-of-mobile-coaching-for-stress-management-to-tackl [ CrossRef ]
• Kannisto KA, Korhonen J, Adams CE, Koivunen MH, Vahlberg T, Välimäki MA. Factors associated with dropout during recruitment and follow-up periods of a mHealth-based randomized controlled trial for mobile.net to encourage treatment adherence for people with serious mental health problems. J Med Internet Res. Feb 21, 2017;19(2):e46. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Abel EA, Shimada SL, Wang K, Ramsey C, Skanderson M, Erdos J, et al. Dual use of a patient portal and clinical video telehealth by veterans with mental health diagnoses: retrospective, cross-sectional analysis. J Med Internet Res. Nov 07, 2018;20(11):e11350. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Peek ST, Wouters EJ, van Hoof J, Luijkx KG, Boeije HR, Vrijhoef HJ. Factors influencing acceptance of technology for aging in place: a systematic review. Int J Med Inform. Apr 2014;83(4):235-248. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Weinstock RS, Aleppo G, Bailey TS, Bergenstal RM, Fisher WA, Greenwood DA, et al. The role of blood glucose monitoring in diabetes management. Compendia. Oct 2022;2020(3):1-32. [ CrossRef ] [ Medline ]
• Rahman QA, Janmohamed T, Pirbaglou M, Ritvo P, Heffernan JM, Clarke H, et al. Patterns of user engagement with the mobile app, manage my pain: results of a data mining investigation. JMIR Mhealth Uhealth. Jul 12, 2017;5(7):e96. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Booth FG, R Bond R, D Mulvenna M, Cleland B, McGlade K, Rankin D, et al. Discovering and comparing types of general practitioner practices using geolocational features and prescribing behaviours by means of K-means clustering. Sci Rep. Sep 14, 2021;11(1):18289. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sulistyono MT, Pane ES, Wibawa AD, Purnomo MH. Analysis of EEG-based stroke severity groups clustering using k-means. In: Proceedings of the 2021 International Seminar on Intelligent Technology and Its Applications. 2021. Presented at: ISITIA '21; July 21-22, 2021;67-74; Surabaya, Indonesia. URL: https://ieeexplore.ieee.org/document/9502250 [ CrossRef ]
• Oskooei A, Chau SM, Weiss J, Sridhar A, Martínez MR, Michel B. DeStress: deep learning for unsupervised identification of mental stress in firefighters from heart-rate variability (HRV) data. In: Shaban-Nejad A, Michalowski M, Buckeridge DL, editors. Explainability and Interpretability: Keys to Deep Medicine. Cham, Switzerland. Springer; 2020;93-105.
• Sheng Y, Doyle J, Bond R, Jaiswal R, Gavin S, Dinsmore J. Home-based digital health technologies for older adults to self-manage multiple chronic conditions: a data-informed analysis of user engagement from a longitudinal trial. Digit Health. Sep 22, 2022;8:20552076221125957. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Böhm AK, Jensen ML, Sørensen MR, Stargardt T. Real-world evidence of user engagement with mobile health for diabetes management: longitudinal observational study. JMIR Mhealth Uhealth. Nov 06, 2020;8(11):e22212. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kim SS, Malhotra NK. A longitudinal model of continued is use: an integrative view of four mechanisms underlying postadoption phenomena. Manag Sci. May 2005;51(5):741-755. [ CrossRef ]
• Yuan S, Ma W, Kanthawala S, Peng W. Keep using my health apps: discover users' perception of health and fitness apps with the UTAUT2 model. Telemed J E Health. Sep 2015;21(9):735-741. [ CrossRef ] [ Medline ]
• O'Connor Y, O'Reilly P, O'Donoghue J. M-health infusion by healthcare practitioners in the national health services (NHS). Health Policy Technol. Mar 2013;2(1):26-35. [ CrossRef ]
• Milani RV, Lavie CJ, Bober RM, Milani AR, Ventura HO. Improving hypertension control and patient engagement using digital tools. Am J Med. Jan 2017;130(1):14-20. [ CrossRef ] [ Medline ]
• Williams B, Mancia G, Spiering W, Agabiti Rosei E, Azizi M, Burnier M, et al. ESC Scientific Document Group. 2018 ESC/ESH guidelines for the management of arterial hypertension: the task force for the management of arterial hypertension of the European Society of Cardiology (ESC) and the European Society of Hypertension (ESH). Eur Heart J. Sep 01, 2018;39(33):3021-3104. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Quinn CC, Butler EC, Swasey KK, Shardell MD, Terrin MD, Barr EA, et al. Mobile diabetes intervention study of patient engagement and impact on blood glucose: mixed methods analysis. JMIR Mhealth Uhealth. Feb 02, 2018;6(2):e31. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sepah SC, Jiang L, Ellis RJ, McDermott K, Peters AL. Engagement and outcomes in a digital diabetes prevention program: 3-year update. BMJ Open Diabetes Res Care. Sep 07, 2017;5(1):e000422. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Carroll JK, Moorhead A, Bond R, LeBlanc WG, Petrella RJ, Fiscella K. Who uses mobile phone health apps and does use matter? a secondary data analytics approach. J Med Internet Res. Apr 19, 2017;19(4):e125. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Demark-Wahnefried W, Schmitz KH, Alfano CM, Bail JR, Goodwin PJ, Thomson CA, et al. Weight management and physical activity throughout the cancer care continuum. CA Cancer J Clin. Jan 2018;68(1):64-89. [ FREE Full text ] [ CrossRef ] [ Medline ]

Abbreviations

Edited by T Leung, T de Azevedo Cardoso; submitted 05.02.23; peer-reviewed by B Chaudhry, M Peeples, A DeVito Dabbs; comments to author 12.09.23; revised version received 25.10.23; accepted 29.01.24; published 28.03.24.

©Yiyang Sheng, Raymond Bond, Rajesh Jaiswal, John Dinsmore, Julie Doyle. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 28.03.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

An official website of the United States government

Here’s how you know

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( Lock A locked padlock ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

https://www.nist.gov/el/intelligent-systems-division-73500/mobile-robotics-systems-research-and-standard-test-methods

Intelligent Systems Division

Mobile robotics systems research and standard test methods.

NIST has developed a mobile manipulator testbed to evaluate the dynamic localization of the mobile manipulator relative to a target with continuous motion (i.e., the manipulator and mobile robot are in simultaneous motion while physically connected) and non-continuous motion, including workpiece perturbations. The testbed enables the research and development of measurement and analysis methodologies. 

Mobile manipulators, comprised of advance mobile robots with onboard manipulation capabilities, have the potential to enable agile manufacturing. Mobile manipulator performance metrics and measurement methods allows  manufacturers and end users to understand the capabilities and advancements need to improve the adaptive capabilities in dynamic, unstructured manufacturing environments and when operating on large-scale components with curved, complex profiles.  

 NIST is leading the way of development of standard test procedures to support the innovation of mobile manipulators for agile manufacturing thru ASTM  Committee F45 on Robotics, Automation, and Autonomous Systems   (Subcommittee F45.05 on Grasping and Manipulation )

 This work informs development of standard test procedures to support the innovation of mobile manipulators for agile manufacturing.

NIST is leading the way for developing performance metrics and measurement methods that are needed by industry to characterize the capabilities and performance of Unmanned Ground Vehicles (A-UGVs), also known as Autonomous Mobile Robots (AMR).

The development of standards thru ASTM Committee F45 on Robotics, Automation, and Autonomous Systems  for environmental condi

tions and effects ( F45.01 ), docking and navigation F45.02 ) , object detection and protection( F45.03 ), and test and record (?)

NIST will to develop measurement methods and technologies for A-UGVs reflecting user requirements and operating environment, including forced de-localization , static and dynamic loading, load structures, homogeneous/heterogenous multi A-UGV operations, and collaborative applications with other robotic systems.

NIST has developed information models, autonomy level guidance, measurement methods, and standards to describe the operating environments of A-UGVs and their performances. The information models provide A-UGV users about what environmental factors need to be considered and what  A-UGV capabilities are required to operate against those factors. The autonomy level guidance describes the various services A-UGVs can provide depending on their capability. Measurement methods enable users to understand the A-UGVs’ performance and their sensitivity to the given environmental factors and operational parameters. Based on the information model, measurement method, and various tests conducted under industry-like environments.