quantitative research systematic review

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Supplements
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 4, Issue Suppl 1
• Synthesising quantitative evidence in systematic reviews of complex health interventions
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Julian P T Higgins 1 ,
• José A López-López 1 ,
• Betsy J Becker 2 ,
• Sarah R Davies 1 ,
• Sarah Dawson 1 ,
• Jeremy M Grimshaw 3 , 4 ,
• Luke A McGuinness 1 ,
• Theresa H M Moore 1 , 5 ,
• Eva A Rehfuess 6 ,
• James Thomas 7 ,
• Deborah M Caldwell 1
• 1 Population Health Sciences , Bristol Medical School, University of Bristol , Bristol , UK
• 2 Department of Educational Psychology and Learning Systems, College of Education , Florida State University , Tallahassee , Florida , USA
• 3 Clinical Epidemiology Program , Ottawa Hospital Research Institute, The Ottawa Hospital , Ottawa , Ontario , Canada
• 4 Department of Medicine , University of Ottawa , Ottawa , Ontario , Canada
• 5 NIHR Collaboration for Leadership in Applied Health Care (CLAHRC) West , University Hospitals Bristol NHS Foundation Trust , Bristol , UK
• 6 Institute for Medical Information Processing , Biometry and Epidemiology, Pettenkofer School of Public Health, LMU Munich , Munich , Germany
• 7 EPPI-Centre, Department of Social Science , University College London , London , UK
• Correspondence to Professor Julian P T Higgins; julian.higgins{at}bristol.ac.uk

Public health and health service interventions are typically complex: they are multifaceted, with impacts at multiple levels and on multiple stakeholders. Systematic reviews evaluating the effects of complex health interventions can be challenging to conduct. This paper is part of a special series of papers considering these challenges particularly in the context of WHO guideline development. We outline established and innovative methods for synthesising quantitative evidence within a systematic review of a complex intervention, including considerations of the complexity of the system into which the intervention is introduced. We describe methods in three broad areas: non-quantitative approaches, including tabulation, narrative and graphical approaches; standard meta-analysis methods, including meta-regression to investigate study-level moderators of effect; and advanced synthesis methods, in which models allow exploration of intervention components, investigation of both moderators and mediators, examination of mechanisms, and exploration of complexities of the system. We offer guidance on the choice of approach that might be taken by people collating evidence in support of guideline development, and emphasise that the appropriate methods will depend on the purpose of the synthesis, the similarity of the studies included in the review, the level of detail available from the studies, the nature of the results reported in the studies, the expertise of the synthesis team and the resources available.

• meta-analysis
• complex interventions
• systematic reviews
• guideline development

Data availability statement

No additional data are available.

This is an open access article distributed under the terms of the Creative Commons Attribution IGO License ( CC BY NC 3.0 IGO ), which permits use, distribution,and reproduction in any medium, provided the original work is properly cited. In any reproduction of this article there should not be any suggestion that WHO or this article endorse any specific organization or products. The use of the WHO logo is not permitted. This notice should be preserved along with the article’s original URL.Disclaimer: The author is a staff member of the World Health Organization. The author alone is responsible for the views expressed in this publication and they do not necessarily represent the views, decisions or policies of the World Health Organization.

https://doi.org/10.1136/bmjgh-2018-000858

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Summary box

Quantitative syntheses of studies on the effects of complex health interventions face high diversity across studies and limitations in the data available.

Statistical and non-statistical approaches are available for tackling intervention complexity in a synthesis of quantitative data in the context of a systematic review.

Appropriate methods will depend on the purpose of the synthesis, the number and similarity of studies included in the review, the level of detail available from the studies, the nature of the results reported in the studies, the expertise of the synthesis team and the resources available.

We offer considerations for selecting methods for synthesis of quantitative data to address important types of questions about the effects of complex interventions.

Public health and health service interventions are typically complex. They are usually multifaceted, with impacts at multiple levels and on multiple stakeholders. Also, the systems within which they are implemented may change and adapt to enhance or dampen their impact. 1 Quantitative syntheses ('meta-analyses’) of studies of complex interventions seek to integrate quantitative findings across multiple studies to achieve a coherent message greater than the sum of their parts. Interest is growing on how the standard systematic review and meta-analysis toolkit can be enhanced to address complexity of interventions and their impact. 2 A recent report from the Agency for Healthcare Research and Quality and a series of papers in the Journal of Clinical Epidemiology provide useful background on some of the challenges. 3–6

This paper is part of a series to explore the implications of complexity for systematic reviews and guideline development, commissioned by WHO. 7 Clearly, and as covered by other papers in this series, guideline development encompasses the consideration of many different aspects, 8 such as intervention effectiveness, economic considerations, acceptability 9 or certainty of evidence, 10 and requires the integration of different types of quantitative as well as qualitative evidence. 11 12 This paper is specifically concerned with methods available for the synthesis of quantitative results in the context of a systematic review on the effects of a complex intervention. We aim to point those collating evidence in support of guideline development to methodological approaches that will help them integrate the quantitative evidence they identify. A summary of how these methods link to many of the types of complexity encountered is provided in table 1 , based on the examples provided in a table from an earlier paper in the series. 1 An annotated list of the methods we cover is provided in table 2 .

• View inline

Quantitative synthesis possibilities to address aspects of complexity

Quantitative graphical and synthesis approaches mentioned in the paper, with their main strengths and weaknesses in the context of complex interventions

We begin by reiterating the importance of starting with meaningful research questions and an awareness of the purpose of the synthesis and any relevant background knowledge. An important issue in systematic reviews of complex interventions is that data available for synthesis are often extremely limited, due to small numbers of relevant studies and limitations in how these studies are conducted and their results are reported. Furthermore, it is uncommon for two studies to evaluate exactly the same intervention, in part because of the interventions’ inherent complexity. Thus, each study may be designed to provide information on a unique context or a novel intervention approach. Outcomes may be measured in different ways and at different time points. We therefore discuss possible approaches when data are highly limited or highly heterogeneous, including the use of graphical approaches to present very basic summary results. We then discuss statistical approaches for combining results and for understanding the implications of various kinds of complexity.

In several places we draw on an example of a review undertaken to inform a recent WHO guideline on protecting, promoting and supporting breast feeding. 13 The review seeks to determine the effects of interventions to promote breast feeding delivered in five types of settings (health services, home, community, workplace, policy context or a combination of settings). 8 The included interventions were predominantly multicomponent, and were implemented in complex systems across multiple contexts. The review included 195 studies, including many from low-income and middle-income countries, and concluded that interventions should be delivered in a combination of settings to achieve high breastfeeding rates.

The importance of the research question

The starting point in any synthesis of quantitative evidence is a clear purpose. The input of stakeholders is critical to ensure that questions are framed appropriately, addressing issues important to those commissioning, delivering and affected by the intervention. Detailed discussion of the development of research questions is provided in an earlier paper in the series, 1 and a subsequent paper explains the importance of taking context into account. 9 The first of these papers describes two possible perspectives. A complex interventions perspective emphasises the complexities involved in conceptualising, specifying and implementing the intervention per se, including the array of possibly interacting components and the behaviours required to implement it. A complex systems perspective emphasises the complexity of the systems into which the intervention is introduced, including possible interactions between the intervention and the system, interactions between individuals within the system and how the whole system responds to the intervention.

The simplest purpose of a systematic review is to determine whether a particular type of complex intervention (or class of interventions) is effective compared with a ‘usual practice’ alternative. The familiar PICO framework is helpful for framing the review: 14 in the PICO framework, a broad research question about effectiveness is uniquely specified by describing the participants (‘P’, including the setting and prevailing conditions) to which the intervention is to be applied; the intervention (‘I’) and comparator (‘C’) of interest, and the outcomes (‘O’, including their time course) that might be impacted by the intervention. In the breastfeeding review, the primary synthesis approach was to combine all available studies, irrespective of setting, and perform separate meta-analyses for different outcomes. 15

More useful than a review that asks ‘does a complex intervention work?’ is one that determines the situations in which a complex intervention has a larger or smaller effect. Indeed, research questions targeted by syntheses in the presence of complexity often dissect one or more of the PICO elements to explore how intervention effects vary both within and across studies (ie, treating the PICO elements as ‘moderators’). For instance, analyses may explore variation across participants, settings and prevailing conditions (including context); or across interventions (including different intervention components that may be present or absent in different studies); or across outcomes (including different outcome measures, at different levels of the system and at different time points) on which effects of the intervention occur. In addition, there may be interest in how aspects of the underlying system or the intervention itself mediate the effects, or in the role of intermediate outcomes on the pathway from intervention to impact. 16 In the breastfeeding review, interest moved from the overall effects across interventions to investigations of how effects varied by such factors as intervention delivery setting, high-income versus low-income country, and urban versus rural setting. 15

The role of logic models to inform a synthesis

An earlier paper describes the benefits of using system-based logic models to characterise a priori theories about how the system operates. 1 These provide a useful starting point for most syntheses since they encourage consideration of all aspects of complexity in relation to the intervention or the system (or both). They can help identify important mediators and moderators, and inform decisions about what aspects of the intervention and system need to be addressed in the synthesis. As an example, a protocol for a review of the health effects of environmental interventions to reduce the consumption of sugar-sweetened beverages included a system-based logic model, detailing how the characteristics of the beverages, and the physiological characteristics and psychological characteristics of individuals, are thought to impact on outcomes such as weight gain and cardiovascular disease. 17 The logic model informs the selection of outcomes and the general plans for synthesis of the findings of included studies. However, system-based models do not usually include details of how implementation of an intervention into the system is likely to affect subsequent outcomes. They therefore have a limited role in informing syntheses that seek to explain mechanisms of action.

A quantitative synthesis may draw on a specific proposed framework for how an intervention might work; these are sometimes referred to as process-orientated logic models, and may be strongly driven by qualitative research evidence. 12 They represent causal processes, describing what components or aspects of an intervention are thought to impact on what behaviours and actions, and what the further consequences of these impacts are likely to be. 18 They may encompass mediators of effect and moderators of effect. A synthesis may simply adopt the proposed causal model at face value and attempt to quantify the relationships described therein. Where more than one possible causal model is available, a synthesis may explore which of the models is better supported by the data, for example, by examining the evidence for specific links within the model or by identifying a statistical model that corresponds to the overall causal model. 18 19

A systematic review on community-level interventions for improving access to food in low-income and middle-income countries was based on a logic model that depicts how interventions might lead to improved health status. 20 The model includes direct effects, such as increased financial resources of individuals and decreased food prices; intermediate effects, such as increased quantity of food available and increase in intake; and main outcomes of interest, such as nutritional status and health indicators. The planned statistical synthesis, however, was to tackle these one at a time.

Considering the types of studies available

Studies of the effects of complex interventions may be randomised or non-randomised, and often involve clustering of participants within social or organisational units. Randomised trials, if sufficiently large, provide the most convincing evidence about the effects of interventions because randomisation should result in intervention and comparator groups with similar distributions of both observed and unobserved baseline characteristics. However, randomised trials of complex interventions may be difficult or impossible to undertake, or may be performed only in specific contexts, yielding results that are not generalisable. Non-randomised study designs include so-called ‘quasi-experiments’ and may be longitudinal studies, including interrupted time series and before-after studies, with or without a control group. Non-randomised studies are at greater risk of bias, sometimes substantially so, although may be undertaken in contexts that are more relevant to decision making. Analyses of non-randomised studies often use statistical controls for confounders to account for differences between intervention groups, and challenges are introduced when different sets of confounders are used in different studies. 21 22

Randomised trials and non-randomised studies might both be included in a review, and analysts may have to decide whether to combine these in one synthesis, and whether to combine results from different types of non-randomised studies in a single analysis. Studies may differ in two ways: by answering different questions, or by answering similar questions with different risks of bias. The research questions must be sufficiently similar and the studies sufficiently free of bias for a synthesis to be meaningful. In the breastfeeding review, randomised, quasi-experimental and observational studies were combined; no evidence suggested that the effects differed across designs. 15 In practice, many methodologists generally recommend against combining randomised with non-randomised studies. 23

Preparing for a quantitative synthesis

Before undertaking a quantitative synthesis of complex interventions, it can be helpful to begin the synthesis non-quantitatively, looking at patterns and characteristics of the data identified. Systematic tabulation of information is recommended, and this might be informed by a prespecified logic model. The most established framework for non-quantitative synthesis is that proposed by Popay et al . 24 The Cochrane Consumers and Communication group succinctly summarise the process as an 'investigation of the similarities and the differences between the findings of different studies, as well as exploration of patterns in the data’. 25 Another useful framework was described by Petticrew and Roberts. 26 They identify three stages in the initial narrative synthesis: (1) Organisation of studies into logical categories, the structure of which will depend on the purpose of the synthesis, possibly relating to study design, outcome or intervention types. (2) Within-study analysis, involving the description of findings within each study. (3) Cross-study synthesis, in which variations in study characteristics and potential biases are integrated and the range of effects described. Aspects of this process are likely to be implemented in any systematic review, even when a detailed quantitative synthesis is undertaken.

In some circumstances the available data are too diverse, too non-quantitative or too sparse for a quantitative synthesis to be meaningful even if it is possible. The best that can be achieved in many reviews of complex interventions is a non-quantitative synthesis following the guidance given in the above frameworks.

Options when effect size estimates cannot be obtained or studies are too diverse to combine

Graphical approaches.

Graphical displays can be very valuable to illustrate patterns in results of studies. 27 We illustrate some options in figure 1 . Forest plots are the standard illustration of the results of multiple studies (see figure 1 , panel A), but require a similar effect size estimate from each study. For studies of complex interventions, the diversity of approaches to the intervention, the context, 1 evaluation approaches and reporting differences can lead to considerable variation across studies in what results are available. Some novel graphical approaches have been proposed for such situations. A recent development is the albatross plot, which plots p values against sample sizes, with approximate effect-size contours superimposed (see figure 1 , panel B). 28 The contours are computed from the p values and sample sizes, based on an assumption about the type of analysis that would have given rise to the p values. Although these plots are designed for situations when effect size estimates are not available, the contours can be used to infer approximate effect sizes from studies that are analysed and reported in highly diverse ways. Such an advantage may prove to be a disadvantage, however, if the contours are overinterpreted.

• Download figure
• Open in new tab
• Download powerpoint

Example graphical displays of data from a review of interventions to promote breast feeding, for the outcome of continued breast feeding up to 23 months. 15 Panel A: Forest plot for relative risk (RR) estimates from each study. Panel B: Albatross plot of p value against sample size (effect contours drawn for risk ratios assuming a baseline risk of 0.15; sample sizes and baseline risks extracted from the original papers by the current authors); Panel C: Harvest plot (heights reflect design: randomised trials (tall), quasi-experimental studies (medium), observational studies (short); bar shading reflects follow-up: longest follow-up (black) to shortest follow-up (light grey) or no information (white)). Panel D: Bubble plot (bubble sizes and colours reflect design: randomised trials (large, green), quasi-experimental studies (medium, red), observational studies (small, blue); precision defined as inverse of the SE of each effect estimate (derived from the CIs); categories are: “Potential Harm”: RR <0.8; “No Effect”: RRs between 0.8 and 1.25; “Potential Benefit”: RR >1.25 and CI includes RR=1; “Benefit”: RR >1.25 and CI excludes RR=1).

Harvest plots have been proposed by Ogilvie et al as a graphical extension of a vote counting approach to synthesis (see figure 1 , panel C). 29 However, approaches based on vote counting of statistically significant results have been criticised on the basis of their poor statistical properties, and because statistical significance is an outdated and unhelpful notion. 30 The harvest plot is a matrix of small illustrations, with different outcome domains defining rows and different qualitative conclusions (negative effect, no effect, positive effect) defining columns. Each study is represented by a bar that is positioned according to its measured outcome and qualitative conclusion. Bar heights and shadings can depict features of the study, such as objectivity of the outcome measure, suitability of the study design and study quality. 29 31 A similar idea to the harvest plot is the effect direction plot proposed by Thomson and Thomas. 32

A device to plot the findings from a large and complex collection of evidence is a bubble plot (see figure 1 , panel D). A bubble plot illustrates the direction of each finding (or whether the finding was unclear) on a horizontal scale, using a vertical scale to indicate the volume of evidence, and with bubble sizes to indicate some measure of credibility of each finding. Such an approach can also depict findings of collections of studies rather than individual studies, and was used successfully, for example, to summarise findings from a review of systematic reviews of the effects of acupuncture on various indications for pain. 33

Statistical methods not based on effect size estimates

We have mentioned that a frequent problem is that standard meta-analysis methods cannot be used because data are not available in a similar format from every study. In general, the core principles of meta-analysis can be applied even in this situation, as is highlighted in the Cochrane Handbook , by addressing the questions: ‘What is the direction of effect?’; 'What is the size of effect?’; ‘Is the effect consistent across studies?’; and 'What is the strength of evidence for the effect?’. 34

Alternatives to the estimation of effect sizes could be used more often than they are in practice, allowing some basic statistical inferences despite diversely reported results. The most fundamental analysis is to test the overall null hypothesis of no effect in any of the studies. Such a test can be undertaken using only minimally reported information from each study. At its simplest, a binomial test can be performed using only the direction of effect observed in each study, irrespective of its CI or statistical significance. 35 Where exact p values are available as well as the direction of effect, a more powerful test can be performed by combining these using, for example, Fisher’s combination of p values. 36 It is important that these p values are computed appropriately, however, accounting for clustering or matching of participants within the studies. Rejecting the null model based on such tests provides no information about the magnitude of the effect, providing information only on whether at least one study shows an effect is present, and if so, its direction. 37

Standard synthesis methods

Meta-analysis for overall effect.

Probably the most familiar approach to meta-analysis is that of estimating a single summary effect across similar studies. This simple approach lends itself to the use of forest plots to display the results of individual studies as well as syntheses, as illustrated for the breastfeeding studies in figure 1 (panel A). This analysis addresses the broad question of whether evidence from a collection of studies supports an impact of the complex intervention of interest, and requires that every study makes a comparison of a relevant intervention against a similar alternative. In the context of complex interventions, this is described by Caldwell and Welton as the ‘lumping’ approach, 38 and by Guise et al as the ‘holistic’ approach. 5 6 One key limitation of the simple approach is that it requires similar types of data from each study. A second limitation is that the meta-analysis result may have limited relevance when the studies are diverse in their characteristics. Fixed-effect models, for instance, are unlikely to be appropriate for complex interventions because they ignore between-studies variability in underlying effect sizes. Results based on random-effects models will need to be interpreted by acknowledging the spread of effects across studies, for example, using prediction intervals. 39

A common problem when undertaking a simple meta-analysis is that individual studies may report many effect sizes that are correlated with each other, for example, if multiple outcomes are measured, or the same outcome variable is measured at several time points. Numerous approaches are available for dealing with such multiplicity, including multivariate meta-analysis, multilevel modelling, and strategies for selecting effect sizes. 40 A very simple strategy that has been used in systematic reviews of complex interventions is to take the median effect size within each study, and to summarise these using the median of these effect sizes across studies. 41

Exploring heterogeneity

Diversity in the types of participants (and contexts), interventions and outcomes are key to understanding sources of complexity. 9 Many of these important sources of heterogeneity are most usefully examined—to the extent that they can reliably be understood—using standard approaches for understanding variability across studies, such as subgroup analyses and meta-regression.

A simple strategy to explore heterogeneity is to estimate the overall effect separately for different levels of a factor using subgroup analyses (referring to subgrouping studies rather than participants). 42 As an example, McFadden et al conducted a systematic review and meta-analysis of 73 studies of support for healthy breastfeeding mothers with healthy term babies. 43 They calculated separate average effects for interventions delivered by a health professional, a lay supporter or with mixed support, and found that the effect on cessation of exclusive breast feeding at up to 6 months was greater for lay support compared with professionals or mixed support (p=0.02). Guise et al provide several ways of grouping studies according to their interventions, for example, grouping studies by key components, by function or by theory. 5 6

Meta-regression provides a flexible generalisation to subgroup analyses, whereby study-level covariates are included in a regression model using effect size estimates as the dependent variable. 44 45 Both continuous and categorical covariates can be included in such models; with a single categorical covariate, the approach is essentially equivalent to subgroup analyses. Meta-regression with continuous covariates in theory allows the extrapolation of relationships to contexts that were not examined in any of the studies, but this should generally be avoided. For example, if the effect of an interventional approach appears to increase as the size of the group to which it is applied decreases, this does not mean that it will work even better when applied to a single individual. More generally, the mathematical form of the relationship modelled in a meta-regression requires careful selection. Most often a linear relationship is assumed, but a linear relationship does not permit step changes such as might occur if an interventional approach requires a particular level of some feature of the underlying system before it has an effect.

Several texts provide guidance for using subgroup analysis and meta-regression in a general context 45 46 and for complex interventions. 3 4 47 In principle, many aspects of complexity in interventions can be addressed using these strategies, to create an understanding of the ‘response surface’. 48–50 However, in practice, the number of studies is often too small for reliable conclusions to be drawn. In general, subgroup analysis and meta-regression are fraught with dangers associated with having few studies, many sources of variation across study features and confounding of these features with each other as well as with other, often unobserved, variables. It is therefore important to prespecify a small number of plausible sources of diversity so as to reduce the danger of reaching spurious conclusions based on study characteristics that correlate with the effects of the interventions but are not the cause of the variation. The ability of statistical analyses to identify true sources of heterogeneity will depend on the number of studies, the sizes of the studies and the true differences between effects in studies with different characteristics.

Synthesis methods for understanding components of the intervention

When interventions comprise distinct components, it is attractive to separate out the individual effects of these components. 51 Meta-regression can be used for this, using covariates to code the presence of particular features in each intervention implementation. As an example, Blakemore et al analysed 39 intervention comparisons from 33 independent studies aiming to reduce urgent healthcare use in adults with asthma. 52 Effect size estimates were coded according to components used in the interventions, and the authors found that multicomponent interventions including skills training, education and relapse prevention appeared particularly effective. In another example, of interventions to support family caregivers of people with Alzheimer’s disease, 53 the authors used methods for decomposing complex interventions proposed by Czaja et al , 54 and created covariates that reduced the complexity of the interventions to a small number of features about the intensity of the interventions. More sophisticated models for examining components have been described by Welton et al , 55 Ivers et al 56 and Madan et al . 57

A component-level approach may be useful when there is a need to disentangle the ‘active ingredients’ of an intervention, for example, when adapting an existing intervention for a new setting. However, components-based approaches require assumptions, such as whether individual components are additive or interact with each other. Furthermore, the effects of components can be difficult to estimate if they are used only in particular contexts or populations, or are strongly correlated with use of other components. An alternative approach is to treat each combination of components as a separate intervention. These separate interventions might then be compared in a single analysis using network meta-analysis. A network meta-analysis combines results from studies comparing two or more of a larger set of interventions, using indirect comparisons via common comparators to rank-order all interventions. 47 58 59 As an example, Achana et al examined the effectiveness of safety interventions on the uptake of three poisoning prevention practices in households with children. Each singular combination of intervention components was defined as a separate intervention in the network. 60 Network meta-analysis may also be useful when there is a need to compare multiple interventions to answer an ‘in principle’ question of which intervention is most effective. Consideration of the main goals of the synthesis will help those aiming to prepare guidelines to decide which of these approaches is most appropriate to their needs.

A case study exploring components is provided in box 1 , and an illustration is provided in figure 2 . The component-based analysis approach can be likened to a factorial trial, in that it attempts to separate out the effects of individual components of the complex interventions, and the network meta-analysis approach can be likened to a multiarm trial approach, where each complex intervention in the set of studies is a different arm in the trial. 47 Deciding between the two approaches can leave the analyst caught between the need to ‘split’ components to reflect complexity (and minimise heterogeneity) and ‘lump’ to make an analysis feasible. Both approaches can be used to examine other features of interventions, including interventions designed for delivery at different levels. For example, a review of the effects of interventions for children exposed to domestic violence and abuse included studies of interventions targeted at children alone, parents alone, children and parents together, and parents and children separately. 61 A network meta-analysis approach was taken to the synthesis, with the people targeted by the intervention used as a distinguishing feature of the interventions included in the network.

Example of understanding components of psychosocial interventions for coronary heart disease

Welton et al reanalysed data from a Cochrane review 89 of randomised controlled trials assessing the effects of psychological interventions on mortality and morbidity reduction for people with coronary heart disease. 55 The Cochrane review focused on the effectiveness of any psychological intervention compared with usual care, and found evidence that psychological interventions reduced non-fatal reinfarctions and depression and anxiety symptoms. The Cochrane review authors highlighted the large heterogeneity among interventions as an important limitation of their review.

Welton et al were interested in the effects of the different intervention components. They classified interventions according to which of five key components were included: educational, behavioural, cognitive, relaxation and psychosocial support ( figure 2 ). Their reanalysis examined the effect of each component in three different ways: (1) An additive model assuming no interactions between components. (2) A two-factor interaction model, allowing for interactions between pairs of components. (3) A network meta-analysis, defining each combination of components as a separate intervention, therefore allowing for full interaction between components. Results suggested that interventions with behavioural components were effective in reducing the odds of all-cause mortality and non-fatal myocardial infarction, and that interventions with behavioural and/or cognitive components were effective for reducing depressive symptoms.

Intervention components in the studies integrated by Welton et al (a sample of 18 from 56 active treatment arms). EDU, educational component; BEH, behavioural component; COG, cognitive component; REL, relaxation component; SUP, psychosocial support component.

A common limitation when implementing these quantitative methods in the context of complex interventions is that replication of the same intervention in two or more studies is rare. Qualitative comparative analysis (QCA) might overcome this problem, being designed to address the ’small N; many variables’ problem. 62 QCA involves: (1) Identifying theoretically driven thresholds for determining intervention success or failure. (2) Creating a 'truth table’, which takes the form of a matrix, cross-tabulating all possible combinations of conditions (eg, participant and intervention characteristics) against each study and its associated outcomes. (3) Using Boolean algebra to eliminate redundant conditions and to identify configurations of conditions that are necessary and/or sufficient to trigger intervention success or failure. QCA can usefully complement quantitative integration, sometimes in the context of synthesising diverse types of evidence.

Synthesis methods for understanding mechanisms of action

An alternative purpose of a synthesis is to gain insight into the mechanisms of action behind an intervention, to inform its generalisability or applicability to a particular context. Such syntheses of quantitative data may complement syntheses of qualitative data, 11 and the two forms might be integrated. 12 Logic models, or theories of action, are important to motivate investigations of mechanism. The synthesis is likely to focus on intermediate outcomes reflecting intervention processes, and on mediators of effect (factors that influence how the intervention affects an outcome measure). Two possibilities for analysis are to use these intermediate measurements as predictors of main outcomes using meta-regression methods, 63 or to use multivariate meta-analysis to model the intermediate and main outcomes simultaneously, exploiting and estimating the correlations between them. 64 65 If the synthesis suggests that hypothesised chains of outcomes hold, this lends weight to the theoretical model underlying the hypothesis.

An approach to synthesis closely identified with this category of interventions is model-driven meta-analysis, in which different sources of evidence are integrated within a causal path model akin to a directed acyclic graph. A model-driven meta-analysis is an explanatory analysis. 66 It attempts to go further than a standard meta-analysis or meta-regression to explore how and why an intervention works, for whom it works, and which aspects of the intervention (factors) are driving overall effect. Such syntheses have been described in frequentist 19 67–70 and Bayesian 71 72 frameworks and are variously known as model-driven meta-analysis, linked meta-analysis, meta-mediation analysis and meta-analysis of structural equation models. In their simplest form, standard meta-analyses estimate a summary correlation independently for each pair of variables in the model. The approach is inherently multivariate, requiring the estimation of multiple correlations (which, if obtained from a single study, are also not independent). 73–75 Each study is likely to contribute fragments of the correlation matrix. A summary correlation matrix, combined either by fixed-effects or random-effects methods, then serves as the input for subsequent analysis via a standardised regression or structural equation model.

An example is provided in box 2 . The model in figure 3 postulates that the effect of ‘Dietary adherence’ on ‘Diabetes complications’ is not direct but is mediated by ‘Metabolic control’. 76 The potential for model-driven meta-analysis to incorporate such indirect effects also allows for mediating effects to be explicitly tested and in so doing allows the meta-analyst to identify and explore the mechanisms underpinning a complex intervention. 77

Theoretical diabetes care model (adapted from Brown et al 68 ).

Example of a model-driven meta-analysis for type 2 diabetes

Brown et al present a model-driven meta-analysis of correlational research on psychological and motivational predictors of diabetes outcomes, with medication and dietary adherence factors as mediators. 76 In a linked methodological paper, they present the a priori theoretical model on which their analysis is based. 68 The model is simplified in figure 3 , and summarised for the dietary adherence pathway only. The aim of their full analysis was to determine the predictive relationships among psychological factors and motivational factors on metabolic control and body mass index (BMI), and the role of behavioural factors as possible mediators of the associations among the psychological and motivational factors and metabolic control and BMI outcomes.

The analysis is based on a comprehensive systematic review. Due to the number of variables in their full model, 775 individual correlational or predictive studies reported across 739 research papers met eligibility criteria. Correlations between each pair of variables in the model were summarised using an overall average correlation, and homogeneity assessed. Multivariate analyses were used to estimate a combined correlation matrix. These results were used, in turn, to estimate path coefficients for the predictive model and their standard errors. For the simplified model illustrated here, the results suggested that coping and self-efficacy were strongly related to dietary adherence, which was strongly related to improved glycaemic control and, in turn, a reduction in diabetic complications.

Synthesis approaches for understanding complexities of the system

Syntheses may seek to address complexities of the system to understand either the impact of the system on the effects of the intervention or the effects of the intervention on the system. This may start by modelling the salient features of the system’s dynamics, rather than focusing on interventions. Subgroup analysis and meta-regression are useful approaches for investigating the extent to which an intervention’s effects depend on baseline features of the system, including aspects of the context. Sophisticated meta-regression models might investigate multiple baseline features, using similar approaches to the component-based meta-analyses described earlier. Specifically, aspects of context or population characteristics can be regarded as ‘components’ of the system into which the intervention is introduced, and similar statistical modelling strategies used to isolate effects of individual factors, or interactions between them.

When interventions act at multiple levels, it may be important to understand the effects at these different levels. Outcomes may be measured at different levels (eg, at patient, clinician and clinical practice levels) and analysed separately. Qualitative research plays a particularly important role in identifying the outcomes that should be assessed through quantitative synthesis. 12 Care is needed to ensure that the unit of analysis issues are addressed. For example, if clinics are the unit of randomisation, then outcomes measured at the clinic level can be analysed using standard methods, whereas outcomes measured at the level of the patient within the clinic would need to account for clustering. In fact, multiple dependencies may arise in such data, when patients receive care in small groups. Detailed investigations of effect at different levels, including interactions between the levels, would lend themselves to multilevel (hierarchical) models for synthesis. Unfortunately, individual participant data at all levels of the hierarchy are needed for such analyses.

Model-based approaches also offer possibilities for addressing complex systems; these include economic models, mathematical models and systems science methods generally. 78–80 Broadly speaking, these provide mathematical representations of logic models, and analyses may involve incorporation of empirical data (eg, from systematic reviews), computer simulation, direct computation or a mixture of these. Multiparameter evidence synthesis methods might be used. 81 82 Approaches include models to represent systems (eg, systems dynamics models) and approaches that simulate individuals within the system (eg, agent-based models). 79 Models can be particularly useful when empirical evidence does not address all important considerations, such as ‘real-world’ contexts, long-term effects, non-linear effects and complexities such as feedback loops and threshold effects. An example of a model-based approach to synthesis is provided in box 3 . The challenge when adopting these approaches is often in the identification of system components, and accurately estimating causes and effects (and uncertainties). There are few examples of the use of these analytical tools in systematic reviews, but they may be useful when the focus of analysis is on understanding the causes of complexity in a given system rather than on the impact of an intervention.

Example of a mathematical modelling approach for soft drinks industry levy

Briggs et al examined the potential impact of a soft drinks levy in the UK, considering possible different types of response to the levy by industry. 90 Various scenarios were posited, with effects on health outcomes informed by empirical data from randomised trials and cohort studies of association between sugar intake and body weight, diabetes and dental caries. Figure 4 provides a simple characterisation of how the empirical data were fed into the model. Inputs into the model included levels of consumption of various types of drinks (by age and sex), volume of drinks sales, and baseline levels of obesity, diabetes and dental caries (by age and sex). The authors concluded that health gains would be greatest if industry reacted by reformulating their products to include less sugar.

Simplified version of the conceptual model used by Briggs et al ( a dapted from Briggs et al 90 ).

Considerations of bias and relevance

It is always important to consider the extent to which (1) The findings from each study have internal validity, particularly for non-randomised studies which are typically at higher risk of bias. (2) Studies may have been conducted but not reported because of unexciting findings. (3) Each study is applicable to the purposes of the review, that is, has external validity (or ‘directness’), in the language of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group. 83 At minimum, internal and external validity should be examined and reported, and the risk of publication bias assessed, and these can be achieved through the GRADE framework. 10 With sufficient studies, information collected might be used in meta-regression analyses to evaluate empirically whether studies with and without specific sources of bias or indirectness differ in their results.

It may be desirable to learn about a specific setting, intervention type or outcome measure more directly than others. For example, to inform a decision for a low-income setting, emphasis should be placed on results of studies performed in low-income countries. One option is to restrict the synthesis to these studies. An alternative is to model the dependence of an intervention’s effect on some feature(s) related to the income setting, and extract predictions from the model that are most relevant to the setting of interest. This latter approach makes fuller use of available data, but relies on stronger assumptions.

Often, however, the accumulated studies are too few or too disparate to draw conclusions about the impact of bias or relevance. On rare occasions, syntheses might implement formal adjustments of individual study results for likely biases. Such adjustments may be made by imposing prior distributions to depict the magnitude and direction of any biases believed to exist. 84 85 The choice of a prior distribution may be informed by formal assessments of risk of bias, by expert judgement, or possibly by empirical data from meta-epidemiological studies of biases in randomised and/or non-randomised studies. 86 For example, Wolf et al implemented a prior distribution based on findings of a meta-epidemiological study 87 to adjust for lack of blinding in studies of interventions to improve quality of point-of-use water sources in low-income and middle-income settings. 88 Unfortunately, empirical evidence of bias is mostly limited to clinical trials, is weak for trials of public health and social care interventions, and is largely non-existent for non-randomised studies.

Our review of quantitative synthesis methods for evaluating the effects of complex interventions has outlined many possible approaches that might be considered by those collating evidence in support of guideline development. We have described three broad categories: (1) Non-quantitative methods, including tabulation, narrative and graphical approaches. (2) Standard meta-analysis methods, including meta-regression to investigate study-level moderators of effect. (3) More advanced synthesis methods, in which models allow exploration of intervention components, investigation of both moderators and mediators, examination of mechanisms, and exploration of complexities of the system.

The choice among these approaches will depend on the purpose of the synthesis, the similarity of the studies included in the review, the level of detail available from the studies, the nature of the results reported in the studies, the expertise of the synthesis team, and the resources available. Clearly the advanced methods require more expertise and resources than the simpler methods. Furthermore, they require a greater level of detail and typically a sizeable evidence base. We therefore expect them to be used seldomly; our aim here is largely to articulate what they can achieve so that they can be adopted when they are appropriate. Notably, the choice among these approaches will also depend on the extent to which guideline developers and users at global, national or local levels understand and are willing to base their decisions on different methods. Where possible, it will thus be important to involve concerned stakeholders during the early stages of the systematic review process to ensure the relevance of its findings.

Complexity is common in the evaluation of public health interventions at individual, organisational or community levels. To help systematic review and guideline development teams decide how to address this complexity in syntheses of quantitative evidence, we summarise considerations and methods in tables 1 and 2 . We close with the important remark that quantitative synthesis is not always a desirable feature of a systematic review. Whereas some sophisticated methods are available to deal with a variety of complex problems, on many occasions—perhaps even the majority in practice—the studies may be too different from each other, too weak in design or have data too sparse, for statistical methods to provide insight beyond a commentary on what evidence has been identified.

Acknowledgments

The authors thank the following for helpful comments on earlier drafts of the paper: Philippa Easterbrook, Matthias Egger, Anayda Portela, Susan L Norris, Mark Petticrew.

• Petticrew M ,
• Thomas J , et al
• Anderson L ,
• Elder R , et al
• Rehfuess E ,
• Noyes J , et al
• Viswanathan M , et al
• World Health Organization
• Rehfuess EA ,
• Stratil JM ,
• Scheel IB , et al
• Flemming K , et al
• Montgomery P ,
• Movsisyan A ,
• Grant SP , et al
• Flemming K ,
• Garside R , et al
• Moore G , et al
• Richardson WS ,
• Wilson MC ,
• Nishikawa J , et al
• Chowdhury R ,
• Sankar MJ , et al
• Collins D ,
• Johnson K ,
• von Philipsborn P ,
• Burns J , et al
• Schoonees A ,
• Becker BJ ,
• Duvendack M , et al
• Reeves BC ,
• Higgins JPT , et al
• Roberts H ,
• Ryan R , Cochrane Consumers and Communication Review Group
• Anzures-Cabrera J ,
• Harrison S ,
• Martin RM , et al
• Ogilvie D ,
• Petticrew M , et al
• Sterne JA ,
• Davey Smith G
• Crowther M ,
• Avenell A ,
• MacLennan G , et al
• Thomson HJ ,
• Taylor SL ,
• Higgins JPT ,
• Bushman BJ ,
• Caldwell DM ,
• Thompson SG ,
• Spiegelhalter DJ
• López-López JA ,
• Lipsey MW , et al
• Grimshaw JM ,
• Thomas RE ,
• Borenstein M ,
• McFadden A ,
• Renfrew MJ , et al
• van Houwelingen HC ,
• Arends LR ,
• Melendez-Torres GJ ,
• Ioannidis JP ,
• Steckelberg A ,
• Richter B , et al
• Presseau J ,
• Newham JJ , et al
• Blakemore A ,
• Dickens C ,
• Anderson R , et al
• Schulz R , et al
• Lee CC , et al
• Welton NJ ,
• Adamopoulos E , et al
• Tricco AC ,
• Trikalinos TA , et al
• Aveyard P , et al
• Calderbank-Batista T
• Achana FA ,
• Cooper NJ ,
• Bujkiewicz S , et al
• Howarth E ,
• Moore THM ,
• Welton NJ , et al
• O’Mara-Eves A ,
• Thompson SG
• Jackson D ,
• Raudenbush SW ,
• García AA , et al
• Cheung MW ,
• Watson SI ,
• Stewart GB ,
• Mengersen K ,
• García AA ,
• Brown A , et al
• Liu H , et al
• Johnson L ,
• Althaus C , et al
• Stamatakis KA
• Greenwood-Lee J ,
• Nettel-Aguirre A , et al
• Caldwell D , et al
• Colbourn T ,
• Asseburg C ,
• Bojke L , et al
• Guyatt GH ,
• Oxman AD , GRADE Working Group
• Turner RM ,
• Spiegelhalter DJ ,
• Smith GCS , et al
• Carlin JB , et al
• Schulz KF , et al
• Savović J ,
• Altman DG , et al
• Prüss-Ustün A ,
• Cumming O , et al
• Bennett P ,
• West R , et al
• Briggs ADM ,
• Mytton OT ,
• Kehlbacher A , et al

Handling editor Soumyadeep Bhaumik

Contributors JPTH co-led the project, conceived the paper, led discussions and wrote the first draft. JAL-L undertook analyses, contributed to discussions and contributed to writing the manuscript. BJB drafted material on mechanisms, contributed to discussions and contributed extensively to writing the manuscript. SRD screened and categorised the results of the literature searches, collated examples and contributed to discussions. SD undertook searches to identify relevant literature and contributed to discussions. JMG contributed to discussions and commented critically on drafts. LAM undertook analyses, contributed to discussions and commented critically on drafts. THMM contributed examples, contributed to discussions and commented critically on drafts. EAR and JT contributed to discussions and commented critically on drafts. DMC co-led the project, contributed to discussions and drafted extensive parts of the paper. All authors approved the final version of the manuscript.

Funding Funding provided by the World Health Organization Department of Maternal, Newborn, Child and Adolescent Health through grants received from the United States Agency for International Development and the Norwegian Agency for Development Cooperation. JPTH was funded in part by Medical Research Council (MRC) grant MR/M025209/1, by the MRC Integrative Epidemiology Unit at the University of Bristol (MC_UU_12013/9) and by the MRC ConDuCT-II Hub (Collaboration and innovation for Difficult and Complex randomised controlled Trials In Invasive procedures – MR/K025643/1). BJB was funded in part by grant DRL-1252338 from the US National Science Foundation (NSF). JMG holds a Canada Research Chair in Health Knowledge Transfer and Uptake. LAM is funded by a National Institute for Health Research (NIHR) Systematic Review Fellowship (RM-SR-2016-07 26). THMM was funded by the NIHR Collaboration for Leadership in Applied Health Research and Care West (NIHR CLAHRC West). JT is supported by the NIHR Collaboration for Leadership in Applied Health Research and Care North Thames at Bart’s Health NHS Trust. DMC was funded in part by NIHR grant PHR 15/49/08 and by the Centre for the Development and Evaluation of Complex Interventions for Public Health Improvement (DECIPHer –MR/KO232331/1).

Disclaimer The views expressed are those of the authors and not necessarily those of the CRC program, the MRC, the NSF, the NHS, the NIHR or the UK Department of Health.

Competing interests JMG reports personal fees from the Campbell Collaboration. EAR reports being a Methods Editor with Cochrane Public Health.

Patient consent Not required.

Provenance and peer review Not commissioned; externally peer reviewed.

Read the full text or download the PDF:

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology
• Systematic Review | Definition, Examples & Guide

Systematic Review | Definition, Examples & Guide

Published on 15 June 2022 by Shaun Turney . Revised on 17 October 2022.

A systematic review is a type of review that uses repeatable methods to find, select, and synthesise all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer.

They answered the question ‘What is the effectiveness of probiotics in reducing eczema symptoms and improving quality of life in patients with eczema?’

In this context, a probiotic is a health product that contains live microorganisms and is taken by mouth. Eczema is a common skin condition that causes red, itchy skin.

Table of contents

What is a systematic review, systematic review vs meta-analysis, systematic review vs literature review, systematic review vs scoping review, when to conduct a systematic review, pros and cons of systematic reviews, step-by-step example of a systematic review, frequently asked questions about systematic reviews.

A review is an overview of the research that’s already been completed on a topic.

What makes a systematic review different from other types of reviews is that the research methods are designed to reduce research bias . The methods are repeatable , and the approach is formal and systematic:

• Formulate a research question
• Develop a protocol
• Search for all relevant studies
• Apply the selection criteria
• Extract the data
• Synthesise the data
• Write and publish a report

Although multiple sets of guidelines exist, the Cochrane Handbook for Systematic Reviews is among the most widely used. It provides detailed guidelines on how to complete each step of the systematic review process.

Systematic reviews are most commonly used in medical and public health research, but they can also be found in other disciplines.

Systematic reviews typically answer their research question by synthesising all available evidence and evaluating the quality of the evidence. Synthesising means bringing together different information to tell a single, cohesive story. The synthesis can be narrative ( qualitative ), quantitative , or both.

Prevent plagiarism, run a free check.

Systematic reviews often quantitatively synthesise the evidence using a meta-analysis . A meta-analysis is a statistical analysis, not a type of review.

A meta-analysis is a technique to synthesise results from multiple studies. It’s a statistical analysis that combines the results of two or more studies, usually to estimate an effect size .

A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarise and evaluate previous work, without using a formal, explicit method.

Although literature reviews are often less time-consuming and can be insightful or helpful, they have a higher risk of bias and are less transparent than systematic reviews.

Similar to a systematic review, a scoping review is a type of review that tries to minimise bias by using transparent and repeatable methods.

However, a scoping review isn’t a type of systematic review. The most important difference is the goal: rather than answering a specific question, a scoping review explores a topic. The researcher tries to identify the main concepts, theories, and evidence, as well as gaps in the current research.

Sometimes scoping reviews are an exploratory preparation step for a systematic review, and sometimes they are a standalone project.

A systematic review is a good choice of review if you want to answer a question about the effectiveness of an intervention , such as a medical treatment.

To conduct a systematic review, you’ll need the following:

• A precise question , usually about the effectiveness of an intervention. The question needs to be about a topic that’s previously been studied by multiple researchers. If there’s no previous research, there’s nothing to review.
• If you’re doing a systematic review on your own (e.g., for a research paper or thesis), you should take appropriate measures to ensure the validity and reliability of your research.
• Access to databases and journal archives. Often, your educational institution provides you with access.
• Time. A professional systematic review is a time-consuming process: it will take the lead author about six months of full-time work. If you’re a student, you should narrow the scope of your systematic review and stick to a tight schedule.
• Bibliographic, word-processing, spreadsheet, and statistical software . For example, you could use EndNote, Microsoft Word, Excel, and SPSS.

A systematic review has many pros .

• They minimise research b ias by considering all available evidence and evaluating each study for bias.
• Their methods are transparent , so they can be scrutinised by others.
• They’re thorough : they summarise all available evidence.
• They can be replicated and updated by others.

Systematic reviews also have a few cons .

• They’re time-consuming .
• They’re narrow in scope : they only answer the precise research question.

The 7 steps for conducting a systematic review are explained with an example.

Step 1: Formulate a research question

Formulating the research question is probably the most important step of a systematic review. A clear research question will:

• Allow you to more effectively communicate your research to other researchers and practitioners
• Guide your decisions as you plan and conduct your systematic review

A good research question for a systematic review has four components, which you can remember with the acronym PICO :

• Population(s) or problem(s)
• Intervention(s)
• Comparison(s)

You can rearrange these four components to write your research question:

• What is the effectiveness of I versus C for O in P ?

Sometimes, you may want to include a fourth component, the type of study design . In this case, the acronym is PICOT .

• Type of study design(s)
• The population of patients with eczema
• The intervention of probiotics
• In comparison to no treatment, placebo , or non-probiotic treatment
• The outcome of changes in participant-, parent-, and doctor-rated symptoms of eczema and quality of life
• Randomised control trials, a type of study design

Their research question was:

• What is the effectiveness of probiotics versus no treatment, a placebo, or a non-probiotic treatment for reducing eczema symptoms and improving quality of life in patients with eczema?

Step 2: Develop a protocol

A protocol is a document that contains your research plan for the systematic review. This is an important step because having a plan allows you to work more efficiently and reduces bias.

Your protocol should include the following components:

• Background information : Provide the context of the research question, including why it’s important.
• Research objective(s) : Rephrase your research question as an objective.
• Selection criteria: State how you’ll decide which studies to include or exclude from your review.
• Search strategy: Discuss your plan for finding studies.
• Analysis: Explain what information you’ll collect from the studies and how you’ll synthesise the data.

If you’re a professional seeking to publish your review, it’s a good idea to bring together an advisory committee . This is a group of about six people who have experience in the topic you’re researching. They can help you make decisions about your protocol.

It’s highly recommended to register your protocol. Registering your protocol means submitting it to a database such as PROSPERO or ClinicalTrials.gov .

Step 3: Search for all relevant studies

Searching for relevant studies is the most time-consuming step of a systematic review.

To reduce bias, it’s important to search for relevant studies very thoroughly. Your strategy will depend on your field and your research question, but sources generally fall into these four categories:

• Databases: Search multiple databases of peer-reviewed literature, such as PubMed or Scopus . Think carefully about how to phrase your search terms and include multiple synonyms of each word. Use Boolean operators if relevant.
• Handsearching: In addition to searching the primary sources using databases, you’ll also need to search manually. One strategy is to scan relevant journals or conference proceedings. Another strategy is to scan the reference lists of relevant studies.
• Grey literature: Grey literature includes documents produced by governments, universities, and other institutions that aren’t published by traditional publishers. Graduate student theses are an important type of grey literature, which you can search using the Networked Digital Library of Theses and Dissertations (NDLTD) . In medicine, clinical trial registries are another important type of grey literature.
• Experts: Contact experts in the field to ask if they have unpublished studies that should be included in your review.

At this stage of your review, you won’t read the articles yet. Simply save any potentially relevant citations using bibliographic software, such as Scribbr’s APA or MLA Generator .

• Databases: EMBASE, PsycINFO, AMED, LILACS, and ISI Web of Science
• Handsearch: Conference proceedings and reference lists of articles
• Grey literature: The Cochrane Library, the metaRegister of Controlled Trials, and the Ongoing Skin Trials Register
• Experts: Authors of unpublished registered trials, pharmaceutical companies, and manufacturers of probiotics

Step 4: Apply the selection criteria

Applying the selection criteria is a three-person job. Two of you will independently read the studies and decide which to include in your review based on the selection criteria you established in your protocol . The third person’s job is to break any ties.

To increase inter-rater reliability , ensure that everyone thoroughly understands the selection criteria before you begin.

If you’re writing a systematic review as a student for an assignment, you might not have a team. In this case, you’ll have to apply the selection criteria on your own; you can mention this as a limitation in your paper’s discussion.

You should apply the selection criteria in two phases:

• Based on the titles and abstracts : Decide whether each article potentially meets the selection criteria based on the information provided in the abstracts.
• Based on the full texts: Download the articles that weren’t excluded during the first phase. If an article isn’t available online or through your library, you may need to contact the authors to ask for a copy. Read the articles and decide which articles meet the selection criteria.

It’s very important to keep a meticulous record of why you included or excluded each article. When the selection process is complete, you can summarise what you did using a PRISMA flow diagram .

Next, Boyle and colleagues found the full texts for each of the remaining studies. Boyle and Tang read through the articles to decide if any more studies needed to be excluded based on the selection criteria.

When Boyle and Tang disagreed about whether a study should be excluded, they discussed it with Varigos until the three researchers came to an agreement.

Step 5: Extract the data

Extracting the data means collecting information from the selected studies in a systematic way. There are two types of information you need to collect from each study:

• Information about the study’s methods and results . The exact information will depend on your research question, but it might include the year, study design , sample size, context, research findings , and conclusions. If any data are missing, you’ll need to contact the study’s authors.
• Your judgement of the quality of the evidence, including risk of bias .

You should collect this information using forms. You can find sample forms in The Registry of Methods and Tools for Evidence-Informed Decision Making and the Grading of Recommendations, Assessment, Development and Evaluations Working Group .

Extracting the data is also a three-person job. Two people should do this step independently, and the third person will resolve any disagreements.

They also collected data about possible sources of bias, such as how the study participants were randomised into the control and treatment groups.

Step 6: Synthesise the data

Synthesising the data means bringing together the information you collected into a single, cohesive story. There are two main approaches to synthesising the data:

• Narrative ( qualitative ): Summarise the information in words. You’ll need to discuss the studies and assess their overall quality.
• Quantitative : Use statistical methods to summarise and compare data from different studies. The most common quantitative approach is a meta-analysis , which allows you to combine results from multiple studies into a summary result.

Generally, you should use both approaches together whenever possible. If you don’t have enough data, or the data from different studies aren’t comparable, then you can take just a narrative approach. However, you should justify why a quantitative approach wasn’t possible.

Boyle and colleagues also divided the studies into subgroups, such as studies about babies, children, and adults, and analysed the effect sizes within each group.

Step 7: Write and publish a report

The purpose of writing a systematic review article is to share the answer to your research question and explain how you arrived at this answer.

Your article should include the following sections:

• Abstract : A summary of the review
• Introduction : Including the rationale and objectives
• Methods : Including the selection criteria, search method, data extraction method, and synthesis method
• Results : Including results of the search and selection process, study characteristics, risk of bias in the studies, and synthesis results
• Discussion : Including interpretation of the results and limitations of the review
• Conclusion : The answer to your research question and implications for practice, policy, or research

To verify that your report includes everything it needs, you can use the PRISMA checklist .

Once your report is written, you can publish it in a systematic review database, such as the Cochrane Database of Systematic Reviews , and/or in a peer-reviewed journal.

A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a dissertation , thesis, research paper , or proposal .

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarise yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

Turney, S. (2022, October 17). Systematic Review | Definition, Examples & Guide. Scribbr. Retrieved 15 April 2024, from https://www.scribbr.co.uk/research-methods/systematic-reviews/

Is this article helpful?

Shaun Turney

Other students also liked, what is a literature review | guide, template, & examples, exploratory research | definition, guide, & examples, what is peer review | types & examples.

Quantitative Research

• Reference work entry
• First Online: 13 January 2019
• Cite this reference work entry

• Leigh A. Wilson 2 , 3  

4184 Accesses

4 Citations

Quantitative research methods are concerned with the planning, design, and implementation of strategies to collect and analyze data. Descartes, the seventeenth-century philosopher, suggested that how the results are achieved is often more important than the results themselves, as the journey taken along the research path is a journey of discovery. High-quality quantitative research is characterized by the attention given to the methods and the reliability of the tools used to collect the data. The ability to critique research in a systematic way is an essential component of a health professional’s role in order to deliver high quality, evidence-based healthcare. This chapter is intended to provide a simple overview of the way new researchers and health practitioners can understand and employ quantitative methods. The chapter offers practical, realistic guidance in a learner-friendly way and uses a logical sequence to understand the process of hypothesis development, study design, data collection and handling, and finally data analysis and interpretation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Babbie ER. The practice of social research. 14th ed. Belmont: Wadsworth Cengage; 2016.

Google Scholar  

Descartes. Cited in Halverston, W. (1976). In: A concise introduction to philosophy, 3rd ed. New York: Random House; 1637.

Doll R, Hill AB. The mortality of doctors in relation to their smoking habits. BMJ. 1954;328(7455):1529–33. https://doi.org/10.1136/bmj.328.7455.1529 .

Article   Google Scholar  

Liamputtong P. Research methods in health: foundations for evidence-based practice. 3rd ed. Melbourne: Oxford University Press; 2017.

McNabb DE. Research methods in public administration and nonprofit management: quantitative and qualitative approaches. 2nd ed. New York: Armonk; 2007.

Merriam-Webster. Dictionary. http://www.merriam-webster.com . Accessed 20th December 2017.

Olesen Larsen P, von Ins M. The rate of growth in scientific publication and the decline in coverage provided by Science Citation Index. Scientometrics. 2010;84(3):575–603.

Pannucci CJ, Wilkins EG. Identifying and avoiding bias in research. Plast Reconstr Surg. 2010;126(2):619–25. https://doi.org/10.1097/PRS.0b013e3181de24bc .

Petrie A, Sabin C. Medical statistics at a glance. 2nd ed. London: Blackwell Publishing; 2005.

Portney LG, Watkins MP. Foundations of clinical research: applications to practice. 3rd ed. New Jersey: Pearson Publishing; 2009.

Sheehan J. Aspects of research methodology. Nurse Educ Today. 1986;6:193–203.

Wilson LA, Black DA. Health, science research and research methods. Sydney: McGraw Hill; 2013.

Download references

Author information

Authors and affiliations.

School of Science and Health, Western Sydney University, Penrith, NSW, Australia

Leigh A. Wilson

Faculty of Health Science, Discipline of Behavioural and Social Sciences in Health, University of Sydney, Lidcombe, NSW, Australia

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Leigh A. Wilson .

Editor information

Editors and affiliations.

Pranee Liamputtong

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Singapore Pte Ltd.

About this entry

Cite this entry.

Wilson, L.A. (2019). Quantitative Research. In: Liamputtong, P. (eds) Handbook of Research Methods in Health Social Sciences. Springer, Singapore. https://doi.org/10.1007/978-981-10-5251-4_54

Download citation

DOI : https://doi.org/10.1007/978-981-10-5251-4_54

Published : 13 January 2019

Publisher Name : Springer, Singapore

Print ISBN : 978-981-10-5250-7

Online ISBN : 978-981-10-5251-4

eBook Packages : Social Sciences Reference Module Humanities and Social Sciences Reference Module Business, Economics and Social Sciences

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• Quantitative vs. Qualitative Research

Research can be   quantitative or qualitative  or both:

• A quantitative systematic review will include studies that have numerical data.
• A qualitative systematic review derives data from observation, interviews, or verbal interactions and focuses on the meanings and interpretations of the participants. It may include focus groups, interviews, observations and diaries.

Video source:  UniversityNow: Quantitative vs. Qualitative Research

For more information on searching for qualitative evidence see:

Booth, A. (2016). Searching for qualitative research for inclusion in systematic reviews: A structured methodological review.  Systematic Reviews, 5 (1), 1–23. https://doi.org/10.1186/S13643-016-0249-X/TABLES/5

• << Previous: Study Types & Terminology
• Next: Critical Appraisal of Studies >>
• Types of Questions
• Key Features and Limitations
• Is a Systematic Review Right for Your Research?
• Integrative Review
• Scoping Review
• Rapid Review
• Meta-Analysis/Meta-Synthesis
• Selecting a Review Type
• Reducing Bias
• Guidelines for Student Researchers
• Training Resources
• Register Your Protocol
• Handbooks & Manuals
• Reporting Guidelines
• PRESS 2015 Guidelines
• Search Strategies
• Selected Databases
• Grey Literature
• Handsearching
• Citation Searching
• Study Types & Terminology
• Critical Appraisal of Studies
• Broad Functionality Programs & Tools
• Search Strategy Tools
• Deduplication Tools
• CItation Screening
• Critical Appraisal Tools
• Quality Assessment/Risk of Bias Tools
• Data Collection/Extraction
• Meta Analysis Tools
• Books on Systematic Reviews
• Finding Systematic Review Articles in the Databases
• Systematic Review Journals
• More Resources
• Evidence-Based Practice Research in Nursing
• Citation Management Programs
• Last Updated: Apr 5, 2024 2:54 PM
• URL: https://libguides.adelphi.edu/Systematic_Reviews

Quantitative synthesis in systematic reviews

Affiliation.

• 1 Division of Clinical Care Research, New England Medical Center, Boston, MA 02111, USA.
• PMID: 9382404
• DOI: 10.7326/0003-4819-127-9-199711010-00008

The final common pathway for most systematic reviews is a statistical summary of the data, or meta-analysis. The complex methods used in meta-analyses should always be complemented by clinical acumen and common sense in designing the protocol of a systematic review, deciding which data can be combined, and determining whether data should be combined. Both continuous and binary data can be pooled. Most meta-analyses summarize data from randomized trials, but other applications, such as the evaluation of diagnostic test performance and observational studies, have also been developed. The statistical methods of meta-analysis aim at evaluating the diversity (heterogeneity) among the results of different studies, exploring and explaining observed heterogeneity, and estimating a common pooled effect with increased precision. Fixed-effects models assume that an intervention has a single true effect, whereas random-effects models assume that an effect may vary across studies. Meta-regression analyses, by using each study rather than each patient as a unit of observation, can help to evaluate the effect of individual variables on the magnitude of an observed effect and thus may sometimes explain why study results differ. It is also important to assess the robustness of conclusions through sensitivity analyses and a formal evaluation of potential sources of bias, including publication bias and the effect of the quality of the studies on the observed effect.

Publication types

• Research Support, U.S. Gov't, P.H.S.
• Meta-Analysis as Topic*
• Randomized Controlled Trials as Topic
• Research Design* / standards
• Statistics as Topic

Grants and funding

• R01 HS 08532/HS/AHRQ HHS/United States
• R01 HS07782/HS/AHRQ HHS/United States
• T32 AI07389/AI/NIAID NIH HHS/United States

• Open access
• Published: 15 December 2015

Qualitative and mixed methods in systematic reviews

• David Gough 1  

Systematic Reviews volume  4 , Article number:  181 ( 2015 ) Cite this article

22k Accesses

46 Citations

23 Altmetric

Metrics details

Expanding the range of methods of systematic review

The logic of systematic reviews is very simple. We use transparent rigorous approaches to undertake primary research, and so we should do the same in bringing together studies to describe what has been studied (a research map) or to integrate the findings of the different studies to answer a research question (a research synthesis). We should not really need to use the term ‘systematic’ as it should be assumed that researchers are using and reporting systematic methods in all of their research, whether primary or secondary. Despite the universality of this logic, systematic reviews (maps and syntheses) are much better known in health research and for answering questions of the effectiveness of interventions (what works). Systematic reviews addressing other sorts of questions have been around for many years, as in, for example, meta ethnography [ 1 ] and other forms of conceptual synthesis [ 2 ], but only recently has there been a major increase in the use of systematic review approaches to answer other sorts of research questions.

There are probably several reasons for this broadening of approach. One may be that the increased awareness of systematic reviews has made people consider the possibilities for all areas of research. A second related factor may be that more training and funding resources have become available and increased the capacity to undertake such varied review work.

A third reason could be that some of the initial anxieties about systematic reviews have subsided. Initially, there were concerns that their use was being promoted by a new managerialism where reviews, particularly effectiveness reviews, were being used to promote particular ideological and theoretical assumptions and to indirectly control research agendas. However, others like me believe that explicit methods should be used to enable transparency of perspectives driving research and to open up access to and participation in research agendas and priority setting [ 3 ] as illustrated, for example, by the James Lind Alliance (see http://www.jla.nihr.ac.uk/ ).

A fourth possible reason for the development of new approaches is that effectiveness reviews have themselves broadened. Some ‘what works’ reviews can be open to criticism for only testing a ‘black box’ hypothesis of what works with little theorizing or any logic model about why any such hypothesis should be true and the mechanisms involved in such processes. There is now more concern to develop theory and to test how variables combine and interact. In primary research, qualitative strategies are advised prior to undertaking experimental trials [ 4 , 5 ] and similar approaches are being advocated to address complexity in reviews [ 6 ], in order to ask questions and use methods that address theories and processes that enable an understanding of both impact and context.

This Special Issue of Systematic Reviews Journal is providing a focus for these new methods of review whether these use qualitative review methods on their own or mixed together with more quantitative approaches. We are linking together with the sister journal Trials for this Special Issue as there is a similar interest in what qualitative approaches can and should contribute to primary research using experimentally controlled trials (see Trials Special Issue editorial by Claire Snowdon).

Dimensions of difference in reviews

Developing the range of methods to address different questions for review creates a challenge in describing and understanding such methods. There are many names and brands for the new methods which may or may not withstand the changes of historical time, but another way to comprehend the changes and new developments is to consider the dimensions on which the approaches to review differ [ 7 , 8 ].

One important distinction is the research question being asked and the associated paradigm underlying the method used to address this question. Research assumes a particular theoretical position and then gathers data within this conceptual lens. In some cases, this is a very specific hypothesis that is then tested empirically, and sometimes, the research is more exploratory and iterative with concepts being emergent and constructed during the research process. This distinction is often labelled as quantitative or positivist versus qualitative or constructionist. However, this can be confusing as much research taking a ‘quantitative’ perspective does not have the necessary numeric data to analyse. Even if it does have such data, this might be explored for emergent properties. Similarly, research taking a ‘qualitative’ perspective may include implicit quantitative themes in terms of the extent of different qualitative findings reported by a study.

Sandelowski and colleagues’ solution is to consider the analytic activity and whether this aggregates (adds up) or configures (arranges) the data [ 9 ]. In a randomized controlled trial and an effectiveness review of such studies, the main analysis is the aggregation of data using a priori non-emergent strategies with little iteration. However, there may also be post hoc analysis that is more exploratory in arranging (configuring) data to identify patterns as in, for example, meta regression or qualitative comparative analysis aiming to identify the active ingredients of effective interventions [ 10 ]. Similarly, qualitative primary research or reviews of such research are predominantly exploring emergent patterns and developing concepts iteratively, yet there may be some aggregation of data to make statements of generalizations of extent.

Even where the analysis is predominantly configuration, there can be a wide variation in the dimensions of difference of iteration of theories and concepts. In thematic synthesis [ 11 ], there may be few presumptions about the concepts that will be configured. In meta ethnography which can be richer in theory, there may be theoretical assumptions underlying the review question framing the analysis. In framework synthesis, there is an explicit conceptual framework that is iteratively developed and changed through the review process [ 12 , 13 ].

In addition to the variation in question, degree of configuration, complexity of theory, and iteration are many other dimensions of difference between reviews. Some of these differences follow on from the research questions being asked and the research paradigm being used such as in the approach to searching (exhaustive or based on exploration or saturation) and the appraisal of the quality and relevance of included studies (based more on risk of bias or more on meaning). Others include the extent that reviews have a broad question, depth of analysis, and the extent of resultant ‘work done’ in terms of progressing a field of inquiry [ 7 , 8 ].

Mixed methods reviews

As one reason for the growth in qualitative synthesis is what they can add to quantitative reviews, it is not surprising that there is also growing interest in mixed methods reviews. This reflects similar developments in primary research in mixing methods to examine the relationship between theory and empirical data which is of course the cornerstone of much research. But, both primary and secondary mixed methods research also face similar challenges in examining complex questions at different levels of analysis and of combining research findings investigated in different ways and may be based on very different epistemological assumptions [ 14 , 15 ].

Some mixed methods approaches are convergent in that they integrate different data and methods of analysis together at the same time [ 16 , 17 ]. Convergent systematic reviews could be described as having broad inclusion criteria (or two or more different sets of criteria) for methods of primary studies and have special methods for the synthesis of the resultant variation in data. Other reviews (and also primary mixed methods studies) are sequences of sub-reviews in that one sub-study using one research paradigm is followed by another sub-study with a different research paradigm. In other words, a qualitative synthesis might be used to explore the findings of a prior quantitative synthesis or vice versa [ 16 , 17 ].

An example of a predominantly aggregative sub-review followed by a configuring sub-review is the EPPI-Centre’s mixed methods review of barriers to healthy eating [ 18 ]. A sub-review on the effectiveness of public health interventions showed a modest effect size. A configuring review of studies of children and young people’s understanding and views about eating provided evidence that the public health interventions did not take good account of such user views research, and that the interventions most closely aligned to the user views were the most effective. The already mentioned qualitative comparative analysis to identify the active ingredients within interventions leading to impact could also be considered a qualitative configuring investigation of an existing quantitative aggregative review [ 10 ].

An example of a predominantly configurative review followed by an aggregative review is realist synthesis. Realist reviews examine the evidence in support of mid-range theories [ 19 ] with a first stage of a configuring review of what is proposed by the theory or proposal (what would need to be in place and what casual pathways would have to be effective for the outcomes proposed by the theory to be supported?) and a second stage searching for empirical evidence to test for those necessary conditions and effectiveness of the pathways. The empirical testing does not however use a standard ‘what works’ a priori methods approach but rather a more iterative seeking out of evidence that confirms or undermines the theory being evaluated [ 20 ].

Although sequential mixed methods approaches are considered to be sub-parts of one larger study, they could be separate studies as part of a long-term strategic approach to studying an issue. We tend to see both primary studies and reviews as one-off events, yet reviews are a way of examining what we know and what more we want to know as a strategic approach to studying an issue over time. If we are in favour of mixing paradigms of research to enable multiple levels and perspectives and mixing of theory development and empirical evaluation, then we are really seeking mixed methods research strategies rather than simply mixed methods studies and reviews.

Noblit G. Hare RD: meta-ethnography: synthesizing qualitative studies. Newbury Park NY: Sage Publications; 1988.

Google Scholar  

Barnett-Page E, Thomas J. Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol. 2009;9:59.

Article   PubMed   PubMed Central   Google Scholar  

Gough D, Elbourne D. Systematic research synthesis to inform policy, practice and democratic debate. Soc Pol Soc. 2002;2002:1.

Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance 2015. BMJ. 2015;350:h1258

Candy B, Jone L, King M, Oliver S. Using qualitative evidence to help understand complex palliative care interventions: a novel evidence synthesis approach. BMJ Support Palliat Care. 2014;4:Supp A41–A42.

Article   Google Scholar  

Noyes J, Gough D, Lewin S, Mayhew A, Michie S, Pantoja T, et al. A research and development agenda for systematic reviews that ask complex questions about complex interventions. J Clin Epidemiol. 2013;66:11.

Gough D, Oliver S, Thomas J. Introduction to systematic reviews. London: Sage; 2012.

Gough D, Thomas J, Oliver S. Clarifying differences between review designs and methods. Syst Rev. 2012;1:28.

Sandelowski M, Voils CJ, Leeman J, Crandlee JL. Mapping the mixed methods-mixed research synthesis terrain. J Mix Methods Res. 2012;6:4.

Thomas J, O’Mara-Eves A, Brunton G. Using qualitative comparative analysis (QCA) in systematic reviews of complex interventions: a worked example. Syst Rev. 2014;3:67.

Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008;8:45.

Oliver S, Rees R, Clarke-Jones L, Milne R, Oakley AR, Gabbay J, et al. A multidimensional conceptual framework for analysing public involvement in health services research. Health Exp. 2008;11:72–84.

Booth A, Carroll C. How to build up the actionable knowledge base: the role of ‘best fit’ framework synthesis for studies of improvement in healthcare. BMJ Qual Saf. 2015. 2014-003642.

Brannen J. Mixed methods research: a discussion paper. NCRM Methods Review Papers, 2006. NCRM/005.

Creswell J. Mapping the developing landscape of mixed methods research. In: Teddlie C, Tashakkori A, editors. SAGE handbook of mixed methods in social & behavioral research. New York: Sage; 2011.

Morse JM. Principles of mixed method and multi-method research design. In: Teddlie C, Tashakkori A, editors. Handbook of mixed methods in social and behavioural research. London: Sage; 2003.

Pluye P, Hong QN. Combining the power of stories and the power of numbers: mixed methods research and mixed studies reviews. Annu Rev Public Health. 2014;35:29–45.

Harden A, Thomas J. Mixed methods and systematic reviews: examples and emerging issues. In: Tashakkori A, Teddlie C, editors. Handbook of mixed methods in the social and behavioral sciences. 2nd ed. London: Sage; 2010. p. 749–74.

Chapter   Google Scholar  

Pawson R. Evidenced-based policy: a realist perspective. London: Sage; 2006.

Book   Google Scholar  

Gough D. Meta-narrative and realist reviews: guidance, rules, publication standards and quality appraisal. BMC Med. 2013;11:22.

Download references

Author information

Authors and affiliations.

EPPI-Centre, Social Science Research Unit, University College London, London, WC1H 0NR, UK

David Gough

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to David Gough .

Additional information

Competing interests.

The author is a writer and researcher in this area. The author declares that he has no other competing interests.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Gough, D. Qualitative and mixed methods in systematic reviews. Syst Rev 4 , 181 (2015). https://doi.org/10.1186/s13643-015-0151-y

Download citation

Received : 13 October 2015

Accepted : 29 October 2015

Published : 15 December 2015

DOI : https://doi.org/10.1186/s13643-015-0151-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Systematic Reviews

ISSN: 2046-4053

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• Open access
• Published: 11 October 2023

Implementation determinants of physical activity interventions in primary health care settings using the TICD framework: a systematic review

• Catarina Santos Silva 1 , 2 ,
• Cristina Godinho 2 , 3 ,
• Jorge Encantado 1 ,
• Bruno Rodrigues 4 ,
• Eliana V. Carraça 5 ,
• Pedro J. Teixeira 1 &
• Marlene Nunes Silva 2 , 5  

BMC Health Services Research volume  23 , Article number:  1082 ( 2023 ) Cite this article

1207 Accesses

1 Altmetric

Metrics details

Translation into practice of effective physical activity interventions in primary care is difficult, due to a complex interaction of implementation determinants. We aimed to identify implementation barriers and facilitators of four primary care interventions: physical activity assessment, counselling, prescription, and referral.

A systematic review of qualitative, quantitative and mixed-methods studies published since 2016 was conducted. The “Tailored Implementation for Chronic Diseases” (TICD) framework was adapted to extract and synthesize barriers and facilitators.

Sixty-two studies met the inclusion criteria. Barriers ( n  = 56) and facilitators ( n  = 55) were identified across seven domains, related to characteristics of the intervention, individual factors of the implementers and receivers, organizational factors, and political and social determinants. The five most frequently reported determinants were: professionals’ knowledge and skills; intervention feasibility/compatibility with primary health care routine; interventions’ cost and financial incentives; tools and materials; and professionals’ cognitions and attitudes. “Social, political and legal factors” domain was the least reported. Physical activity counselling, prescription, and referral were influenced by determinants belonging to all the seven domains.

The implementation of physical activity interventions in primary care is influenced by a broader range of determinants. Barriers and facilitators related with health professionals, intervention characteristics, and available resources were the most frequently reported. A deep understanding of the local context, with particularly emphasis on these determinants, should be considered when preparing an intervention implementation, in order to contribute for designing tailored implementation strategies and optimize the interventions’ effectiveness.

Peer Review reports

The importance of maintaining regular physical activity (PA) is well established both for preventive care [ 1 ] and as a therapeutic adjuvant [ 2 ], in several chronic conditions. However, worldwide physical inactivity prevalence remains high [ 3 , 4 , 5 ].

The critical role of health systems in the promotion of PA as a way of tackling non-communicable diseases has been highlighted by the World Health Organization (WHO) during the last decade [ 6 ] with primary health care services gaining more relevance particularly since 2016 [ 7 , 8 ]. More recently, the WHO Global Action Plan for PA Promotion 2018–2030 [ 9 ] has established the development of PA promotion systems within health care services – directed at patients and implemented by appropriately trained health professionals – as a priority action. A toolkit specifically designed to primary care [ 10 ] has since been created, encompassing strategies developed to support countries implementing and strengthening systems of patients’ PA assessment and counselling, as part of universal health care. Despite efforts made, only 40% of countries reported having a national protocol in this regard in 2021 [ 3 ].

Several types of primary care intervention models have been developed. They can be grouped in four major intervention types [ 10 ]: i. PA screening/assessment , which corresponds to a systematic application of an enquiry to identify patients’ levels of PA and sedentary behaviour [ 10 , 11 ]; ii. PA brief counselling/advice , comprising a verbal encouragement and/or a verbal or written recommendation for PA, performed by a professional during routine care, also involving an approach to motivations, barriers, preferences, readiness, patient's health, and opportunities to perform PA [ 10 , 12 , 13 ]; iii. exercise prescription , comprising an initial assessment of the patients’ physical and functional fitness, body composition, past PA and clinical history, and goals/motivations, followed by a detailed selection and explanation of exercises according to the patients’ initial assessment, and also including a systematic monitoring and evaluation of effects [ 12 ]; and iv. exercise referral scheme , made by a primary care professional to a third-party service, which is responsible to prescribe a tailored PA/exercise program to the patient [ 10 , 13 , 14 , 15 ]. These intervention types can be implemented individually or in combination.

Previous research evaluating these interventions has revealed clinically relevant increases in patients’ PA levels [ 16 , 17 , 18 , 19 , 20 ]. However, studies assessing interventions’ external validity, when implemented in real-world settings and integrated in primary health care assistance activities, are lacking, limiting the generalizability of such results [ 20 ]. The current research-to-practice evidence gap highlights the importance of addressing contextual determinants (barriers and facilitators) to generate evidence for implementation strategies, thus contributing for the translation of evidence-based interventions into healthcare practice [ 13 , 21 , 22 , 23 ].

Key determinants of healthcare practice may be related to environmental (e.g., socio-political and legal factors) or organizational characteristics (e.g., decision-making processes, capacity for organizational change, and the existence or absence of resources and incentives), but also with characteristics of implementers, receivers, and/or the intervention itself. These determinants have been systematized through different checklists, frameworks, taxonomies, and classification systems [ 24 , 25 , 26 , 27 , 28 , 29 ]. Based on these, a comprehensive and integrated checklist of determinants was specifically developed for healthcare professional practice – the “Tailored Implementation for Chronic Diseases” (TICD) checklist [ 30 ], to optimize reflection and data collection on determinants of implementation. When introducing quality improvements or new interventions in healthcare, a proper investigation of implementation barriers and facilitators is critical to reveal the most relevant intervention- and context-specific ones, aiming at the development of tailored implementation strategies and more effective interventions [ 30 ].

There is a limited number of systematic reviews aimed at reporting implementation barriers and facilitators of PA interventions [ 31 ]. Some have focused in the primary health care system, but have not included PA-only interventions alone (considering weight management programs and lifestyle interventions, for instance), and were limited to analysing stakeholders’ views [ 32 ] or health professionals’ determinants and views [ 33 , 34 , 35 ], and/or considered a single PA intervention type [ 19 , 35 ]. Thus, there is a need for systematic identification of whole-system implementation barriers and facilitators of the most common PA-specific promotion interventions implemented in primary care.

This systematic review aimed to identify implementation barriers and facilitators, according to the TICD framework, within the four described PA promotion interventions delivered in primary health care settings by health professionals to adult patients.

This systematic review was reported in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) 2020 statement [ 36 ] (see Additional file 1 ).

Eligibility criteria

We included peer-reviewed studies published since January 2016, the publication year of both the “Physical activity strategy for the WHO European Region 2016–2025” [ 7 ] and the guide “Integrating diet, physical activity and weight management services into primary care” [ 8 ]. Although there are studies on this topic published before this year, constant changes in health care systems, scientific knowledge, and population health pattern might make older studies not representative of today’s reality. Furthermore, 2016 marked a stronger and more focused WHO’s recommendation of PA promotion interventions in primary health care. Therefore, only studies published since 2016 were considered. We considered studies with primary care health professionals, patients (≥ 18 years), and stakeholders involved in one of the four types of PA promotion and/or sedentary time reduction interventions (i.e., PA assessment, counselling, prescription and/or referral), delivered in primary health care settings, at least in part, face-to-face. Included studies should formally assess interventions’ implementation barriers and facilitators. Several types of study design were included (i.e., qualitative, quantitative or mixed-methods).

Studies including rehabilitation patients, or patients with contraindications to perform PA autonomously, those testing interventions not specifically targeting PA promotion alone (e.g., lifestyle interventions, weight management interventions, etc.) or digital-only interventions, study protocols, literature reviews, opinion articles, conference books or papers, non-peer reviewed scientific literature (e.g., books, book chapters), and non-English or Portuguese written literature were excluded.

Information sources

A systematic literature search for titles and abstracts was conducted in five electronic databases: Web of Science, Scopus, PsycInfo, PubMed, and Medline. Databases were last searched in July 12 th , 2023.

Search strategy

The search strategy comprised a combination of terms from four different categories: behaviours of interest, interventions of interest, implementation context, and review’s main outcomes (i.e., implementation determinants). The full search stem can be found in Additional file 2 .

Selection and data collection processes

Two reviewers (CSS and JE) independently screened titles and abstracts and three reviewers (CSS, JE, and BR) independently analysed full text articles against eligibility criteria. A consistency check between the authors was performed in 15% of randomly selected titles and abstracts and in 20% of randomly selected full-texts to obtain inter-reviewer agreement (Cohen’s kappa and Fleiss’ kappa, respectively). Authors were blind to each other’s decisions and, given that good to excellent agreement was found in their assessments (Cohen k  = 1; Fleiss’ k  = 0.615), they independently screened the other 85% of titles and abstracts and 80% of full text articles. Disagreements between individual decisions were discussed to reach consensus. CADIMA® online software was used to record decisions on title and abstract screening and full text analysis. When full text articles were unavailable, authors were contacted and readily made their work available in all cases. Three reviewers (CSS, JE, and BR) independently extracted data. An excel spreadsheet was used to record extracted data. TICD framework categories [ 30 ] were used to guide data extraction.

Extracted data comprised the following outcome items of significance to the review objectives: guideline factors; individual health professional factors; patient factors; professional interactions; incentives and resources; capacity for organizational change; social, political, and legal factors; and any other factor assessed as a barrier and/or facilitator of implementation of the interventions of interest. Relevant statistical data on the outcomes of interest was also extracted, when applicable, as an indicator of its relevance. Other study information was also extracted: author; year; country of implementation; type of study; methodology; trial (if applicable); intervention; outcome; and participants’ characteristics (number of participants; health professional or stakeholder category or if the sample consisted of patients; mean age; sex distribution; patients’ chronic diseases, if applicable).

Study quality assessment

Two authors (CSS and JE) independently performed a critical appraisal of all articles included in the review. A consistency check between the two authors was performed in 15% of randomly selected studies, having obtained a good inter-reviewer agreement (Cohen’s k  = 0.653). Joanna Brigs Institute (JBI) critical appraisal tools [ 37 ] were used to assess studies’ quality. For studies using a mixed-methods methodology, the Mixed-Methods Appraisal Tool (MMAT) [ 38 ] was applied, as there is no specific JBI tool for mixed-methods studies. The critical appraisal assessment is presented for each study against each checklist item, in table format [ 39 ].

Synthesis methods

As this systematic review includes very different studies and its output is qualitative, a narrative synthesis was performed. First, a preliminary synthesis was made using a thematic analysis approach [ 40 ], based on the TICD framework, and studies’ results were presented in tabular form, structured into the framework’s main themes/domains, barriers vs . facilitators, and type of PA promotion intervention. Then, a frequency table of the studies mentioning each kind of implementation barrier and facilitator was made. Last, the studies and their results were presented and relationships in the data were explored, to better interpret the facilitators and barriers of each type of PA promotion intervention. This allowed to understand the different implementation determinants in an articulated, integrated, and systematic way.

Certainty assessment of the systematic review

The Supporting the Use of Research Evidence (SURE) checklist was used, to evaluate the identification, selection and appraisal of studies (5 criteria), how findings were analysed (5 criteria), and to reflect on other considerations (one criterion) [ 41 ].

Study selection

The search strategy identified a total of 4508 records (see Fig.  1 ). After duplicates removal and title and abstract screening, the full-text of 187 records were assessed for eligibility. After exclusion of 125 records for not meeting inclusion criteria, a total of 62 articles were included in this review [ 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 73 , 74 , 75 , 76 , 77 , 78 , 79 , 80 , 81 , 82 , 83 , 84 , 85 , 86 , 87 , 88 , 89 , 90 , 91 , 92 , 93 , 94 , 95 , 96 , 97 , 98 , 99 , 100 , 101 , 102 , 103 ].

PRISMA flowchart

Study characteristics

From the 62 articles included, 48.4% ( n  = 30) employed a qualitative design [ 43 , 46 , 48 , 49 , 54 , 55 , 58 , 60 , 62 , 65 , 67 , 68 , 71 , 76 , 77 , 78 , 79 , 80 , 83 , 84 , 87 , 88 , 89 , 92 , 93 , 94 , 95 , 96 , 97 , 99 ], 37.1% ( n  = 23) a quantitative design [ 42 , 44 , 45 , 52 , 53 , 57 , 61 , 63 , 64 , 66 , 69 , 72 , 73 , 74 , 75 , 81 , 82 , 85 , 91 , 98 , 100 , 101 , 102 ], and 14.5% ( n  = 9) a mixed-methods study design [ 47 , 50 , 51 , 56 , 59 , 70 , 86 , 90 , 103 ]. The majority of the studies (87.1%) were conducted in high income countries (United Kingdom, n  = 15; Canada, n  = 9; USA, n  = 7; other countries, n  = 23), while only 12.9% were conducted in upper middle income countries (Brazil, n  = 4; Thailand, n  = 2; Jordan, n  = 1; Turkey, n  = 1), according to the categorization of the World Bank [ 104 ]. Study characteristics are outlined in Table 1 .

Quality assessment of the studies is reported in Additional file 3 . The main issues found in qualitative studies were the lack of a clear statement of the authors’ philosophical perspective, not addressing researcher’s cultural and theoretical location, as well as researcher-research influence. In mixed-methods studies, the main issue was the non-accomplishment of quality criteria for both study components (qualitative and quantitative). In analytical cross-sectional studies, the main issue was related to the validity and reliability of the instruments used. In prevalence studies, it was unclear whether health conditions were identified using validated methods, and there were also issues related with insufficient coverage of sample subgroups in data analysis. In quasi-experimental studies, the main issues were related to the absence of an independent control group and of a description and analysis of differences between groups at follow-up. As for the analysed randomized controlled trial, the only not fulfilled quality criteria was participants’ blinding.

Barriers and facilitators to implementation of physical activity interventions in primary care

A total of 56 barriers and 55 facilitators to implementation were identified across the seven domains/themes. A supporting codebook, based on TICD framework [ 30 ], is available in Additional file 4 and a full list of these implementation determinants is organized in Table 2 . Detailed data extraction information is available in Additional file 5 .

Intervention/guideline factors

“(Lack of) feasibility/compatibility” and “intervention components/characteristics/content” were the most reported determinants within this domain.

The absence of feasibility/compatibility of PA interventions’ implementation within health professionals’ usual tasks and activities was a key highlighted barrier. Extended time was emphasized as a requirement to implement interventions regularly, while simultaneously addressing the primary reason for the patient’s visit and parallel professional demands and responsibilities. PA interventions requiring a more structured local organization (e.g., a specific PA consultation) were also associated to complex logistics (e.g., specific space, more time needed), more difficult to accommodate. Some studies [ 46 , 58 , 88 ] reported ways by which increasing feasibility/compatibility of the intervention would be a facilitator, for instance, transferring the implementation responsibility to health care professionals who have more consultation time (as dietitians or nurse practitioners).

Some “ intervention components/characteristics/content ” were reported as key facilitators, namely goal setting, action planning, self-monitoring and social support components. Interventions incorporating written prescriptions and regular follow-ups were also seen as facilitators, both by health care professionals and patients. On the other hand, complex methods requiring extensive knowledge by implementers and intervention activities considered chores by the patients (e.g., PA diaries) exemplify the barriers reported in primary studies.

Other intervention/guideline factors were less studied or reported. Evidence is suggestive of the potential facilitator role of “tailored intervention/patient-centred” and “recruitment strategy” used.

Individual health professional factors

“Knowledge and skills”, “cognitions/attitudes”, and “professional behaviour” were the most highlighted determinants within this domain.

Health professional’s “ knowledge and skills ” to promote PA was the most frequently reported/studied determinant, both as barrier and facilitator (50 times in 62 studies). The lack of training or expertise in the area of PA and behaviour change techniques, unfamiliarity with guidelines, lack of knowledge on safety issues concerning PA practice by people with chronic conditions, and unfamiliarity with suitable PA opportunities in the community illustrate the barriers highlighted by the studies’ participants. Receiving training in medical school about PA promotion, training the health care teams working in health surgeries, especially regarding PA promotion in chronically ill patients and in behaviour change techniques, and attending local activities with information about local PA offers were examples of reported facilitators.

Health professionals’ “ cognitions and attitudes ” were also reported both as barriers and as facilitators. Health professionals’ belief that PA is not a relevant and/or effective prevention strategy or treatment, giving it a low priority or finding other lifestyle changes more important, was reported in several studies as barriers. Having a good attitude towards PA promotion, an increased understanding of the importance of PA promotion in healthcare, perception of no barriers to counselling, and considering PA as an important behaviour for good health were in turn emphasised by health professionals as implementation facilitators.

Although less reported than the previous, “ professional behaviour ” was also frequently reported, especially as a facilitator. For instance, patients appreciated trustworthy, supportive, and non-judgmental advice by genuinely interested health professionals. A previous assessment of PA levels and patients’ readiness to change facilitated the implementation of PA counselling and prescription, according to health professionals. Feeling that patients’ PA promotion is outside their professional “ scope of practice/professional role ”, or that it is a role shared by all healthcare professionals and not exclusively by themselves was the third most highlighted barrier.

Patient factors

“Motivation” and “health status” were the two most frequently reported patient-related determinants, being considered both as barrier and facilitator.

Health care professionals perceiving lack of “ motivation ” by their patients was referred as a key barrier. From the patients’ view, no interest in receiving PA counselling was reported, for instance, when they felt they were already sufficiently active or when they already had pre-existing conditions requiring regular contact with health services and did not desire further testing. On the other hand, patients’ perception of PA positive effects on health, the social recognition and feelings of enjoyment derived from PA practice, contributed to their motivation, working as a facilitator.

Patients’ “ health status ”, namely some comorbidities, prevent patients to fully engage in the intervention, while in other cases, the “perceived threat” (e.g., type 2 diabetes) was not sufficient to mobilize change. For health professionals, patients’ illnesses, and the implementation of treatments other than PA competed for attention. Specifically, for some diseases, such as cancer, a significantly low proportion of health professionals recommended PA. On the other hand, health professionals were more likely to recommend PA to patients with overweight or obesity, type 2 diabetes or pre-diabetes, dyslipidaemia, and hypertension.

Although less studied/reported, two other determinants gathered evidence of relevance, as they were the second most reported barriers within this theme: health professionals perceived “ lack of compliance/engagement ” by patients and frustration of patients’ “ expectations ” (e.g., health professionals felt that some patients expected drug treatment instead of exercise, whereas other patients felt that the program was missing more intense exercise training options).

Professional interactions

Professional interactions were mainly reported as facilitators. “Team processes” and “networks” were the two most relevant, playing a key facilitating role in implementation.

Highlighted positive “ team processes ” were mainly related with a good cooperation between PA counsellors and health care professionals, or with a good functioning dynamic of the family health teams.

Another key facilitator was “ networks ”. Health professionals stressed the importance of a connection between sectors, which may result in increased referral of patients, and the importance of involving all stakeholders in a shared mission.

Although less studied/reported, “team communication (constraints)” and “referral processes (constraints)” were the third most reported determinants.

Incentives and resources

“(Cost and lack of) financial incentives” and “assistance tools and materials” were the most frequently highlighted determinants, both as barriers and as facilitators.

“ Cost and lack of financial incentives ” was often felt as a barrier. Patients and health professionals frequently reported expensive memberships in PA facilities for patients. Health professionals also highlighted the lack of financial reimbursements to implementers. Indeed, health professionals’ reimbursements of PA prescriptions and economic subsidies for patients to reduce the cost of joining an exercise facility, or even having a trial period before membership, were often reported as a “financial incentives” facilitator.

Regarding “ assistance tools and materials ” constraints, health professionals often highlighted lack of instructional material and effective tools and educational information to give to patients. On the other hand, the availability of specific intervention support tools and materials (e.g., practitioner toolbox; standardized and up-to-date information about where to refer patients, as a "community mapping” including PA facilities within the geographical area; decision algorithms) were believed to facilitate the implementation process, with technological tools being especially welcomed by health professionals.

Indeed, the “ information system ” was mainly reported as facilitator. Health professionals welcomed procedures’ digitalization to reduce time and money, namely through the integration of PA promotion tools in the electronic health system, as referral forms, prescription pads, and modules for PA counselling, for instance. Having access to patients’ interdisciplinary health care charts was also reported by health professionals to support tailored counselling.

Providing a “ continuing education system ” offer for health care staff (e.g., regarding PA promotion, its pathways and modes of delivery) was also highlighted as a relevant facilitator.

Capacity for organizational change

“ Capable leadership ” was the most frequently reported implementation determinant. Health professionals and stakeholders identified the election of a formal coordinator/leader, regularly present in the working group and providing support and updated information/knowledge sharing to the implementation team, as an implementation facilitating factor. Managers’ championing and endorsement of the intervention was also emphasized. Cases where the primary care management was not explicitly fulfilling this role hindered the implementation.

Other determinants within this theme were less studied.

Social, political and legal factors

Determinants within this domain were the least studied/reported. “ (Lack of) funder policies ” and “ (economic constraints on the) health care budget ” were reported in five studies, both as barriers and facilitators.

Implementation determinants’ themes according to primary care physical activity intervention type

Table 3 provides a summary of the implementation determinants (main themes) reported in each intervention type.

Three interventions – PA counselling; PA prescription; PA referral schemes – and one combination – PA counselling and referral – gathered implementation barriers and facilitators from all domains, whereas those involving PA assessment seemed to be more influenced by determinants pertaining to intervention/guideline-, deliverers-, and patient-related factors. Intervention/guideline factors and individual health professional factors were reported in all intervention types and combinations, proving to be key determinants to consider when implementing PA interventions in primary healthcare. Patient factors and incentives and resources’ barriers and facilitators were also central to implementation, being reported in the four intervention types. Professional interactions , capacity for organizational change , and social, political, and legal factors did not seem to be considered pivotal in implementation processes of simpler interventions, as PA assessment alone. These groups of determinants played a more relevant role in interventions with more complexity, requiring further delivering resources, as PA counselling, PA prescription, and those involving referral processes.

Considering the reporting frequency of the main themes by each intervention type, PA counselling implementation seems to be mainly hindered by factors related to the intervention/guideline, individual health professionals and patients, and mainly facilitated by individual health professional factors. PA prescription implementation seems to be particularly influenced by barriers and facilitators pertaining incentives and resources, whereas PA referral schemes are predominantly facilitated by factors related to professional interactions.

The SURE tool indicated that this is a good quality systematic review with minor limitations regarding selection procedure: i. language bias, as only studies written in English were selected; and ii. status of publication, as only published studies were included (see Additional file 6 ). A more comprehensive search avoiding these limitations could, thus, have retrieved a higher number of studies. Even so, English is the universal language for science communication, the best available science works tend to be published, and a seven-year time interval can be considered adequate to have an updated picture of today’s health services panorama. Considering the critical appraisal of the included studies and that the output of this systematic review is qualitative, the three quality criteria that probably most negatively influence certainty of the evidence were the non-accomplishment of quality criteria for both study components (qualitative and quantitative) in mixed-methods studies, issues related with the validity and reliability of the instruments used in analytical cross-sectional studies, as well as insufficient coverage of sample subgroups in data analysis in some prevalence studies. However, it is important to stress that the vast majority of the included studies did not present any of these issues. Together, the findings of the present systematic review can be considered reliable for evidence-informed health policymaking. Results of this review should, nevertheless, be interpreted taking these minor limitations into consideration.

This systematic review assessed implementation barriers and facilitators in real-world PA promotion and/or sedentary time reduction interventions (i.e., PA assessment, brief counselling, prescription, and referral scheme) delivered in primary healthcare settings, using the TICD framework [ 30 ]. Five determinants of implementation success stood out from our review, given their reported frequency: having health professionals with a good degree of knowledge and skills regarding PA and its promotion; the need for the intervention to be feasible/compatible with professionals’ and health services’ usual tasks; interventions’ cost and the provision of financial incentives; having adequate tools and materials to implement the intervention; and fostering positive health professionals’ cognitions and attitudes, while minimizing negative ones. These determinants belong to three domains: individual health professional factors ; intervention/guideline factors , and incentives and resources . Despite being less or rarely reported, other determinants may play a particularly facilitating or hindering role regarding interventions’ implementation (e.g., networks). Apart from PA assessment, implementation of all intervention types (excluding combinations) is influenced by factors belonging to all the seven main domains, although some domains were predominantly highlighted in a certain type of intervention: PA counselling seems to be particularly hampered by intervention/guideline and individual (health professionals and patients) factors and facilitated by individual health professionals’ ones; PA prescription seems to be particularly influenced by incentives and resources’ barriers and facilitators; and PA referral schemes seem to be specially facilitated by factors related to professional interactions. PA assessment seems to be more dependent on individual factors (from patients or professionals) and available resources – whereas more complex interventions seem to rely also on organisational, political, and social determinants –, but the limited number of primary studies assessing PA assessment alone can be biasing this specific result.

Health professionals’ knowledge and skills was the most frequent reported determinant and has been previously highlighted as important for proper implementation [ 13 , 32 , 33 , 35 , 105 ]. WHO’s monitoring of the implementation of the Global Action Plan for Physical Activity also reinforced that more pre- and post-graduated training of health professionals is needed – also for professionals outside the health sector – combined with the provision of adequate tools and guidance [ 3 ]. However, training is not always sufficient to determine health professionals’ PA counselling behaviours [ 106 , 107 ]. Despite this, PA promotion in medical schools’ curricula is still a hot topic, as there seems to be a recurrent gap in the pre-graduate medical training [ 108 , 109 , 110 ]. The importance of knowing PA pathways to community resources and behaviour change techniques was mentioned in several works. This reinforces the need for proper training of health professionals, not only in terms of PA content, but also in modes of delivery. Adequate and innovative information systems may be promising tools in supporting face-to-face delivery of behaviour change techniques applied to PA promotion [ 111 ]. A continuing education system that can support in-service professionals (the third most reported facilitator within incentives and resources ’ theme) can also play a relevant role in this regard.

Concerning interventions’ feasibility/compatibility, a recent systematic review on the views of stakeholders also identified the congruence of the intervention with team activities as key facilitator [ 32 ]. The (lack of) compatibility of the intervention with usual tasks may be interrelated with other reported determinants (for instance, having enough human resources). Of these, a significant one is the optimization of the information system, the second most reported facilitator within the “incentives and resources” domain. Indeed, the availability of computerised solutions that help health professionals save time and efforts during interventions’ delivery may be, once more, paramount.

Interventions’ cost has long been a concern regarding PA promotion in primary care and health system sustainability. Particularly, PA counselling and referral brief interventions are very well positioned to be nationally/locally endorsed, as they are considered a “best-buy” to tackle non-communicable diseases, giving their evidence of cost-effectiveness [ 10 , 112 ]. Financial incentives for patients have also gathered evidence in increasing patients’ PA in the short and long term [ 113 ], which can be an effect of an increased patients’ adherence to the intervention. The establishment of networks between healthcare and community PA programmes and resources that brings reduced costs or even free PA options for patients can offer a solution in this regard. Also, a specific budget allocated to health-enhancing physical activity promotion is considered strategic [ 114 ]. Financial incentives for healthcare professionals could, thus, be analysed in this context.

Adequate assistance tools and materials and health professionals’ cognitions and attitudes were also found to be key determinants. This result was shown in other works [ 32 , 33 ], including community-based interventions [ 31 ]. Positive attitudes were linked with patients engagement and facilitated adaptation processes throughout implementation, whereas placing low value on the intervention hindered the implementation [ 31 ]. The relevance attributed to PA promotion in healthcare by medical doctors had also been identified as a significant predictor of clinical practice in this area [ 106 ].

“Social, political, and legal factors” were the least reported domain. Considering that national public health policy and legislation is recognized as crucial by international guidelines [ 9 ], this finding may reflect the scarcity of research specifically addressing health policy/legislation impact in this area. In fact, only one of the included studies [ 44 ] assessed the impact of a legislative framework on PA prescription.

Although the frequency of reporting is useful to obtain a picture of the most and least studied implementation determinants, it does not necessarily reflect the degree of importance of each barrier and facilitator. Caution is needed, as interpretation bias may be introduced if one equates the relevance of each determinant with its reporting frequency. Even so, the identified implementation determinants were under the seven domains of the TICD framework, with even distribution between barriers and facilitators in each domain, evidencing that the studies included explored an extensive set of factors influencing implementation.

This review presents suggestive evidence that other determinants may play an important role and should not be overlooked: patients’ motivation (barrier/facilitator); intervention components/characteristics/content (facilitator); positive team processes (facilitator); and the establishment of networks between sectors/stakeholders (facilitator). Having the knowledge and skills to implement an intervention evidencing compatibility/feasibility with routine care does not mean that implementation cannot be easily hindered by other determinants in place. Together, this evidence suggests that there are some more general implementation determinants and others more context-specific. A broad assessment of implementation barriers and facilitators should, thus, be made when preparing an intervention implementation to understand the local context.

The entire chain of interacting actors within and outside the health sector, influences implementation success. Each one brings unique contributions to the implementation and scaling-up phases. Planning beforehand to identify and engage all relevant stakeholders from the entire delivery chain is of outmost importance to tackle future translational challenges. Nonetheless, primary studies often overlooked the views of politicians, health coordinators or community stakeholders, suggesting an evidence gap. The need for a coordinated systems-approach to foster the implementation of PA interventions in healthcare settings, involving several key stakeholders, has been reported in multiple works in this area [ 13 , 105 , 115 , 116 , 117 ].

Another finding was that adequate implementation of more complex interventions implies the commitment of more structures, beyond the specific contexts of local health facilities, professionals and patients. In line with the “PA vital sign” proposal [ 118 ], it can be hypothesised that the universal implementation of PA assessment should be the first step for PA promotion in primary care, with the more complex ones being gradually introduced. Implementing PA assessment was even reported in primary studies as a facilitator of the subsequent implementation of PA counselling. However, the limited number of primary studies addressing PA assessment alone do not allow to draw firm conclusions on this issue.

Generating knowledge about key implementation barriers and facilitators of PA promotion interventions in primary healthcare contributes to define tailored implementation strategies to improve the adoption, implementation, sustainability, and scaling-up of such interventions [ 23 ]. An iterative planning process should occur to potentiate success: 1) characterizing the delivery context and anticipating barriers and facilitators; 2) designing tailored implementation strategies; 3) monitoring implementation and dealing with implementation determinants that effectively emerge during translation and scale-up; and 4) incorporating these outcomes in the implementation processes to optimize them [ 119 , 120 , 121 ].

Strengths and limitations

To our knowledge, this is the first systematic review analysing theoretically framed implementation barriers and facilitators of four PA interventions (assessment, counselling, prescription, referral) implemented in the primary health care, integrating the views of patients, health professionals and stakeholders. The framework used herein to systematize barriers and facilitators of implementation also constitutes a strength of this review, as it was specifically developed to identify determinants of practice in healthcare contexts, facilitating its identification and organisation in a parsimonious way.

Still, this review is not without limitations. Attention should be paid to the fact that more than one third of the included studies used quantitative designs. As such, some determinants may be intentionally selected and more frequently studied by researchers (e.g., in questionnaires with closed-ended questions), as opposed to implementation determinants that unintentionally emerge from qualitative data. Furthermore, only 31% of the primary studies clearly reported the use of a published framework when identifying implementation determinants, which presents a high risk of bias, as acknowledged barriers and facilitators could have been overlooked. Also, further studies including the views of stakeholders, outside the health sector, remain scarce, precluding a more comprehensive picture of implementation determinants. Most studies included in this systematic review reflect interventions implemented in high income countries, suggesting that the findings presented may not necessarily play a similar role in implementation processes occurring in countries of other income levels. Also, lack of sufficient detail in studies’ description of the PA promotion interventions was common, which may have led to an incorrect classification of the interventions. Earlier described methodological limitations of the primary studies are also concerning factors, as they could have biased the results. Lastly, the time limitation of the literature search poses a methodological limitation, as studies published before 2016 were not considered. Despite this, and together with the reasonable number of included studies obtained ( n  = 62), a fair picture of today’s reality of implementation determinants of PA promotion interventions in primary care was probably achieved. Caution is needed, however, when analysing the results for PA assessment, as only two primary studies addressed this type of intervention alone.

Future research

In order to bridge the gap between research and practice, future research should focus on proper implementation preparation of evidence-based interventions and enhanced dissemination, considering: a) the wide range of agents that should be involved (stakeholders from all levels); b) implementation barriers and facilitators, considering mixed-methods design studies (combining quantitative components, that estimate the degree of influence of each determinant in real-world conditions, with qualitative components that allow the identification of potential barriers and facilitators), with proper interventions’ descriptions, and investing in studies of interventions also delivered in upper middle and low income contexts; c) tailored implementation strategies and implementation plans. In implementing interventions in real-world conditions, an adaptation phase should always be expected, involving constant loops of monitoring and feedback to increase the effect, aligning with the evidence, while fully embed the intervention in a new system and carefully keeping its active ingredients – future research agenda should support these processes as well.

The present review identifies the most relevant implementation determinants of PA-specific promotion interventions in primary health care, from the point of view of health professionals, patients, and stakeholders. These findings address a research-to-practice gap and will support the translation process of science-based interventions.

Although implementation of PA promotion interventions in primary care is determined by a wide set of barriers and facilitators, health professionals-, intervention-, and resources-specific ones seem to be particularly relevant. As such, a careful consideration of these factors is needed when preparing interventions’ delivery. Tailored implementation strategies should be designed for successful implementation, particularly those addressing deliverers’ knowledge/skills, attitudes and cognitions; interventions’ feasibility/compatibility with routine care and cost; and the availability of adequate supporting materials and tools. Suggestive evidence also highlights some barriers and facilitators related with patients’ motivation, intervention characteristics, and professionals’ interactions as relevant. Moreover, implementation determinants are modulated by the type of PA intervention. From a practical implication perspective, there seems to be more context- and intervention-specific determinants, so a deep understanding of the local context combined with intervention’s characteristics is highly recommended when preparing an intervention implementation.

The findings of this review should be considered by primary care authorities and coordination teams aiming to optimize interventions’ implementation and effectiveness in real world conditions – from the design of tailored implementation strategies to the development of national policies, tools and systems to support regional or nationwide scale-up.

Registration and protocol

This systematic review was registered in PROSPERO (CRD42022318632). The protocol was not previously published.

Availability of data and materials

All relevant data used for the current study are within the paper and its supporting information.

Abbreviations

Joanna Brigs Institute

Mixed-Methods Appraisal Tool

Physical activity

Preferred Reporting Items for Systematic reviews and Meta-Analyses

Supporting the Use of Research Evidence

Tailored Implementation for Chronic Diseases

World Health Organization

Lee IM, Shiroma EJ, Lobelo F, Puska P, Blair SN, Katzmarzyk PT, et al. Effect of physical inactivity on major non-communicable diseases worldwide: an analysis of burden of disease and life expectancy. Lancet. 2012;380(9838):219–29. https://doi.org/10.1016/S0140-6736(12)61031-9 .

Article   PubMed   PubMed Central   Google Scholar  

Pedersen BK, Saltin B. Exercise as medicine - evidence for prescribing exercise as therapy in 26 different chronic diseases. Scand J Med Sci Sports. 2015;25(Suppl 3):1–72. https://doi.org/10.1111/sms.12581 .

Article   PubMed   Google Scholar  

World Health Organization. Global status report on physical activity 2022. Geneva: World Health Organization; 2022. Licence: CC BY-NC-SA 3.0 IGO.

Guthold R, Stevens GA, Riley LM, Bull FC. Worldwide trends in insufficient physical activity from 2001 to 2016: a pooled analysis of 358 population-based surveys with 1·9 million participants. Lancet Glob Health. 2018. https://doi.org/10.1016/S2214-109X(18)30357-7 .

Guthold R, Stevens GA, Riley LM, Bull FC. Global trends in insufficient physical activity among adolescents: a pooled analysis of 298 population-based surveys with 1·6 million participants. Lancet Child Adolesc Health. 2020;4(1):23–35. https://doi.org/10.1016/S2352-4642(19)30323-2 .

World Health Organization. Global action plan for the prevention and control of non-communicable diseases 2013–2020. Geneva: World Health Organization; 2013.

World Health Organization. Physical activity strategy for the WHO European Region 2016–2025. Copenhagen: World Health Organization; 2016.

World Health Organization. Integrating diet, physical activity and weight management services into primary care. Geneva: World Health Organization; 2016.

World Health Organization. Global action plan on physical activity 2018–2030: more active people for a healthier world. Geneva: World Health Organization; 2018.

World Health Organization. Promoting physical activity through primary health care: a toolkit. Geneva: World Health Organization; 2021. Licence: CC BY-NC-SA 3.0 IGO.

Wald NJ. The definition of screening. J Med Screen. 2001;8(1):1.

Article   CAS   PubMed   Google Scholar  

Order No. 8932/2017. Diary of the Republic No. 195/2017, Series II. 2017. Available online: https://diariodarepublica.pt/dr/detalhe/despacho/8932-2017-108275649 . Accessed 27 July 2022.

Lion A, Vuillemin A, Thornton JS, Theisen D, Stranges S, Ward M. Physical activity promotion in primary care: a utopian quest? Health Promot Int. 2019;34(4):877–86. https://doi.org/10.1093/heapro/day038 .

Williams NH, Hendry M, France B, Lewis R, Wilkinson C. Effectiveness of exercise-referral schemes to promote physical activity in adults: systematic review. Br J Gen Pract. 2007;57(545):979–86. https://doi.org/10.3399/096016407782604866 .

Anokye NK, Trueman P, Green C, Pavey TG, Hillsdon M, Taylor RS. The cost-effectiveness of exercise referral schemes. BMC Public Health. 2011;11:954. https://doi.org/10.1186/1471-2458-11-954 .

Abu-Omar K, Rütten A, Burlacu I, Schätzlein V, Messing S, Suhrcke M. The cost-effectiveness of physical activity interventions: a systematic review of reviews. Prev Med Rep. 2017;8:72–8. https://doi.org/10.1016/j.pmedr.2017.08.006 .

Lamming L, Pears S, Mason D, Morton K, Bijker M, Sutton S, et al. What do we know about brief interventions for physical activity that could be delivered in primary care consultations? A systematic review of reviews. Prev Med. 2017;99:152–63. https://doi.org/10.1016/j.ypmed.2017.02.017 .

Kallings LV, Leijon M, Hellénius ML, Ståhle A. Physical activity on prescription in primary health care: a follow-up of physical activity level and quality of life. Scand J Med Sci Sports. 2008;18(2):154–61. https://doi.org/10.1111/j.1600-0838.2007.00678.x .

Campbell F, Holmes M, Everson-Hock E, Davis S, Buckley Woods H, Anokye N, et al. A systematic review and economic evaluation of exercise referral schemes in primary care: a short report. Health Technol Assess. 2015;19(60):1–110. https://doi.org/10.3310/hta19600 .

Sanchez A, Bully P, Martinez C, Grandes G. Effectiveness of physical activity promotion interventions in primary care: a review of reviews. Prev Med. 2015;76(Suppl):S56-67. https://doi.org/10.1016/j.ypmed.2014.09.012 .

Fernandez ME, Ten Hoor GA, van Lieshout S, Rodriguez SA, Beidas RS, Parcel G, et al. Implementation mapping: using intervention mapping to develop implementation strategies. Front Public Health. 2019;7:158. https://doi.org/10.3389/fpubh.2019.00158 .

Green LW, Ottoson JM, García C, Hiatt RA. Diffusion theory and knowledge dissemination, utilization, and integration in public health. Annu Rev Public Health. 2009;30:151–74. https://doi.org/10.1146/annurev.publhealth.031308.100049 .

Powell BJ, Fernandez ME, Williams NJ, Aarons GA, Beidas RS, Lewis CC, et al. Enhancing the impact of implementation strategies in healthcare: a research agenda. Front Public Health. 2019;7:3. https://doi.org/10.3389/fpubh.2019.00003 .

Wensing M, Bosch M, Foy R, van der Weijden T, Eccles M, Grol R. Factors in theories on behaviour change to guide implementation and quality improvement in health care. Nijmegen: Centre for Quality of Care Research (WOK); 2005.

Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud PA, et al. Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999;282(15):1458–65. https://doi.org/10.1001/jama.282.15.1458 .

Mäkelä M, Thorsen T. A framework for guidelines implementation studies. In: Thorsen T, Mäkelä M, editors. Changing professional practice: theory and practice of clinical guidelines implementation. 1st ed. Copenhagen: DSI - Danish Institute for Health Services Research; 1999. p. 23–53.

Google Scholar  

Grol R, Wensing M. What drives change? Barriers to and incentives for achieving evidence-based practice. Med J Aust. 2004;180(S6):S57-60. https://doi.org/10.5694/j.1326-5377.2004.tb05948.x .

Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A, et al. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005;14(1):26–33. https://doi.org/10.1136/qshc.2004.011155 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Rainbird K, Sanson-Fisher R, Buchan H. Identifying barriers to evidence uptake. Melbourne: National Institute of Clinical Studies (NICS); 2006.

Flottorp SA, Oxman AD, Krause J, Musila NR, Wensing M, Godycki-Cwirko M, et al. A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci. 2013;8:35. https://doi.org/10.1186/1748-5908-8-35 .

Cooper J, Murphy J, Woods C, Van Nassau F, McGrath A, Callaghan D, et al. Barriers and facilitators to implementing community-based physical activity interventions: a qualitative systematic review. Int J Behav Nutr Phys Act. 2021;18(1):118. https://doi.org/10.1186/s12966-021-01177-w .

Manta SW, Sandreschi PF, dos Santos MC, Konrad LM, Tassitano RM, Benedetti TRB. Barriers and facilitators on the implementation of physical activity in Primary Health Care: a systematic review. Prev Med Rep. 2022;28:101875. https://doi.org/10.1016/j.pmedr.2022.101875 .

Albert FA, Crowe MJ, Malau-Aduli AEO, Malau-Aduli BS. Physical activity promotion: a systematic review of the perceptions of healthcare professionals. Int J Environ Res Public Health. 2020;17(12). https://doi.org/10.3390/ijerph17124358 .

Huijg JM, Gebhardt WA, Verheijden MW, van der Zouwe N, de Vries JD, Middelkoop BJ, et al. Factors influencing primary health care professionals’ physical activity promotion behaviors: a systematic review. Int J Behav Med. 2015;22(1):32–50. https://doi.org/10.1007/s12529-014-9398-2 .

Hébert ET, Caughy MO, Shuval K. Primary care providers’ perceptions of physical activity counselling in a clinical setting: a systematic review. Br J Sports Med. 2012;46(9):625–31. https://doi.org/10.1136/bjsports-2011-090734 .

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. https://doi.org/10.31222/osf.io/jb4dx .

Critical Appaisal Tools. Joanna Briggs Institute. Available online: https://jbi.global/critical-appraisal-tools (Accessed on 22 June 2022).

Hong Q, Pluye P, Fàbregues S, Bartlett G, Boardman F, Cargo M, et al. Mixed Methods Appraisal Tool (MMAT), version 2018. Registration of Copyright (#1148552), Canadian Intellectual Property Office, Industry Canada. 2018. Available online: http://mixedmethodsappraisaltoolpublic.pbworks.com/w/file/fetch/127916259/MMAT_2018_criteria-manual_2018-08-01_ENG.pdf . Accessed 5 May 2022.

Tod D, Booth A, Smith B. Critical appraisal. Int Rev Sport Exerc Psychol. 2022;15(1):52–72. https://doi.org/10.1080/1750984x.2021.1952471 .

Article   Google Scholar  

Clarke V, Braun V, Hayfield N. Thematic analysis. In: Smith JA, editor. Qualitative psychology: a practical guide to research methods. London: SAGE Publications; 2015. p. 222–48.

Supporting the Use of Research Evidence (SURE) Collaboration. SURE checklist for making judgements about how much confidence to place in a systematic review. In: SURE guides for preparing and using evidence-based policy briefs: 4. Deciding on and describing policy options – Additional resources. Version 2.1 [updated November 2011]. Available online: https://epoc.cochrane.org/sites/epoc.cochrane.org/files/public/uploads/SURE-Guides-v2.1/Collectedfiles/sure_guides.html . Accessed 28 Mar 2023.

Kyei-Frimpong J, Blood-Siegfried J, Wijetilaka R, Gendler A. Exercise as medicine: providing practitioner guidance on exercise prescription. Prev Med Rep. 2021;22:101323. https://doi.org/10.1016/j.pmedr.2021.101323 .

Syrjälä MB, Fharm E, Dempsey PC, Nordendahl M, Wennberg P. Reducing occupational sitting time in adults with type 2 diabetes: qualitative experiences of an office-adapted mHealth intervention. Diabet Med. 2021;38(6):e14514. https://doi.org/10.1111/dme.14514 .

Charles M, Ouchchane L, Thivel D, Celine L, Duclos M. Does legislative framework favors prescription of physical activity in primary care? The French experience. Phys Sportsmed. 2022;50(1):47–53. https://doi.org/10.1080/00913847.2020.1864676 .

Souza-Neto JM, Florindo AA, Costa FFD. Associated factors with physical activity counseling among Brazilian Family Health Strategy workers. Cien Saude Colet. 2021;26(1):369–78. https://doi.org/10.1590/1413-81232020261.10082019 .

Huntington J, Dwyer JJM, Shama S, Brauer P. Registered dietitians’ beliefs and behaviours related to counselling patients on physical activity and sedentary behaviour from a theory of planned behaviour perspective. BMC Nutr. 2020;6(1):66. https://doi.org/10.1186/s40795-020-00392-1 .

Buckley BJR, Finnie SJ, Murphy RC, Watson PM. “You’ve got to pick your battles”: a mixed-methods investigation of physical activity counselling and referral within general practice. Int J Environ Res Public Health. 2020;17(20). https://doi.org/10.3390/ijerph17207428 .

Sissons A, Grant A, Kirkland A, Currie S. Using the theoretical domains framework to explore primary health care practitioner’s perspectives and experiences of preconception physical activity guidance and promotion. Psychol Health Med. 2020;25(7):844–54. https://doi.org/10.1080/13548506.2019.1679846 .

Carstairs SA, Rogowsky RH, Cunningham KB, Sullivan F, Ozakinci G. Connecting primary care patients to community-based physical activity: a qualitative study of health professional and patient views. BJGP Open. 2020;4(3). https://doi.org/10.3399/bjgpopen20x101100 .

Alghafri TS, Al Harthi SM, Al-Ajmi F, Al-Farsi Y, Craigie AM, Bannerman E, et al. Acceptability of the “MOVEdiabetes” physical activity intervention in diabetes primary care settings in Oman: findings from participants and practitioners. BMC Public Health. 2020;20(1):887. https://doi.org/10.1186/s12889-020-09029-1 .

Parjanen S. The elements of cross-sectoral collaboration between primary care and the sports sector. Manag Sport Leis. 2021;26(3):226–44. https://doi.org/10.1080/23750472.2020.1757492 .

Yonis OB, Saadeh R, Chamseddin Z, Alananzeh H. Exercise counseling by primary care physicians in Jordan - a preliminary study. J Prim Care Community Health. 2020;11:2150132720946947. https://doi.org/10.1177/2150132720946947 .

Agadayı E, Karasimav Ö, Sencan İ, Agadayı S, Kasım İ, Kahveci R, et al. Knowledge and attitude relating to exercise prescription of family medicine research assistant and specialist physicians who are working in Ankara. Gulhane Med J. 2019;61(4):152. https://doi.org/10.26657/gulhane.00075 .

Joelsson M, Lundqvist S, Larsson MEH. Tailored physical activity on prescription with follow-ups improved motivation and physical activity levels. A qualitative study of a 5-year Swedish primary care intervention. Scand J Prim Health Care. 2020;38(4):399–410. https://doi.org/10.1080/02813432.2020.1842965 .

Christiansen M, White D, Christian J, Waugh E, Gakhal N, King L, et al. “It... Doesn’t always make it [to] the top of the list” Primary care physicians’ experiences with prescribing exercise for knee osteoarthritis. Can Fam Physician. 2020;66(1):E14-E20.

Bird EL, Biddle MSY, Powell JE. General practice referral of “at risk” populations to community leisure services: applying the RE-AIM framework to evaluate the impact of a community-based physical activity programme for inactive adults with long-term conditions. BMC Public Health. 2019;19(1):1308. https://doi.org/10.1186/s12889-019-7701-5 .

Alahmed Z, Lobelo F. Correlates of physical activity counseling provided by physicians: a cross-sectional study in Eastern Province, Saudi Arabia. PLoS One. 2019;14(7):e0220396. https://doi.org/10.1371/journal.pone.0220396 .

Westland H, Koop Y, Schröder CD, Schuurmans MJ, Slabbers P, Trappenburg JCA, et al. Nurses’ perceptions towards the delivery and feasibility of a behaviour change intervention to enhance physical activity in patients at risk for cardiovascular disease in primary care: a qualitative study. BMC Fam Pract. 2018;19(1):194. https://doi.org/10.1186/s12875-018-0888-1 .

Harris T, Kerry S, Victor C, Iliffe S, Ussher M, Fox-Rushby J, et al. A pedometer-based walking intervention in 45- to 75-year-olds, with and without practice nurse support: the PACE-UP three-arm cluster RCT. Health Technol Assess. 2018;22(37):1–274. https://doi.org/10.3310/hta22370 .

Leenaars KEF, Smit E, Wagemakers A, Molleman GRM, Koelen MA. The role of the care sport connector in the Netherlands. Health Promot Int. 2018;33(3):422–35. https://doi.org/10.1093/heapro/daw097 .

Baillot A, Baillargeon JP, Paré A, Poder TG, Brown C, Langlois MF. Physical activity assessment and counseling in Quebec family medicine groups. Can Fam Physician. 2018;64(5):e234–41.

PubMed   PubMed Central   Google Scholar  

Gustavsson C, Nordqvist M, Bröms K, Jerdén L, Kallings LV, Wallin L. What is required to facilitate implementation of Swedish physical activity on prescription? - interview study with primary healthcare staff and management. BMC Health Serv Res. 2018;18(1):196. https://doi.org/10.1186/s12913-018-3021-1 .

Al-Ghamdi S, Alajmi M, Al-Gonaim A, Al-Juhayyim S, Al-Qasem S, Al-Tamimi I. Perceptions and attitudes of primary healthcare providers in Riyadh City, Saudi Arabia, toward the promotion of physical activity. Int J Health Promot Educ. 2018;56(2):105–19. https://doi.org/10.1080/14635240.2018.1430601 .

Fowles JR, O’Brien MW, Solmundson K, Oh PI, Shields CA. Exercise is Medicine Canada physical activity counselling and exercise prescription training improves counselling, prescription, and referral practices among physicians across Canada. Appl Physiol Nutr Metab. 2018;43(5):535–9. https://doi.org/10.1139/apnm-2017-0763 .

Henderson HE, Evans AB, Allen-Collinson J, Siriwardena NA. The ‘wild and woolly’ world of exercise referral schemes: contested interpretations of an exercise as medicine programme. Qual Res Sport Exerc Health. 2018;10(4):505–23. https://doi.org/10.1080/2159676x.2017.1352018 .

Johnson NA, Ewald B, Plotnikoff RC, Stacey FG, Brown WJ, Jones M, et al. Predictors of adherence to a physical activity counseling intervention delivered by exercise physiologists: secondary analysis of the NewCOACH trial data. Patient Prefer Adherence. 2018;12:2537–43. https://doi.org/10.2147/ppa.s183938 .

Alghafri TS, Alharthi SM, Al-Balushi S, Al-Farsi Y, Al-Busaidi Z, Bannerman E, et al. Health professionals' perceptions about physical activity promotion in diabetes care within primary health care settings in Oman. Heliyon. 2017;3(12):e00495. https://doi.org/10.1016/j.heliyon.2017.e00495 .

Bélanger M, Phillips EW, O'Rielly C, Mallet B, Aubé S, Doucet M, et al. Longitudinal qualitative study describing family physicians’ experiences with attempting to integrate physical activity prescriptions in their practice: 'it's not easy to change habits’. BMJ Open. 2017;7(7):e017265. https://doi.org/10.1136/bmjopen-2017-017265 .

Cottrell E, Foster NE, Porcheret M, Rathod T, Roddy E. GPs’ attitudes, beliefs and behaviours regarding exercise for chronic knee pain: a questionnaire survey. BMJ Open. 2017;7(6):e014999. https://doi.org/10.1136/bmjopen-2016-014999 .

Barrett EM, Hussey J, Darker CD. Feasibility of a physical activity pathway for Irish primary care physiotherapy services. Physiotherapy. 2017;103(1):106–12. https://doi.org/10.1016/j.physio.2016.02.001 .

Matthews A, Jones N, Thomas A, van den Berg P, Foster C. An education programme influencing health professionals to recommend exercise to their type 2 diabetes patients - understanding the processes: a case study from Oxfordshire, UK. BMC Health Serv Res. 2017;17(1):130. https://doi.org/10.1186/s12913-017-2040-7 .

O’Brien MW, Shields CA, Oh PI, Fowles JR. Health care provider confidence and exercise prescription practices of Exercise is Medicine Canada workshop attendees. Appl Physiol Nutr Metab. 2017;42(4):384–90. https://doi.org/10.1139/apnm-2016-0413 .

Leavitt PT. Improving exercise prescribing in a rural New England free clinic. J Nurse Pract. 2017;13(1):e29–33. https://doi.org/10.1016/j.nurpra.2016.09.014 .

Hidalgo KD, Mielke GI, Parra DC, Lobelo F, Simões EJ, Gomes GO, et al. Health promoting practices and personal lifestyle behaviors of Brazilian health professionals. BMC Public Health. 2016;16(1):1114. https://doi.org/10.1186/s12889-016-3778-2 .

Aittasalo M, Kukkonen-Harjula K, Toropainen E, Rinne M, Tokola K, Vasankari T. Developing physical activity counselling in primary care through participatory action approach. BMC Fam Pract. 2016;17(1):141. https://doi.org/10.1186/s12875-016-0540-x .

Dutton SN, Dennis SM, Zwar N, Harris MF. An explorative qualitative study on acceptability of physical activity assessment instruments among primary care professionals in southern Sydney. BMC Fam Pract. 2016;17(1):138. https://doi.org/10.1186/s12875-016-0536-6 .

Leenaars KE, Florisson AM, Smit E, Wagemakers A, Molleman GR, Koelen MA. The connection between the primary care and the physical activity sector: professionals’ perceptions. BMC Public Health. 2016;16(1):1001. https://doi.org/10.1186/s12889-016-3665-x .

Avery L, Charman SJ, Taylor L, Flynn D, Mosely K, Speight J, et al. Systematic development of a theory-informed multifaceted behavioural intervention to increase physical activity of adults with type 2 diabetes in routine primary care: Movement as Medicine for type 2 diabetes. Implement Sci. 2016;11:99. https://doi.org/10.1186/s13012-016-0459-6 .

Clark RE, Milligan J, Ashe MC, Faulkner G, Canfield C, Funnell L, et al. A patient-oriented approach to the development of a primary care physical activity screen for embedding into electronic medical records. Appl Physiol Nutr Metab. 2021;46(6):589–96. https://doi.org/10.1139/apnm-2020-0356 .

Attwood S, Morton KL, Mitchell J, Van Emmenis M, Sutton S, Team VP. Reasons for non-participation in a primary care-based physical activity trial: a qualitative study. BMJ Open. 2016;6(5):e011577. https://doi.org/10.1136/bmjopen-2016-011577 .

Taylor AH, Taylor RS, Ingram WM, Anokye N, Dean S, Jolly K, et al. Adding web-based behavioural support to exercise referral schemes for inactive adults with chronic health conditions: the e-coachER RCT. Health Technol Assess. 2020;24(63):1–106. https://doi.org/10.3310/hta24630 .

Hefnawi B, Leung L, Tomasone JR. Exploring barriers medical residents and established physicians face counselling patients on physical activity by stage of the transtheoretical model. Psychol Health Med. 2021;26(6):684–91. https://doi.org/10.1080/13548506.2020.1754437 .

Hanson CL, Oliver EJ, Dodd-Reynolds CJ, Allin LJ. How do participant experiences and characteristics influence engagement in exercise referral? A qualitative longitudinal study of a scheme in Northumberland, UK. BMJ Open. 2019;9(2):e024370. https://doi.org/10.1136/bmjopen-2018-024370 .

Wattanapisit A, Thanamee S, Wongsiri S. Physical activity counselling among GPs: a qualitative study from Thailand. BMC Fam Pract. 2019;20(1):72. https://doi.org/10.1186/s12875-019-0968-x .

Omura JD, Bellissimo MP, Watson KB, Loustalot F, Fulton JE, Carlson SA. Primary care providers’ physical activity counseling and referral practices and barriers for cardiovascular disease prevention. Prev Med. 2018;108:115–22. https://doi.org/10.1016/j.ypmed.2017.12.030 .

Lewis ZH, Ottenbacher KJ, Fisher SR, Jennings K, Brown AF, Swartz MC, et al. The feasibility and RE-AIM evaluation of the TAME health pilot study. Int J Behav Nutr Phys Act. 2017;14(1):106. https://doi.org/10.1186/s12966-017-0560-5 .

Brandborg CE, Skjerning HT, Nielsen RO. Physical activity through social prescribing: an interview-based study of Danish general practitioners’ opinions. Health Soc Care Community. 2022;30(5):1969–78. https://doi.org/10.1111/hsc.13577 .

Bowen PG, William OA, Olivia A, Levi P, Wingo N. Physical activity promotion in a safety-net clinic: does the provider make a difference? A pilot study. Transl J Am Coll Sports Med. 2021;6(4). https://doi.org/10.1249/tjx.0000000000000175 .

Downey J, Shearn K, Brown N, Wadey R, Breckon J. Behaviour change practices in exercise referral schemes: developing realist programme theory of implementation. BMC Health Serv Res. 2021;21(1):335. https://doi.org/10.1186/s12913-021-06349-9 .

Huebschmann AG, Glasgow RE, Leavitt IM, Chapman K, Rice JD, Lockhart S, et al. Integrating a physical activity coaching intervention into diabetes care: a mixed-methods evaluation of a pilot pragmatic trial. Transl Behav Med. 2022;12(4):601–10. https://doi.org/10.1093/tbm/ibac014 .

Dranebois S, Lalanne-Mistrih ML, Nacher M, Thelusme L, Deungoue S, Demar M, et al. Prescription of physical activity by general practitioners in type 2 diabetes: practice and barriers in French Guiana. Front Endocrinol (Lausanne). 2022;12:790326. https://doi.org/10.3389/fendo.2021.790326 .

Morgan K, Lewis J, Hawkins J, Moore G. From a research trial to routine practice: stakeholders’ perceptions and experiences of referrals to the National Exercise Referral Scheme (NERS) in Wales. BMC Health Serv Res. 2021;21(1):1232. https://doi.org/10.1186/s12913-021-07266-7 .

Albert FA, Malau-Aduli AEO, Crowe MJ, Malau-Aduli BS. The 'PRICE' of Physical Activity Referral Schemes (PARS): stakeholders' recommendations for delivering quality care to patients. Int J Environ Res Public Health. 2021;18(16). https://doi.org/10.3390/ijerph18168627 .

Wattanapisit A, Amaek W, Wattanapisit S, Tuangratananon T, Wongsiri S, Pengkaew P. Challenges of implementing an mHealth application for personalized physical activity counselling in primary health care: a qualitative study. Int J Gen Med. 2021;14:3821–31. https://doi.org/10.2147/ijgm.s317241 .

Calonge-Pascual S, Fuentes Jiménez F, Arnal-Selfa R, Belmonte-Cortés S, González-Gross M. Self-perception of primary health-care staff about physical activity on prescription: a qualitative semi-structured interview. Semergen. 2023;49(1):101856. https://doi.org/10.1016/j.semerg.2022.101856 .

Wangler J, Jansky M. Promotion of exercise and health for older people in primary care: a qualitative study on the potential, experiences and strategies of general practitioners in Germany. J Prev. 2023. https://doi.org/10.1007/s10935-023-00730-6 .

Buckley BJR, Newton J, Knox S, Noonan B, Smith M, Watson PM. Multi-stakeholder perspectives on co-production: five key recommendations following the Liverpool Co-PARS project. Qual Res Sport Exerc Health. 2023;15(2):220–34. https://doi.org/10.1080/2159676x.2022.2152085 .

Alyafei M, Alchawa M, Farooq A, Selim N, Bougmiza I. Physical activity among primary health care physicians and its impact on counseling practices. Biomed Hub. 2023;8(1):31–41. https://doi.org/10.1159/000530085 .

De Guzman KR, Pratt M, Hwang A, Linke SE. Patient feedback and evaluation measures of a physical activity initiative: Exercise is Medicine program. Fam Pract. 2022;39(5):813–8. https://doi.org/10.1093/fampra/cmab178 .

Moraes SQ, Santos ACB, Fermino RC, Rech CR. Physical activity counseling in primary health care. Cien Saude Colet. 2022;27(9):3603–14. https://doi.org/10.1590/1413-81232022279.20192021en .

Snége A, Silva AAPD, Mielke GI, Rech CR, Siqueira FCV, Rodriguez-Añez CR, et al. Sedentary behavior counseling received from healthcare professionals: an exploratory analysis in adults at primary health care in Brazil. Int J Environ Res Public Health. 2022;19(16). https://doi.org/10.3390/ijerph19169963 .

Pellerine LP, O'Brien MW, Shields CA, Crowell SJ, Strang R, Fowles JR. Health care providers' perspectives on promoting physical activity and exercise in health care. Int J Environ Res Public Health. 2022;19(15). https://doi.org/10.3390/ijerph19159466 .

Albert FA, Malau-Aduli AEO, Crowe MJ, Malau-Aduli BS. Optimising care coordination strategies for physical activity referral scheme patients by Australian health professionals. PLoS One. 2022;17(7):e0270408. https://doi.org/10.1371/journal.pone.0270408 .

Countries and economies. The World Bank. Available online: https://data.worldbank.org/country (Accessed 9 Sept 2022).

AuYoung M, Linke SE, Pagoto S, Buman MP, Craft LL, Richardson CR, et al. Integrating physical activity in primary care practice. Am J Med. 2016;129(10):1022–9. https://doi.org/10.1016/j.amjmed.2016.02.008 .

Silva CS, Mendes R, Godinho C, Monteiro-Pereira A, Pimenta-Ribeiro J, Martins HS, et al. Predictors of physical activity promotion in clinical practice: a cross-sectional study among medical doctors. BMC Med Educ. 2022;22(1):624. https://doi.org/10.1186/s12909-022-03686-z .

Dacey ML, Kennedy MA, Polak R, Phillips EM. Physical activity counseling in medical school education: a systematic review. Med Educ Online. 2014;19:24325. https://doi.org/10.3402/meo.v19.24325 .

Cardinal BJ, Park EA, Kim M, Cardinal MK. If exercise is medicine, where is exercise in medicine? Review of U.S. medical education curricula for physical activity-related content. J Phys Act Health. 2015;12(9):1336–43. https://doi.org/10.1123/jpah.2014-0316 .

Strong A, Stoutenberg M, Hobson-Powell A, Hargreaves M, Beeler H, Stamatakis E. An evaluation of physical activity training in Australian medical school curricula. J Sci Med Sport. 2017;20(6):534–8. https://doi.org/10.1016/j.jsams.2016.10.011 .

Osborne SA, Adams JM, Fawkner S, Kelly P, Murray AD, Oliver CW. Tomorrow’s doctors want more teaching and training on physical activity for health. Br J Sports Med. 2017;51(8):624–5. https://doi.org/10.1136/bjsports-2016-096807 .

Mendes R, Silva MN, Silva CS, Marques A, Godinho C, Tomás R, et al. Physical activity promotion tools in the Portuguese primary health care: an implementation research. Int J Environ Res Public Health. 2020;17(3). https://doi.org/10.3390/ijerph17030815 .

World Health Organization. Tackling NCDs: ‘Best buys’ and other recommended interventions for the prevention and control of non-communicable diseases. Geneva: World Health Organization; 2017.

Mitchell MS, Orstad SL, Biswas A, Oh PI, Jay M, Pakosh MT, et al. Financial incentives for physical activity in adults: systematic review and meta-analysis. Br J Sports Med. 2020;54(21):1259–68. https://doi.org/10.1136/bjsports-2019-100633 .

World Health Organization. 2021 Physical activity factsheets for the European Union member states in the WHO european region. Copenhagen: WHO Regional Office for Europe; 2021. License: CC BY-NC-SA 3.0 IGO.

Ward M. Health-Enhancing Physical Activity (HEPA) promotion in health care settings - policy, practice & evidence. Copenhagen: WHO Regional Office for Europe; 2016.

Lobelo F, Young DR, Sallis R, Garber MD, Billinger SA, Duperly J, et al. Routine assessment and promotion of physical activity in healthcare settings: a scientific statement from the American Heart Association. Circulation. 2018;137(18):e495–522. https://doi.org/10.1161/cir.0000000000000559 .

Skivington K, Matthews L, Simpson SA, Craig P, Baird J, Blazeby JM, et al. A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance. BMJ. 2021;374:n2061. https://doi.org/10.1136/bmj.n2061 .

Sallis RE, Baggish AL, Franklin BA, Whitehead JR. The call for a physical activity vital sign in clinical practice. Am J Med. 2016;129(9):903–5. https://doi.org/10.1016/j.amjmed.2016.05.005 .

Koorts H, Eakin E, Estabrooks P, Timperio A, Salmon J, Bauman A. Implementation and scale up of population physical activity interventions for clinical and community settings: the PRACTIS guide. Int J Behav Nutr Phys Act. 2018;15(1):51. https://doi.org/10.1186/s12966-018-0678-0 .

Koorts H, Bauman A, Edwards N, Bellew W, Brown WJ, Duncan MJ, et al. Tensions and paradoxes of scaling up: a critical reflection on physical activity promotion. Int J Environ Res Public Health. 2022;19(21). https://doi.org/10.3390/ijerph192114284 .

Tabak RG, Khoong EC, Chambers DA, Brownson RC. Bridging research and practice: models for dissemination and implementation research. Am J Prev Med. 2012;43(3):337–50. https://doi.org/10.1016/j.amepre.2012.05.024 .

Download references

Acknowledgements

Not applicable

This research received no external funding. C.S.S. is supported by the Portuguese Foundation for Science and Technology (SFRH/BD/139603/2018). The funder had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Author information

Authors and affiliations.

CIPER, Faculdade de Motricidade Humana, Universidade de Lisboa, Lisboa, Portugal

Catarina Santos Silva, Jorge Encantado & Pedro J. Teixeira

Programa Nacional Para a Promoção da Atividade Física, Direção-Geral da Saúde, Lisboa, Portugal

Catarina Santos Silva, Cristina Godinho & Marlene Nunes Silva

NOVA National School of Public Health, Public Health Research Centre, Comprehensive Health Research Center, CHRC, NOVA University Lisbon, Lisboa, Portugal

Cristina Godinho

CIAFEL, Faculdade de Desporto da Universidade Do Porto, Porto, Portugal

Bruno Rodrigues

CIDEFES, Centro de Investigação em Desporto, Educação Física, Exercício e Saúde, Universidade Lusófona, Lisboa, Portugal

Eliana V. Carraça & Marlene Nunes Silva

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization, CSS, MNS, CG, and PJT; Methodology, CSS, CG, JE, BR, EVC and MNS; Data Curation and formal analysis, CSS, JE, and BR; writing—original draft preparation, CSS, CG, and MNS; writing—review and editing, CSS, CG, JE, BR, EVC, PJT, and MNS; Supervision, CG, PJT, and MNS. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Catarina Santos Silva .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

PRISMA 2020 Checklist.

Additional file 2.

Search steam.

Additional file 3:

Critical appraisal of the included studies.

Additional file 4:

Codebook of the implementation determinants.

Additional file 5:

Detailed report of implementation determinants, with supporting extracted data.

Additional file 6.

Certainty assessment of the systematic review (SURE checklist).

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Silva, C.S., Godinho, C., Encantado, J. et al. Implementation determinants of physical activity interventions in primary health care settings using the TICD framework: a systematic review. BMC Health Serv Res 23 , 1082 (2023). https://doi.org/10.1186/s12913-023-09881-y

Download citation

Received : 09 June 2023

Accepted : 07 August 2023

Published : 11 October 2023

DOI : https://doi.org/10.1186/s12913-023-09881-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Determinants of implementation
• Facilitators
• Physical activity interventions
• Primary care

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

• Systematic Review
• Open access
• Published: 02 January 2023

Pain mechanisms in complex regional pain syndrome: a systematic review and meta-analysis of quantitative sensory testing outcomes

• Mohamed Gomaa Sobeeh 1 , 2 ,
• Karima Abdelaty Hassan 1 ,
• Anabela Gonçalves da Silva 3 ,
• Enas Fawzy Youssef 1 ,
• Nadia Abdelazim Fayaz 1 &
• Maha Mostafa Mohammed 1  

Journal of Orthopaedic Surgery and Research volume  18 , Article number:  2 ( 2023 ) Cite this article

3706 Accesses

5 Citations

1 Altmetric

Metrics details

Complex regional pain syndrome (CRPS) is a chronic condition following inciting events such as fractures or surgeries with sensorimotor and autonomic manifestations and poor prognosis. This review aimed to provide conclusive evidence about the sensory phenotype of CRPS based on quantitative sensory testing (QST) to understand the underlying pain mechanisms and guide treatment strategies.

Eight databases were searched based on a previously published protocol. Forty studies comparing QST outcomes (thermal, mechanical, vibration, and electric detection thresholds, thermal, mechanical, pressure, and electric pain thresholds, wind-up ratio, mechanical pain sensitivity, allodynia, flare area, area after pinprick hyperalgesia, pleasantness after C-tactile stimulation, and pain ratings) in chronic CRPS (adults and children) versus healthy controls were included.

From 37 studies (14 of low quality, 22 of fair quality, and 1 of good quality), adults with CRPS showed: (i) significant loss of thermal, mechanical, and vibration sensations, significant gain of thermal and mechanical pain thresholds, significant elevation of pain ratings, and no difference in wind-up ratio; (ii) significant reduction of pleasantness levels and increased area of pinprick hyperalgesia, in the affected limb. From three fair-quality studies, adolescents and children with CRPS showed loss of cold detection with cold hyperalgesia in the affected limb. There was moderate to substantial overall heterogeneity.

Diffuse thermal and mechanical hypoesthesia with primary and secondary hyperalgesia, enhanced pain facilitation evidenced by increased area of pinprick hyperalgesia, and elevated pain ratings are dominant in adults with CRPS. Adolescents and children with CRPS showed less severe sensory abnormalities.

Introduction

Complex regional pain syndrome (CRPS) is a chronic debilitating pain condition of the limbs following trauma or surgery with an incidence rate of 26.2 per 100,000 person-years [ 1 , 2 ] . CRPS occurs commonly in elderly people, in females more than males, and the upper extremity more than in the lower extremity [ 2 ]. Two main types of CRPS were identified: CRPS types 1 and 2 [ 3 ]. CRPS type 1 or reflex sympathetic dystrophy is characterized by sensory, motor, and autonomic abnormalities without electrophysiological evidence of nerve lesion. On contrary, CRPS type 2 is characterized by identifiable nerve lesions that can be detected through electrophysiological findings and it is considered typical neuropathic pain [ 1 ].

CRPS is, usually, associated with poor outcomes, long-term complaints, and comorbidities (e.g., depression and photophobia) [ 4 , 5 , 6 ]; however, the pain mechanisms involved in CRPS are not fully understood. [ 7 ]. Neurogenic inflammation, peripheral sensitization (PS), central sensitization (CS), small nerve fiber pathology, autonomic dysregulation, and psychological states represent the shared model of the underlying pathophysiology of CRPS [ 8 , 9 , 10 , 11 , 12 ]. Neurogenic inflammation is caused by neuropeptides released from the primary afferents resulting in axon reflex vasodilatation and protein extravasation [ 8 ]. PS is defined as enhanced responsiveness and decreased threshold of nociceptive neurons within the afflicted receptive field, and it was demonstrated in CRPS by the presence of primary hyperalgesia in the affected regions [ 13 ]. Signs of PS in CRPS can include gain of thermal and mechanical pain thresholds at the affected sites [ 14 , 15 , 16 ] .

In CRPS, secondary hyperalgesia in distant locations away from the affected area was found to be indicative of CS, which is an increased response of nociceptive neurons in the central nervous system to normal or sub-threshold afferent input [ 17 ]. Signs of CS in CRPS can include widespread gain of thermal and mechanical pain thresholds, enhanced pain facilitation as evidenced by elevated pain ratings, and/or impaired pain inhibition [ 14 , 18 ].

It has been demonstrated that CRPS patients have a bilateral reduction in intraepidermal small nerve fiber density, and these fibers are responsible for nociception and perceiving temperature [ 19 ]. Conceivably, reduction of the small nerve fiber density would be responsible for altered perception of these sensations. Autonomic dysregulation could result in enhanced pain perception as evidenced by increased expression of α1-adrenergic receptors [ 11 ]. Also, post-traumatic stress disorder and pain catastrophizing seem to increase pain response in CRPS [ 12 ].

A valid and standardized tool to assess pain mechanisms involved in different chronic pain conditions (inflammatory, neuropathic, and mixed chronic pain conditions) is quantitative sensory testing (QST) [ 20 ]. As far as we are aware, this is the first review to consolidate and evaluate the QST data of affected areas and remote areas away from the affected site in adults and children with CRPS type 1 compared to healthy controls. Additionally, we analyzed a broad range of variables including flare area after induction of noxious stimulus, pain area after pinprick induced hyperalgesia, pain ratings after noxious thermal stimulus, electric pain threshold, current perception thresholds, and pleasantness levels after C-tactile perception in an attempt to reach more conclusive results on the sensory profile and pain mechanisms of CRPS type 1.

Protocol registration

The review protocol was registered as an a priori study at the International Prospective Register of Systematic Reviews (PROSPERO) (registration number: CRD42021237157) and we used PRISMA guidelines ( www.prisma-statement.org ) to report this review.

Eligibility criteria

Studies were included if they (1) compared adults (age ≥ 18 years) or adolescents and children (age < 18 years) with CRPS type 1 (symptoms duration ≥ 8 weeks) to healthy controls, (2) diagnosed CRPS type 1 (unilateral or bilateral) through clinical assessment and the International Association for the Study of Pain (IASP) or the Budapest criteria, (3) investigated any modality of QST, flare areas after noxious stimulus, conditioned pain modulation, pain rating after noxious stimulus, and pain area after induced pinprick hyperalgesia, and (4) were written in English. We excluded studies that combined results of sensory testing of CRPS with other neuropathic conditions and studies that used the unaffected side as the control site. Additionally, we focused on the QST outcomes for CRPS type 1 only, which is a deviation from the previously published protocol. The protocol stated that both the QST outcomes for CRPS type 1 and type 2 would be included. However, a meta-analysis requires at least two studies, and we found one study only on CRPS type 2 that met the eligibility criteria [ 15 ]. Also, there is an identifiable nerve lesion in CRPS type 2 but not in CRPS type 1, which precludes including studies on CRPS type 2 and 1 in the same meta-analysis as that would prevent us from reaching a comprehensive understanding of the sensory profile and type of pain present in such a complex syndrome.

The main included parameters to study the sensory profile of CRPS type 1 were (1) detection thresholds including warm detection threshold (WDT), cold detection threshold (CDT), thermal sensory limen (TSL), vibration detection threshold (VDT), and mechanical detection threshold (MDT); (2) pain thresholds including heat pain threshold (HPT), cold pain threshold (CPT), pressure pain threshold (PPT), and mechanical pain threshold (MPT); (3) temporal summation or wind up ratio (WUR); (4) conditioned pain modulation (CPM); (5) mechanical pain sensitivity (MPS); (6) dynamic mechanical allodynia (DMA) ; (7) flare area; (8) pain area after pinprick induced hyperalgesia; (9) current perception threshold; (10) electric pain threshold; and (11) pain ratings after thermal and mechanical stimuli. The definition of each variable is included in Table 1 [ 21 , 22 , 23 , 24 ].

Search strategy and investigated databases

The main keywords of our search included complex regional pain syndrome, reflex sympathetic dystrophy, causalgia, central nervous system sensitization, hyperalgesia, quantitative sensory testing, conditioned pain modulation, hypoesthesia, wind-up ratio, mechanical hyperalgesia, temporal summation, thermal hyperalgesia, heat pain threshold, warm detection threshold, mechanical detection threshold, pressure pain threshold, allodynia, cold pain threshold, vibration detection threshold, cold detection threshold, mechanical pain sensitivity, mechanical pain threshold, thermal sensory limen, pain perception, electric pain threshold, current perception threshold, flare area, and laser Doppler imaging. Scopus, EMBASE, Web of Science, PubMed, EBSCO host , SAGE, Cochrane library, and ProQuest databases/search engines were searched from inception to January 2022 (Table 2 ). To identify other eligible articles, a manual search of references of the included studies was done.

Study selection

After removing duplicates, two independent researchers (M.G.S. and K.A.H) screened the titles and abstracts of the relevant retrieved articles. The same two researchers obtained the full-text versions of the relevant articles and assessed them against the eligibility criteria. Conflicts were solved by discussion until a consensus was reached.

Risk of bias assessment

Two researchers (M.G.S. and K.A.H) independently used the Newcastle–Ottawa quality assessment scale (NOS) for case–control and cohort studies to perform the risk of bias assessment. Three aspects were evaluated through the NOS using a star rating system: the selection of the study groups, the comparability of the groups, and the ascertainment of the exposure or outcome of interest. Each aspect contains several items that can be scored with one star, except for comparability, which can score up to two stars (Table 3 ) [ 25 ]. The highest possible NOS score is 9. According to Agency for Health Research and Quality (AHRQ) standards, studies were deemed to be of good quality if they received three or four stars in the selection domain, one or two stars in the comparability domain, and two or three stars in the outcome/exposure domain. Studies were deemed to be of fair quality if they received two stars in the selection domain, one or two stars in the comparability domain, and two or three stars in the outcome/exposure domain. Studies were deemed to be of low quality if they received a score of zero or one in the selection domain, zero star in the comparability domain, or zero or one star in the outcome/exposure domain. Researchers were blind to the study authors when performing the risk of bias assessment. Inter-rater agreement between the two researchers was calculated using non-weighted Kappa statistics and respective 95% confidence interval (CI). A third researcher (A.G.S) was contacted if consensus was not reached.

Data extraction

Data extracted from the included articles were: authors, year of publication, number of participants, diagnostic criteria for CRPS, type, and raw data of measurements (CPT, HPT, PPT, CDT, WDT, TSL, VDT, MDT, MPS, MPT, DMA, WUR, pain area after pinprick hyperalgesia, pain ratings, and CPM), body site where measurements were taken, pain intensity, and details of QST parameters and measurement procedures (including method, number of trials, and devices used) (Table 4 ). Data extraction was performed by one researcher (M.G.S.) and revised by another researcher (A.G.S.) to confirm the data were correctly gathered. Corresponding authors of the included studies were contacted if there were missing data.

Data management and meta-analysis

The raw data from individual articles were extracted (Table 4 ), grouped based on the applied measurements (CPT, HPT, PPT, CDT, WDT, TSL, VDT, MDT, MPS, MPT, DMA, WUR, pain area after pinprick hyperalgesia, pain ratings, and CPM), and further clustered according to age into: (1) patients with chronic CRPS type 1 ≥ 18 years and (2) patients with CRPS type 1 < 18 years. For each age group, the outcomes were clustered according to body location into (1) affected area and (2) remote areas away from the affected site. If a cluster of specific measurements contained at least two studies reporting means and standard deviations for patients with CRPS and healthy controls, a meta-analysis was performed [ 26 ].

Meta-analysis was conducted using the Review Manager computer program (RevMan 5.4) by Cochrane collaboration. The standardized mean difference (SMD) and the corresponding 95% CI were calculated based on inverse variance weighting [ 27 ]. SMD effect size values between 0.2 and 0.5 are regarded as small, 0.5 to 0.8 as medium, and values higher than 0.8 as large [ 28 ]. Egger’s regression test was conducted when there were 10 or more effect sizes to assess publication bias [ 29 , 30 ] and represented graphically by Begg’s funnel plot [ 31 ]. If the p value of Egger’s regression test was less than 0.10, it is considered significant. Whenever publication bias was found, we applied the trim and fill method of Duvall and Tweedie to enhance the symmetry through adding the studies supposed to be missed [ 32 ]. To assess the heterogeneity, I2 was measured and classified into: 0%–40%: no heterogeneity, 30%–60%: moderate, 50%–90%: substantial, and 75%–100%: considerable [ 33 ]. We determined the borderline I2 values based on the magnitude and direction of effects and the strength of evidence for heterogeneity. So, if there is 50% heterogeneity with a narrower confidence interval and a large effect size, the amount of heterogeneity becomes moderate, whereas heterogeneity is substantial with a wide confidence interval and a small effect size. [ 33 ].

The overall effect was significant if the p value was less than 0.05. Studies not included in the meta-analysis were reported separately. Sensitivity analyses were performed to account for the studies with high risk of bias based on the NOS assessment.

GRADE assessment was conducted to check for the certainty of obtained results [ 34 , 35 ]. One author checked the quality of the evidence considering five domains: (i) risk of bias, (ii) inconsistency of results, (iii) indirectness, (iv) imprecision, and (v) publication bias. At the baseline rating, the studies were considered “low-quality” evidence, due to the observational study design, and then, the rating was upgraded or downgraded the ratings based on the judgment for each of the five domains listed above. The overall quality rating of the evidence was classified as high, moderate, low, or very low evidence [ 34 , 35 ].

A few studies included median and interquartile ranges, and Wan’s method was used to convert this data into mean and SD [ 36 ]. Cochrane guidelines formula was used to convert CI and standard error of mean into SD to be added in the meta-analysis [ 37 ].

The search yielded 4918 articles identified through different databases, with 4 additional studies identified through manual search [ 38 , 39 , 40 , 41 ] . The flowchart of the systematic review is shown in Fig.  1 . The titles and abstracts of the remaining articles after removing duplicates were screened ( n  = 4001), and the full texts of 116 articles were read. Forty articles were included in this review [ 14 , 15 , 16 , 18 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 73 , 76 ] articles were excluded. Reasons for exclusion were: use of animal models (e.g., Ohmichi et al.’s study [ 74 ]), different experimental design (e.g., Drummond et al. study [ 75 ]), absence of a control group or of a group of individuals with CRPS (e.g., Vaneker et al. study [ 76 ]), or inability to obtain the full text (eight studies) . The corresponding authors of five publications were contacted requesting data for the meta-analysis [ 39 , 66 , 69 , 71 , 72 ]. Three authors replied and sent the required information [ 15 , 39 , 69 ].

PRISMA flow diagram

Study characteristics

Ten studies were included in the qualitative analysis based on z-scores [ 14 , 39 , 40 , 53 , 61 , 66 , 68 , 71 , 72 , 73 ], and the frequencies of gain and loss of sensations in CRPS were mentioned in six studies (Table 5 ) [ 14 , 15 , 44 , 53 , 65 , 69 ]. Twenty-six studies were included in the quantitative analysis. Two studies investigated the sensory profile of patients with CRPS accompanied by dystonia [ 50 , 70 ], and we included these results in the meta-analysis as we aimed to summarize the sensory profile and underlying pain mechanisms in individuals with CRPS in general. Two studies assessed the level of pleasantness after c-tactile touch perception in CRPS, and we included these results in the meta-analysis to illustrate the functionality of this specific type of C-fibers in CRPS [ 71 , 72 ].

Rooijen et al. reported the QST results for two groups of individuals with CRPS: one group with dystonia and one group without dystonia [ 50 ]. We included the results of both groups in our review. Huge et al. investigated the results of QST in acute and chronic CRPS, but we included only the results of the chronic group in our review [ 47 ]. Gierthmühlen et al. described the results of QST for two groups of CRPS (a group with type 1 and the other group with type 2), comparing them to the control group, while we added only the results of QST of CRPS type 1 to the quantitative analysis and after contacting the authors we got the reference values based on Magerl et al. [ 15 , 77 ]. Kemler et al. reported the results of QST for two groups of individuals with CRPS (one group with upper extremity CRPS and one group with lower extremity CRPS) [ 44 ]. We included the results of both groups in our meta-analysis. Thimineur et al. investigated pain ratings after the application of diluted ethanol on the tongue [ 57 ]. The results of this study were not included in the meta-analysis of pain ratings after noxious stimulus, because the methods used were very different from the methods used in the other studies. Mainka et al. and Terkelsen et al. [ 18 , 49 ] assessed both joint and muscle PPTs, which were included in a separate meta-analysis, one related to the muscle and the other to the joint PPTs, respectively.

Uçeyler et al. and Enax-krumova et al. [ 16 , 66 ] used the same cohort of patients with CRPS and controls. Thus, we added only the results of Uçeyler et al. in the quantitative analysis.

König et al. [ 40 ] investigated a subgroup of patients with CRPS that was previously investigated in König et al. [ 39 ]. Thus, only the results of König et al. [ 39 ] were used in our review.

Two studies investigated the pleasantness level after C-tactile touch perception using brush stroking with a velocity of 3 cm/s both at the affected and contralateral sides. This variable was included in our review, despite addressing a variable not reported in the study protocol, as pleasantness levels could expand our knowledge about the sensory profile and the underlying pain mechanisms in CRPS [ 71 , 72 ].

Studies that investigated endogenous pain modulation could not be used in the meta-analysis because of different methodological approaches [ 45 , 53 ]. One study used repetitive electrical stimuli [ 45 ], while the other study used a restricted CPM paradigm [ 53 ].

Risk of bias

Quality assessment of the included studies is represented in Table 3 , and Kappa statistics for agreement between the two reviewers was 0.76 (95% CI, 0.56–0.95), which is considered substantial agreement [ 78 ]. None of the 41 articles included in this review had a score above 7 points out of a maximum score of 9. Most of the included studies were of fair quality as the mean quality score was greater than 4. Only one study reported the non-response rate [ 18 ], and all studies had the same ascertainment for cases and controls.

Sensory profile of adult patients with CRPS

Cold detection threshold.

Seven studies (one with low quality and six with fair quality), including a total of 505 patients with CRPS, investigated CDT on the affected area [ 15 , 43 , 44 , 47 , 50 , 67 , 70 ] and showed a significant loss of cold detection sensation with moderate heterogeneity (Additional file 1 : Fig. S1) (Table 6 ). Furthermore, there was symmetry in the funnel plot of included effect sizes (Additional file 2 : Fig. S2).

Six studies (one with low quality and five with fair quality), including a total of 245 patients with CRPS, investigated CDT [ 43 , 44 , 47 , 51 , 67 , 70 ] in areas remote from the affected area showing a significant loss of cold sensation with moderate heterogeneity (Additional file 3 : Fig. S3) (Table 6 ). Also, there was no significant publication bias ( p  = 0.9) (Additional file 4 : Fig. S4).

Seven studies (two with low quality and five with fair quality) using z-scores to investigate CDT showed loss of cold sensation on the affected side [ 39 , 47 , 66 , 68 , 71 , 72 , 73 ], and two studies (one with low quality and one with fair quality) showed loss of cold sensation on the contralateral limb [ 39 , 47 ]. One study of fair quality showed no between-group difference [ 14 ].

According to the GRADE assessment, there was low-quality evidence suggesting loss of the cold sensation in patients with CRPS, either at the affected site or the remote areas away from the affected site (Table 6 ).

Warm detection threshold

The meta-analysis of seven studies (one with low quality and six with fair quality) including a total of 505 CRPS patients (Additional file 5 : Fig. S5) (Table 6 ) [ 15 , 43 , 44 , 47 , 50 , 67 , 70 ] showed a significant loss of warm sensation on the affected site, with moderate heterogeneity. Furthermore, there was symmetry in the funnel plot of included effect sizes (Additional file 6 : Fig. S6).

The meta-analysis of six studies (one with low quality and five with fair quality) including a total of 245 CRPS patients for areas remote from the affected area (Additional file 7 : Fig. S7) (Table 6 ) [ 43 , 44 , 47 , 51 , 67 , 70 ] showed a significant loss of warm sensation, with moderate heterogeneity. Also, there was no significant publication bias ( p  = 0.14) (Additional file 8 : Fig. S8).

Nine studies (two with low quality and seven with fair quality) using z-scores showed loss of warm sensation at the affected side [ 14 , 39 , 47 , 53 , 66 , 68 , 71 , 72 , 73 ], and two studies (one with low quality and one with fair quality) showed loss of warm sensation on the contralateral limb [ 39 , 47 ].

According to the GRADE assessment, there was low-quality evidence suggesting loss warm sensations in patients with CRPS, either at the affected site or the remote areas away from the affected site (Table 6 ).

Thermal sensory limen

Four studies (one with low quality and three with fair quality) with a total of 659 patients with CRPS showed a significant loss of thermal sensations on the affected regions, with moderate heterogeneity (I2 = 65%; p  = 0.02) (Additional file 9 : Fig. S9) (Table 6 ) [ 15 , 47 , 57 , 67 ].

A meta-analysis of three studies (one with low quality and two with fair quality) with a total of 894 patients with CRPS for areas remote from the affected area showed a significant loss of thermal sensation, with moderate heterogeneity (Additional file 10 : Fig. S10) (Table 6 ) [ 47 , 57 , 67 ].

Eight studies (two with low quality and six with fair quality) using z-scores showed loss of thermal sensations at the affected side [ 39 , 47 , 53 , 66 , 68 , 71 , 72 , 73 ], and two studies (one with low quality and one with fair quality) showed loss of thermal sensations on the contralateral limb [ 39 , 47 ].

According to the GRADE assessment, there was low-quality evidence suggesting loss of thermal sensations in patients with CRPS, either at the affected site or the remote areas away from the affected side (Table 6 ).

Mechanical detection threshold

A meta-analysis of five studies (three with low quality and two with fair quality) including a total of 513 patients with CRPS showed a significant loss of mechanical detection sensation on the affected regions, without heterogeneity (Additional file 11 : Fig. S11) (Table 6 ) [ 15 , 44 , 45 , 52 , 57 ].

A meta-analysis of four studies (three with low quality and one with fair quality) with a total of 292 patients with CRPS showed a significant loss of mechanical detection sensation on the remote areas, without significant heterogeneity (Additional file 12 : Fig. S12) (Table 6 ) [ 44 , 45 , 52 , 57 ].

Four studies (one with low quality and three with fair quality) using z-scores showed loss of mechanical detection sensation in patients with CRPS [ 14 , 39 , 47 , 72 ], and three studies (one with low quality and two with fair quality) showed no between-group differences [ 66 , 68 , 73 ]. Two studies (one with low quality and one with fair quality) showed loss of mechanical detection sensation in the contralateral limb [ 39 , 47 ].

According to the GRADE assessment, there was low-quality evidence suggesting loss of mechanical detection sensations in patients either at the affected site or the remote areas away from the affected site (Table 6 ).

Vibration detection threshold

A meta-analysis of four studies of fair quality including a total of a total of 385 patients with CRPS showed a significant loss of vibration detection sensation on the affected regions, without significant heterogeneity (Additional file 13 : Fig. S13) (Table 6 ) [ 15 , 38 , 50 , 67 ].

A meta-analysis of three studies of fair quality including a total of 163 patients with CRPS reported a significant loss of vibration sensation on areas remote from the affected area, without significant heterogeneity (Additional file 14 : Fig. S14) (Table 6 ) [ 38 , 51 , 67 ].

Six studies (two with low quality and four with fair quality) using z-scores showed loss of vibration sensation on the affected side [ 39 , 47 , 66 , 68 , 72 , 73 ], one study of fair quality showed no between-group difference [ 14 ] , and two studies (one with low quality and one with fair quality) showed loss of vibration sensation on the contralateral side [ 39 , 47 ].

According to the GRADE assessment, there was moderate-quality evidence suggesting loss of vibration sensations in patients with CRPS, either at the affected site or the remote areas away from the affected site (Table 6 ).

Cold pain threshold

Seven studies (one with low quality, five with fair quality, and one with good quality) investigated CPT on the affected areas in 481 patients with CRPS showing significant gain of CPT compared to healthy controls, with substantial heterogeneity (Additional file 15 : Fig. S15) (Table 6 ) [ 15 , 18 , 43 , 44 , 47 , 50 , 67 ]. Furthermore, there was asymmetry in the funnel plot of included effect sizes (Additional file 16 : Fig. S16).

Meta-analysis of six studies (one with low quality, four with fair quality, and one with good quality) including a total of 240 patients with CRPS investigated CPT in areas remote from the affected area and showed a significant gain of CPT in CRPS compared to healthy controls, without significant heterogeneity (Additional file 17 : Fig. S17) (Table 6 ) [ 18 , 43 , 44 , 47 , 51 , 67 ]. There was also no publication bias ( p  = 0.5) (Additional file 18 : Fig. S18).

Six studies (one with low quality and five with fair quality) showed a sensory gain of CPT based on z-scores at the affected site of CRPS [ 39 , 47 , 53 , 68 , 71 , 72 ], while three studies (one with low quality and two with fair quality) showed no between-group differences [ 14 , 66 , 73 ] and two studies (one with low quality and one with fair quality) showed a gain of cold pain sensation on the contralateral side [ 39 , 47 ].

According to the GRADE assessment, there was low-quality evidence suggesting gain of cold pain thresholds in patients with CRPS at the affected site, but at remote areas, there was moderate-quality evidence (Table 6 ).

Heat pain threshold

A meta-analysis of nine studies (one with low quality, seven with fair quality, and one with good quality) including a total of 548 patients with CRPS showed a significant gain of HPT on the affected area of patients with CRPS, with moderate heterogeneity (Additional file 19 : Fig. S19) (Table 6 ) [ 15 , 18 , 43 , 44 , 47 , 50 , 62 , 67 , 70 ]. Furthermore, there was no significant publication bias ( p  = 0.60) (Additional file 20 : Fig. S20).

A meta-analysis of eight studies (one with low quality, six with fair quality, and one with good quality) including a total of 288 patients with CRPS reported a significant gain of HPT in areas remote from the affected area, without significant heterogeneity (Additional file 21 : Fig. S21) (Table 6 ) [ 18 , 43 , 44 , 47 , 51 , 62 , 67 , 70 ]. Also, there was no significant publication bias ( p  = 0.4) (Additional file 22 : Fig. S22).

Six studies (one with low quality and five with fair quality) showed a sensory gain of HPT on the affected site using z-scores [ 14 , 39 , 47 , 68 , 71 , 72 ], while two studies (one with low quality and one with fair quality) showed no differences [ 66 , 73 ] and two studies (one with low quality and one with fair quality) showed a gain of heat pain sensation on the contralateral side [ 39 , 47 ].

According to the GRADE assessment, there was moderate-quality evidence suggesting gain of heat pain thresholds in patients with CRPS, either at the affected site or the remote areas away from the affected site (Table 6 ).

Mechanical pain threshold

On the affected side, a meta-analysis of four studies (two with low quality and two with fair quality) including a total of 375 patients with CRPS reported a significant gain of MPT in patients with CRPS, with considerable heterogeneity (Additional file 23 : Fig. S23) (Table 6 ) [ 15 , 45 , 56 , 67 ].

On the remote areas, a meta-analysis of two studies (one with low quality and one with fair quality) with a total of 47 patients with CRPS and 34 healthy controls showed no group difference, without heterogeneity (Additional file 24 : Fig. S24) (Table 6 ) [ 45 , 67 ].

Based on z-scores, five studies (two of low quality and three of fair quality) showed a sensory gain of MPT on the affected site in patients with CRPS [ 39 , 47 , 68 , 72 , 73 ], while three studies of fair quality showed no between-group differences [ 14 , 66 , 71 ] and two studies (one of low quality and one of fair quality) showed a gain of MPT on the contralateral side [ 39 , 47 ].

According to the GRADE assessment, there was very low-quality evidence suggesting gain of mechanical pain thresholds in patients with CRPS at the affected site, but at remote areas, there was low-quality evidence suggesting that there was no difference (Table 6 ).

Pressure pain threshold

The meta-analysis of nine studies (three with low quality, five with fair quality, and one with good quality) with a total of 507 patients with CRPS showed a significant gain of muscle PPT on the affected site in CRPS, with moderate heterogeneity (Additional file 25 : Fig. S25) (Table 6 ) [ 15 , 18 , 38 , 48 , 49 , 50 , 52 , 63 , 67 ]. There was also no significant publication bias ( p  = 0.12) (Additional file 26 : Fig. S26).

On the remote areas, a meta-analysis of nine studies (four with low quality, four with fair quality, and one with good quality) investigating muscle PPT showed a significant gain of PPT in CRPS, with substantial heterogeneity (I2 = 84%; p  < 0.01) (Additional file 27 : Fig. S27) (Table 6 ) [ 18 , 38 , 49 , 51 , 52 , 54 , 57 , 63 , 67 ]. Also, there was a significant publication bias. After adjusting for publication bias, the PPT difference between CRPS and controls was increased (SMD, − 0.44; 95% CI, − 0.55, − 0.12), with no change in the significance level ( p  < 0.01); heterogeneity remained considerable (Additional file 28 : Fig. S28).

Eight studies (three with low quality and five with fair quality) using z-scores showed a gain of muscle PPT at the affected site of patients with CRPS [ 14 , 39 , 47 , 66 , 68 , 71 , 72 , 73 ], while at the contralateral side, one study of fair quality showed a gain of PPT in CRPS [ 47 ] and another one of low quality showed no difference [ 39 ]. Moreover, one study of fair quality showed a significant gain of PPT on the affected side and remote areas including face, chest, abdomen, and back [ 55 ] .

According to the GRADE assessment, there was low-quality evidence suggesting gain of pressure pain thresholds of the affected muscles in patients with CRPS, either at the affected site or the remote areas away from the affected site (Table 6 ).

A meta-analysis of two studies (one with low quality and one with good quality) investigating PPT on affected joints reported a significant gain of PPT in CRPS, without significant heterogeneity (Additional file 29 : Fig. S29) (Table 6 ) [ 18 , 49 ].

In the remote joints, a meta-analysis of two studies (one with low quality and one with good quality) reported no difference of PPT in CRPS, with considerable heterogeneity (Additional file 30 : Fig. S30) (Table 6 ) [ 18 , 49 ].

According to the GRADE assessment, there was moderate-quality evidence suggesting gain of pressure pain thresholds of the affected joints in patients with CRPS, but at remote joints, there was low-quality evidence suggesting that there was no difference (Table 6 ).

Mechanical pain sensitivity

The meta-analysis of five studies (two with low quality and three with fair quality) including a total of 396 patients with CRPS showed a significant elevation of MPS in CRPS, with moderate heterogeneity (Additional file 31 : Fig. S31) (Table 6 ) [ 15 , 56 , 62 , 63 , 67 ].

In the remote areas, a meta-analysis of three studies (one with low quality and two with fair quality) showed no difference, with substantial heterogeneity (Additional file 32 : Fig. S32) (Table 6 ) [ 62 , 63 , 67 ].

Five studies (one with low quality and four with fair quality) showed an elevated MPS on the affected site of patients with CRPS based on z-scores [ 39 , 47 , 68 , 71 , 72 ], while three studies (one with low quality and two with fair quality) showed no differences [ 14 , 66 , 73 ] and two studies (one with low quality and one with fair quality) showed elevated MPS on the contralateral side of CRPS [ 39 , 47 ].

According to the GRADE assessment, there was moderate-quality evidence suggesting enhanced mechanical pain sensitivity of the affected site in patients with CRPS, but at remote areas, there was very low-quality evidence suggesting that there was no difference (Table 6 ).

Wind-up ratio

A meta-analysis of five studies (one with low quality and four with fair quality) including a total of 374 patients with CRPS found no difference of WUR at the affected area, with moderate heterogeneity (Additional file 33 : Fig. S33) (Table 6 ) [ 15 , 50 , 56 , 62 , 67 ].

On the remote areas, a meta-analysis of two studies with fair quality investigated WUR in 37 patients with CRPS showed no difference, with moderate heterogeneity (Additional file 34 : Fig. S34) (Table 6 ) [ 62 , 67 ].

Based on z-scores, four studies (two with low quality and two with fair quality) showed no differences in WUR on the affected site [ 14 , 39 , 66 , 73 ] and one study of fair quality showed elevated WUR on the affected area in patients with CRPS [ 72 ].

According to the GRADE assessment, there was low-quality evidence suggesting that there was no difference between the levels of wind-up ratio, either at the affected site or the remote areas away from the affected site (Table 6 ).

Pain ratings after the noxious stimulus

A meta-analysis of five studies (three with low quality, one with fair quality, and one with good quality) reported a significant elevation of pain ratings in CRPS on the affected site, with substantial heterogeneity (Additional file 35 : Fig. S35) (Table 6 ) [ 18 , 42 , 43 , 45 , 56 ].

In the remote areas, a meta-analysis of four studies (two with low quality, one with fair quality, and one with good quality) reported a significant elevation of pain ratings in CRPS, without significant heterogeneity (Additional file 36 : Fig. S36) (Table 6 ) [ 18 , 42 , 43 , 45 ].

According to the GRADE assessment, there was low-quality evidence suggesting elevated pain ratings in patients with CRPS, either at the affected site or the remote areas away from the affected site (Table 6 ).

Area after pinprick hyperalgesia

Meta-analysis of two low-quality studies including a total of 47 patients with CRPS showed a significant increase in the area of hyperalgesia on the affected site of patients with CRPS, with moderate heterogeneity (Additional file 37 : Fig. S37) (Table 6 ) [ 45 , 56 ].

According to the GRADE assessment, there was low-quality evidence suggesting a significant increase in the area of hyperalgesia on the affected site of patients with CRPS (Table 6 ).

Flare area after electric stimulus

Two studies (one with low quality and one with fair quality) investigated flare areas using laser Doppler imaging [ 45 , 58 ]. Weber et al. showed a significant increase in flare area after the application of electric stimulus, while Seifert et al. showed no difference between patients with CRPS and healthy controls. We could not add the results in the meta-analysis because of the different techniques used; Weber et al. inserted cutaneous microdialysis fiber to assess protein extravasation while blocking the radial and peroneal nerves at the wrist and ankle, respectively. This could interfere with the assessment of the flare area that occurred after inserting the microdialysis fiber. Seifert et al. assessed the flare area before and after electric stimulation of the affected area without inserting the microdialysis fiber or blocking the radial and peroneal nerves.

Electric pain threshold and current detection threshold

Two low-quality studies investigated the sensory profile after the application of electric current [ 45 , 59 ]. Seifert et al. used a 1 Hz electric current to measure both pain and detection thresholds and found no differences between CRPS patients (affected and contralateral sides) and healthy controls [ 45 ]. Raj et al. used electric current of different frequencies and showed that 64% of patients with CRPS had abnormal electric pain threshold, while a percentage of 33% showed abnormal current detection threshold on the affected side, with some abnormalities on the contralateral side [ 59 ]. Thus, there were inconsistent findings regarding both electric pain and detection thresholds in CRPS, which need further investigations.

Dynamic mechanical allodynia

Several studies indicated the presence of DMA in CRPS [ 15 , 42 , 43 , 44 , 45 , 55 , 59 , 67 , 69 ].

Paradoxical heat sensation

Several studies indicated that PHS is not frequent in CRPS [ 14 , 15 , 47 , 53 , 67 , 69 , 73 ].

Endogenous pain modulation

Two studies (one with low quality and one with fair quality) investigated endogenous pain modulation in CRPS [ 45 , 53 ]. One study used conditioned pain modulation and found comparable descending pain modulation in patients with CRPS and controls [ 53 ]. Seifert et al. showed enhanced pain facilitation in CRPS after using repetitive electric pulse stimulation [ 45 ].

Level of pleasantness in CRPS

Two fair-quality studies looked at the pleasantness level following c-tactile touch perception on the affected side, and their meta-analysis revealed that CRPS patients had significantly lower pleasantness levels than healthy controls, without heterogeneity (Additional file 38 : Fig. S38) (Table 6 ) [ 71 , 72 ].

On the contralateral side, the meta-analysis of two studies of fair quality investigating the pleasantness level after c-tactile touch perception showed no difference in pleasantness level on the contralateral limb of CRPS compared with healthy controls, with moderate heterogeneity (Additional file 39 : Fig. S39) (Table 6 ) [ 71 , 72 ].

According to the GRADE assessment, there was moderate-quality evidence suggesting a significant reduction of pleasantness levels at the affected site in patients with CRPS, but at remote joints, there was low-quality evidence suggesting that there was no difference (Table 6 ).

Sensory profile of children with CRPS

The meta-analysis of two fair-quality studies including a total of 76 children with CRPS showed a significant loss of cold sensation on the affected areas of CRPS, with substantial heterogeneity (Additional file 40 : Fig. S40) (Table 6 ) [ 46 , 64 ].

On the contralateral side, a meta-analysis of two fair-quality studies including a total of 76 children with CRPS showed no difference in CDT between patients with CRPS and controls, with considerable heterogeneity (Additional file 41 : Fig. S41) (Table 6 ) [ 46 , 64 ].

According to the GRADE assessment, there was low-quality evidence suggesting loss of cold sensations of the affected site in patients with CRPS, but at the contralateral side, there was low-quality evidence suggesting that there was no difference (Table 6 ).

The meta-analysis of two studies with fair quality including a total of 76 children with CRPS reported no difference in warm sensation on the affected areas between patients with CRPS and controls, with considerable heterogeneity (Additional file 42 : Fig. S42) (Table 6 ) [ 46 , 64 ].

On the contralateral side, a meta-analysis of two fair-quality studies including a total of 76 children with CRPS reported no difference in WDT between patients with CRPS and controls, with considerable heterogeneity (Additional file 43 : Fig. S43) (Table 6 ) [ 46 , 64 ].

According to the GRADE assessment, there was low-quality evidence suggesting that there was no difference of warm sensations in patients with CRPS, either at the affected site or the contralateral side (Table 6 ).

A meta-analysis of three fair-quality studies including a total of 102 children with CRPS showed a significant gain of CPT on the affected site of CRPS, with considerable heterogeneity (Additional file 44 : Fig. 44) (Table 6 ) [ 41 , 46 , 64 ].

On the contralateral side, a meta-analysis of two fair-quality studies including a total of 76 children with CRPS reported no difference in CPT between patients with CRPS and controls, without significant heterogeneity (Additional file 45 : Fig. S45) (Table 6 ) [ 46 , 64 ].

According to the GRADE assessment, there was low-quality evidence suggesting gain of cold pain thresholds of the affected site in patients with CRPS, but at the contralateral side, there was low-quality evidence suggesting that there was no difference (Table 6 ).

On the affected side, a meta-analysis of three fair-quality studies including a total of 102 children with CRPS reported no difference in HPT between patients with CRPS and controls, with considerable heterogeneity (Additional file 46 : Fig. 46) (Table 6 ) [ 41 , 46 , 64 ].

On the contralateral side, a meta-analysis of two fair-quality studies including a total of 76 children with CRPS reported no difference in HPT between patients with CRPS and controls, with considerable heterogeneity (Additional file 47 : Fig. S47) (Table 6 ) [ 46 , 64 ].

According to the GRADE assessment, there was low-quality evidence suggesting that there was no difference of heat pain thresholds in patients with CRPS, either at the affected site or the contralateral side (Table 6 ).

Frequencies of sensory abnormalities in adult with CRPS

Regarding the percentage of sensory loss and hyperalgesia, 25% to 33% of patients with CRPS showed a thermal and mechanical sensory loss, between 60 to 100% of patients showed pressure pain hyperalgesia, and 30% to 40% of patients showed thermal hyperalgesia (Table 5 ) [ 14 , 15 , 69 ].

Sensitivity analysis

A sensitivity analysis was carried out, and studies with a high risk of bias were omitted. As a result, p values of the effect sizes were not significantly impacted for all outcomes except TSL of remote areas and MPT of the afflicted site, which showed a non-significant difference. Levels of heterogeneity were also not significantly impacted except for CDT of the affected site, WUR of the affected site, pain rating of the affected site, MPT of the affected site, and MPS of the affected site and the remote areas, which showed a significant reduction. However, after adjusting for low-quality studies, levels of heterogeneity of MDT of the affected site and TSL of the remote areas were significantly increased.

This systematic review aimed to summarize the current literature on QST measurements, pain ratings after noxious stimulus, area of pinprick hyperalgesia, and flare area in patients with CRPS to examine the sensory profile and underlying pain mechanisms.

Adult patients with CRPS showed loss of all detection thresholds (CDT, WDT, MDT, VDT, and TSL) compared to controls, both in the affected and contralateral sides. Also, there was a significant gain in CPT, HPT, and PPT both in the affected and remote areas. Furthermore, pain ratings after noxious stimulus showed significant elevation in the affected and contralateral areas, while MPS was elevated in the affected area only. The area of pinprick hyperalgesia was larger in CRPS compared to healthy controls, while the results for flare area were contradictory. The sensory profile of children with CRPS showed loss of cold sensation and cold hyperalgesia in the affected region without apparent sensory deficits at the remote areas away from the affected site.

Interestingly, adult patients with CRPS showed both sensory loss and primary and secondary hyperalgesia for all pain stimuli in the affected and remote areas, which strongly suggests the involvement of central nervous system and central sensitization [ 79 , 80 , 81 ] . This has also been supported by investigations in CRPS patients, which revealed bilateral structural and functional abnormalities in brain areas important for pain processing, cognition, and motor behavior [ 79 , 81 , 82 ]. Thus, central sensitization can be initiated by the enhanced peripheral sensitization (enhanced local hyperalgesia) [ 47 , 83 ], or neuroplasticity at the spinal and brain levels (hemisensory abnormalities and increased area after pinprick hyperalgesia) [ 45 , 63 , 70 , 84 , 85 ], or the release of inflammatory mediators after tissue injury as substance p, bradykinin, calcitonin gene-related peptide, interleukin-1 β , -2, -6, and tumor necrosis factor- α [ 8 , 86 , 87 ]. The diffuse sensory loss discovered in this meta-analysis could be attributed to decreased neurite density in both affected and unaffected sides of CRPS patients, or it could have a central origin [ 19 , 43 , 72 , 88 ]. Finally, the reduced pleasantness level in CRPS could indicate loss of small nerve fibers and central nervous system remodeling as the pleasantness levels reduced more in patients with CRPS accompanied with depression and allodynia than those without allodynia and depression [ 71 , 72 ].

Comparing the sensory phenotype in CRPS with neuropathic pain conditions reveals distinct sensory patterns. In carpal tunnel syndrome, recent study revealed dominant sensory loss localized only to the affected hand area with inconclusive evidence about central sensitization [ 89 ]. Also, in different radiculopathies, the sensory loss was localized to maximum pain area and dermatomal area with inconclusive picture about the presence of hyperalgesia [ 90 , 91 , 92 ]. Even in migraine, the impaired pain processing was localized to the affected area [ 93 ]. Recently, a new study suggested contralateral spread of sensory loss in painful and painless unilateral neuropathy with slightly limited spread of hyperalgesia [ 94 ]. In contrast, the sensory loss and thermal and mechanical hyperalgesia in CRPS were diffuse as evidenced by bilateral sensory loss and bilateral reduction of neurite density. Comparing CRPS to other chronic conditions as tendinitis and arthritis, CRPS showed more prominent thermal and mechanical hyperalgesia [ 95 , 96 , 97 ]. Comparing CRPS to chronic conditions with unknown etiology such as fibromyalgia shows comparable results both at the level of diffuse sensory loss or hyperalgesia or reduced level of pleasantness after C-tactile perception [ 52 , 98 , 99 ], which could suggest shared pain mechanisms and etiologies. Such findings could support classifying CRPS as a nociplastic pain type instead of neuropathic pain type [ 100 ], in agreement with the recent definition and grading system of neuropathic pain and IASP recent classification which excluded CRPS [ 100 , 101 , 102 ]. Interestingly, there was evidence of the presence of different comorbidities in CRPS such as sleep disturbances, post-traumatic stress disorder, and increased sensitivity to light and auditory stimuli [ 6 , 12 , 103 , 104 , 105 ] that strongly suggest a nociplastic mechanism for CRPS. Also, the frequency of sensory abnormalities in CRPS is more consistent than the frequencies found in previous studies for neuropathic pain conditions. In carpal tunnel syndrome, the percentage of patients with sensory loss was found to range from 22 to 33%, thermal hyperalgesia from 1 to 45%, and mechanical hyperalgesia from 20 to 45% [ 92 , 106 , 107 ].

Regarding CPM in CRPS, there were two studies discussing endogenous pain modulation in CRPS. One study showed enhanced pain facilitation rather than impaired descending pain inhibition after using repetitive noxious electrical stimuli [ 45 ]. The other study showed unimpaired descending pain inhibition when using the restricted CPM paradigm (heat was used as a test stimulus and cold as a conditioning stimulus) [ 53 ]. These contradictory results might be explained by the different disease duration (mean duration was 22 months in the study of Seifert et al., while the maximum disease duration was 12 months in the study of Kumowski et al.) and/or by the different procedures of assessment of endogenous pain modulation. Fortunately, offset analgesia is a paradigm which can also assess endogenous pain modulation that showed impaired pain inhibition in patients with CRPS [ 108 ].

No difference was found for temporal summation, represented by WUR, between individuals with CRPS and controls both in the affected and the contralateral limb. This might be due to the small cohort of patients with CRPS in the included studies that investigated WUR, except for Gierthmühlen et al. [ 15 ], who showed elevated WUR in a large cohort of patients with CRPS. Importantly, the diffuse loss of small nerve fibers bilaterally can cause the absence of WUR both in the affected and the contralateral regions [ 43 ]. Interestingly, WUR of CRPS type II (with evidence of nerve injury) showed no difference when compared to the control group [ 15 ], similar to the findings of WUR in CTS (median nerve injury) which showed no difference also [ 89 ].

Sensory profile of children and adolescents with CRPS showed loss of cold sensation and cold hyperalgesia at the affected region only, indicating less severe form of CRPS in this age group. Interestingly, children and adolescent with CRPS showed better prognosis and improvement than adults with CRPS, which might be related to the less severe sensory abnormalities [ 109 ]. Importantly, the findings of sensory profile of children and adolescents with CRPS are based on three studies only, which prevents us from drawing a comprehensive sensory profile.

Limitations of the review

Since the overall level of certainty ranged from very low to moderate based on the GRADE assessment [ 34 , 35 ], the results should be regarded with caution. There were various issues that decreased the general level of certainty. At first, the included studies were observational studies with poor to good quality ratings. Second, there was moderate to substantial heterogeneity across the obtained results. Finally, the meta-analysis of several QST outcomes was based on a small number of studies, and the effect sizes occasionally appear small with large confidence intervals.

It is important to highlight that the sensitivity analysis controlling for low-quality studies (meta-analyses were repeated while excluding studies with high risk of bias) showed a non-significant effect either at the levels of heterogeneity or the obtained effect sizes and corresponding p values of most outcomes. Therefore, the degree of heterogeneity seen in the results might not be explained by the risk of bias of the included studies.

Possible causes of heterogeneity might include the different disease duration of CRPS across the included studies (ranging from six months to five years). Disease duration seems to result in different sensory profiles in patients with CRPS [ 14 , 47 , 70 ]. Thus, future studies might consider comparing sensory profiles of patients with CRPS of different durations. This heterogeneity may be also explained by several factors, starting with the diagnostic criteria for CRPS, which were modified to rely on the Budapest criteria [ 1 ] rather than the previous IASP standards [ 110 ]. Second, based on the predominant pathophysiology, a recent categorization is better able to distinguish between three clusters of individuals with CRPS type 1 and type 2: CRPS of central phenotype, CRPS of peripheral phenotype, and CRPS of mixed phenotype [ 111 ]. As a result, limiting the classification of CRPS to type 1 and type 2 may produce inconsistent results. It is interesting to note that the outcomes of this review are comparable to the findings of the one study that looked at the QST outcomes in CRPS type 2 [ 15 ]. This could provide credibility to the current division into three phenotypes.

It is noteworthy to mention that some of the included studies recruited a mix of CRPS type 1 and type 2 which might represent a potential cause of heterogeneity. However, the number patients with CRPS type 2 included in these studies was very small. For example, Terkelsen et al. recruited 2 patients with CRPS type 2 and 18 patients with CRPS type 1[ 18 ].

The results of the quantitative sensory testing outcomes of adolescents and children with CRPS were only examined in three studies, which limited the conclusions. Therefore, additional research is required to support the findings of the present review.

A mix of diffuse thermal and mechanical sensory loss and hyperalgesias in the affected and remote areas is the dominant sensory phenotype in CRPS indicating the dominant peripheral and central sensitization as key underlying pain mechanisms. There is some evidence regarding the enhanced pain facilitation more than impaired descending pain inhibition as evident by elevated thermal and mechanical pain ratings and increased areas of pinprick hyperalgesia. Such results could indicate the involvement of small nerve fibers both at the affected and remote areas. Adolescents and children with CRPS showed less severe form of sensory abnormalities as evident with loss of cold detection sensation and cold hyperalgesia at the affected site.

Future implications of the review

Further research is needed investigating the efficacy of the descending pain inhibition in patients with CRPS, as well as the widespread sensory loss and hyperalgesia, the pleasantness level after C-tactile stimulation, the electric pain and detection thresholds, and the area of pinprick hyperalgesia of the affected site and remote areas.

As evident from this review, there was a diffuse loss of sensation in patients with CRPS. Thus, the previous studies which compared the QST outcomes of the affected area to that of the contralateral healthy side might result in inconsistent findings as well as might hinder the progress in providing better treatment options. We suggest comparing the affected or contralateral side with reference values of healthy subjects or control group, to avoid any bias.

Previous research revealed that the sensory deficits extended from the affected area to the ipsilateral body sites more compared to the contralateral side [ 84 , 85 ]. Thus, such studies lacked the presence of control group, while we suggest comparing the results of QST in affected areas, areas in the ipsilateral side away from the affected region, and control group. It is noteworthy that Rooijen et al. investigated the sensory deficits in CRPS affected area, contralateral area, and ipsilateral areas away from the affected region but this study included both patients with CRPS with dystonia and without dystonia [ 51 ]. Moreover, face area showed specific sensory abnormalities in patients with CRPS [ 51 , 63 ] which indeed needs further investigations.

A group of CRPS patients had elevated WUR, whereas another group had no difference when compared to healthy controls. Future research will therefore be required to determine the relationship between the decline in small fiber density and the change in WUR, as it is possible that the decline in small fiber density could prevent the change of the WUR.

Finally, in order to inform better treatment options, it is crucial to compare the new classification of CRPS into three phenotypes (central, peripheral, and mixed) with the existing classification into type 1 and 2. The first step is to investigate the sensory profile of CRPS type 2 and compare it to the results of our review. This could indicate the same sensory profiles and the same underlying pain mechanisms. Thus, the necessity to switch over to the new classification would then likely be of vital importance.

Harden RN, et al. Validation of proposed diagnostic criteria (the “Budapest Criteria”) for complex regional pain syndrome. Pain. 2010;150(2):268–74.

Article   Google Scholar  

de Mos M, et al. The incidence of complex regional pain syndrome: a population-based study. Pain. 2007;129(1–2):12–20.

Goh EL, Chidambaram S, Ma D. Complex regional pain syndrome: a recent update. Burns Trauma 2017;5:2.

Bean DJ, Johnson MH, Kydd RR. Relationships Between Psychological Factors, Pain, and Disability in Complex Regional Pain Syndrome and Low Back Pain. The Clinical Journal of Pain, 2014;30(8):647–53.

Lohnberg JA, Altmaier EM. A review of psychosocial factors in complex regional pain syndrome. J Clin Psychol Med Settings. 2013;20(2):247–54.

Drummond PD, Finch PM. Photophobia in complex regional pain syndrome: visual discomfort is greater on the affected than unaffected side. Pain. 2020;162(4):1233–40.

Shim H, et al. Complex regional pain syndrome: a narrative review for the practising clinician. Br J Anaesth. 2019;123(2):e424–33.

Article   CAS   Google Scholar  

Birklein F, Schmelz MJ. Neuropeptides, neurogenic inflammation and complex regional pain syndrome (CRPS). Neurosci Lett. 2008;437(3):199–202.

Cheng J-K, Ji R-R. Intracellular signaling in primary sensory neurons and persistent pain. Neurochem Res. 2008;33(10):1970–8.

Schwartzman RJ, Alexander GM, Grothusen J. Pathophysiology of complex regional pain syndrome. Expert Rev Neurother. 2006;6(5):669–81.

Finch PM, et al. Up-regulation of cutaneous α1—adrenoceptors in complex regional pain syndrome type I. Pain Med. 2014;15(11):1945–56.

Speck V, et al. Increased prevalence of posttraumatic stress disorder in CRPS. Eur J Pain. 2017;21(3):466–73.

Merskey. International association for the study of pain: pain terms: a current list with definitions and notes on usage. PAIN 1986;3:212–225

Reimer M, et al. Sensitization of the nociceptive system in complex regional pain syndrome. PLoS ONE. 2016;11(5):e0154553.

Gierthmuhlen J, et al. Sensory signs in complex regional pain syndrome and peripheral nerve injury. Pain. 2012;153(4):765–74.

Enax-Krumova E, et al. Changes of the sensory abnormalities and cortical excitability in patients with complex regional pain syndrome of the upper extremity after 6 months of multimodal treatment. Pain Med. 2017;18(1):95–106.

Latremoliere A, Woolf CJ. Central sensitization: a generator of pain hypersensitivity by central neural plasticity. J Pain. 2009;10(9):895–926.

Terkelsen AJ, et al. Bilateral hypersensitivity to capsaicin, thermal, and mechanical stimuli in unilateral complex regional pain syndrome. Anesthesiology. 2014;120(5):1225–36.

Oaklander AL, et al. Evidence of focal small-fiber axonal degeneration in complex regional pain syndrome-I (reflex sympathetic dystrophy). Pain. 2006;120(3):235–43.

Hansson P, Backonja M, Bouhassira D. Usefulness and limitations of quantitative sensory testing: clinical and research application in neuropathic pain states. Pain. 2007;129(3):256–9.

Brodal P. The central nervous system: structure and function. Oxford: Oxford university Press; 2004.

Google Scholar  

Young PA, Young PH, Tolbert DL. Basic clinical neuroscience. Philadelphia: Lippincott Williams & Wilkins; 2015.

Schmelz M, et al. Encoding of burning pain from capsaicin-treated human skin in two categories of unmyelinated nerve fibres. Brain. 2000;123(Pt 3):560–71.

Weidner C, et al. Functional attributes discriminating mechano-insensitive and mechano-responsive C nociceptors in human skin. J Neurosci. 1999;19(22):10184–90.

Stang A. Critical evaluation of the Newcastle–Ottawa scale for the assessment of the quality of nonrandomized studies in meta-analyses. Eur J Epidemiol. 2010;25(9):603–5.

Pigott T. Advances in meta-analysis. Berlin: Springer Science & Business Media; 2012.

Book   Google Scholar  

Fleiss JL. The statistical basis of meta-analysis. Stat Methods Med Res. 1993;2(2):121–45.

Andrade C. Mean difference, standardized mean difference (SMD), and their use in meta-analysis: as simple as it gets. J Clin Psychiatry. 2020;81(5):11349.

Peters JL, et al. Comparison of two methods to detect publication bias in meta-analysis. JAMA. 2006;295(6):676–80.

Egger M, et al. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315(7109):629–34.

Begg CB, Mazumdar M. Operating characteristics of a rank correlation test for publication bias. Biometrics. 1994;50(4):1088–101.

Duval S, Tweedie R. Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics. 2000;56(2):455–63.

Higgins JP. Cochrane handbook for systematic reviews of interventions version 5.0.1. Cochrane Collab. 2008. http://www.cochrane-handbook.org.ISO690 .

Schünemann HJ, et al. Grading quality of evidence and strength of recommendations for diagnostic tests and strategies. BMJ. 2008;336(7653):1106–10.

Hultcrantz M, et al. The GRADE Working Group clarifies the construct of certainty of evidence. J Clin Epidemiol. 2017;87:4–13.

Wan X, et al. Estimating the sample mean and standard deviation from the sample size, median, range and/or interquartile range. BMC Med Res Methodol. 2014;14(1):135.

Deeks JJ, Higgins JP, Altman DG. Cochrane statistical methods group. Analysing data and undertaking meta‐analyses. Cochrane handbook for systematic reviews of interventions, 2019;pp. 241–84.

Bank PJM, et al. Force modulation deficits in complex regional pain syndrome: a potential role for impaired sense of force production. Eur J Pain (United Kingdom). 2014;18(7):1013–23.

CAS   Google Scholar  

König S, et al. The serum protease network—one key to understand complex regional pain syndrome pathophysiology. Pain. 2019;160(6):1402–9.

König S, Engl C, Bayer M, Escolano-Lozano F, Rittner H, Rebhorn C, & Birklein F. Substance P Serum degradation in complex regional pain syndrome–another piece of the puzzle? J Pain, 2022;23(3):501–07.

Becerra L, et al. Intrinsic brain networks normalize with treatment in pediatric complex regional pain syndrome. NeuroImage Clin. 2014;6:347–69.

Vartiainen NV, Kirveskari E, Forss N. Central processing of tactile and nociceptive stimuli in complex regional pain syndrome. Clin Neurophysiol. 2008;119(10):2380–8.

Rasmussen VF, et al. Bilaterally reduced intraepidermal nerve fiber density in unilateral CRPS-I. Pain Med (United States). 2018;19(10):2021–30.

Kemler MA, et al. Thermal thresholds in complex regional pain syndrome type I: sensitivity and repeatability of the methods of limits and levels. Clin Neurophysiol. 2000;111(9):1561–8.

Seifert F, et al. Differential endogenous pain modulation in complex-regional pain syndrome. Brain. 2009;132(3):788–800.

Sethna NF, et al. Cutaneous sensory abnormalities in children and adolescents with complex regional pain syndromes. Pain. 2007;131(1–2):153–61.

Huge V, et al. Interaction of hyperalgesia and sensory loss in complex regional pain syndrome type I (CRPS I). PLoS ONE. 2008;3(7):1–10.

Vatine JJ, Tsenter J, Nirel R. Experimental pressure pain in patients with complex regional pain syndrome, type I (Reflex sympathetic dystrophy). Am J Phys Med Rehabil. 1998;77(5):382–7.

Mainka T, et al. Comparison of muscle and joint pressure-pain thresholds in patients with complex regional pain syndrome and upper limb pain of other origin. Pain. 2014;155(3):591–7.

Van Rooijen DE, et al. Muscle hyperalgesia correlates with motor function in complex regional pain syndrome type 1. J Pain. 2013;14(5):446–54.

Van Rooijen DE, Marinus J, Van Hilten JJ. Muscle hyperalgesia is widespread in patients with complex regional pain syndrome. Pain. 2013;154(12):2745–9.

Palmer S, et al. Sensory function and pain experience in arthritis, complex regional pain syndrome, fibromyalgia syndrome, and pain-free volunteers a cross-sectional study. Clin J Pain. 2019;35(11):894–900.

Kumowski N, et al. Unimpaired endogenous pain inhibition in the early phase of complex regional pain syndrome. Eur J Pain (United Kingdom). 2017;21(5):855–65.

Knudsen L, Finch PM, Drummond PD. The specificity and mechanisms of hemilateral sensory disturbances in complex regional pain syndrome. J Pain. 2011;12(9):985–90.

Edinger L, et al. Objective sensory evaluation of the spread of complex regional pain syndrome. Pain Physician. 2013;16(6):581–91.

Wolanin MW. et al. Loss of surround inhibition and after sensation as diagnostic parameters of complex regional pain syndrome. Neurosci Med. 2012;3:344–53.

Thimineur M, et al. Central nervous system abnormalities in complex regional pain syndrome (CRPS): clinical and quantitative evidence of medullary dysfunction. Clin J Pain. 1998;14(3):256–67.

Weber M, et al. Facilitated neurogenic inflammation in complex regional pain syndrome. Pain. 2001;91(3):251–7.

Raj PP, et al. Painless electrodiagnostic current perception threshold and pain tolerance threshold values in CRPS subjects and healthy controls: a multicenter study. Pain Pract. 2001;1(1):53–60.

Meyer-Frießem CH, et al. Pain thresholds and intensities of CRPS type I and neuropathic pain in respect to sex. Eur J Pain (United Kingdom). 2020;24(6):1058–71.

Huge V, et al. Complex interaction of sensory and motor signs and symptoms in chronic CRPS. PLoS ONE. 2011;6(4):1–13.

Sieweke N, et al. Patterns of hyperalgesia in complex regional pain syndrome. Pain. 1999;80(1–2):171–7.

Drummond P, Finch P. The source of hemisensory disturbances in complex regional pain syndrome. Clin J Pain. 2021;37(2):79–85.

Truffyn EE, et al. Sensory function and psychological factors in children with complex regional pain syndrome type 1. J Child Neurol. 2021;36:823–30.

Dietz C, et al. What is normal trauma healing and what is complex regional pain syndrome I? An analysis of clinical and experimental biomarkers. Pain. 2019;160(10):2278–89.

Üçeyler N, et al. Differential expression patterns of cytokines in complex regional pain syndrome. Pain. 2007;132(1–2):195–205.

Kolb L, et al. Cognitive correlates of “neglect-like syndrome” in patients with complex regional pain syndrome. Pain. 2012;153(5):1063–73.

Wittayer M, et al. Correlates and importance of neglect-like symptoms in complex regional pain syndrome. Pain. 2018;159(5):978–86.

Maier C, et al. Quantitative sensory testing in the German Research Network on Neuropathic Pain (DFNS): Somatosensory abnormalities in 1236 patients with different neuropathic pain syndromes. Pain. 2010;150(3):439–50.

Munts AG, et al. Thermal hypesthesia in patients with complex regional pain syndrome related dystonia. J Neural Transm. 2011;118(4):599–603.

Habig K, et al. Low mechano-afferent fibers reduce thermal pain but not pain intensity in CRPS. BMC Neurol. 2021;21(1):272.

Gossrau G, et al. C-tactile touch perception in patients with chronic pain disorders. Pain Reports. 2021;6(2):e941–e941.

Eberle T, et al. Warm and cold complex regional pain syndromes: differences beyond skin temperature? Neurology. 2009;72(6):505–12.

Ohmichi Y, et al. Physical disuse contributes to widespread chronic mechanical hyperalgesia, tactile allodynia, and cold allodynia through neurogenic inflammation and spino-parabrachio-amygdaloid pathway activation. Pain. 2020;161(8):1808–23.

Drummond E, et al. Topical prazosin attenuates sensitivity to tactile stimuli in patients with complex regional pain syndrome. European journal of pain (London, England), 2015;20:926–35.

Vaneker M, et al. Patients initially diagnosed as ‘warm’ or ‘cold’ CRPS 1 show differences in central sensory processing some eight years after diagnosis: a quantitative sensory testing study. Pain. 2005;115(1):204–11.

Magerl W, et al. Reference data for quantitative sensory testing (QST): refined stratification for age and a novel method for statistical comparison of group data. Pain. 2010;151(3):598–605.

Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74.

Maihöfner C, et al. Brain processing during mechanical hyperalgesia in complex regional pain syndrome: a functional MRI study. Pain. 2005;114(1):93–103.

Geha PY, et al. The brain in chronic CRPS pain: abnormal gray-white matter interactions in emotional and autonomic regions. Neuron. 2008;60(4):570–81.

Shokouhi M, et al. Structural and functional brain changes at early and late stages of complex regional pain syndrome. J Pain. 2018;19(2):146–57.

Lenz M, et al. Bilateral somatosensory cortex disinhibition in complex regional pain syndrome type I. Neurology. 2011;77(11):1096.

Bruehl S. An update on the pathophysiology of complex regional pain syndrome. Anesthesiology. 2010;113(3):713–25.

Rommel O, et al. Quantitative sensory testing, neurophysiological and psychological examination in patients with complex regional pain syndrome and hemisensory deficits. Pain. 2001;93(3):279–93.

Drummond P, et al. Hemisensory disturbances in patients with complex regional pain syndrome. Pain. 2018;159(9):1824–32.

Ferreira SH, Lorenzetti BB, Poole S. Bradykinin initiates cytokine-mediated inflammatory hyperalgesia. Br J Pharmacol. 1993;110(3):1227–31.

Birklein F, et al. Complex regional pain syndrome—phenotypic characteristics and potential biomarkers. Nat Rev Neurol. 2018;14(5):272–84.

Albrecht PJ, et al. Pathologic alterations of cutaneous innervation and vasculature in affected limbs from patients with complex regional pain syndrome. Pain. 2006;120(3):244–66.

Sobeeh MG, Ghozy S, Elshazli RM, Landry M. Pain mechanisms in carpal tunnel syndrome: a systematic review and meta-analysis of quantitative sensory testing outcomes. Pain. 2022;163(10):e1054–94.

Tampin B, et al. Quantitative sensory testing somatosensory profiles in patients with cervical radiculopathy are distinct from those in patients with nonspecific neck-arm pain. Pain. 2012;153(12):2403–14.

Tampin B, et al. Association of quantitative sensory testing parameters with clinical outcome in patients with lumbar radiculopathy undergoing microdiscectomy. Eur J Pain. 2020;24(7):1377–92.

Tampin B, Vollert J, Schmid AB. Sensory profiles are comparable in patients with distal and proximal entrapment neuropathies, while the pain experience differs. Curr Med Res Opin. 2018;34(11):1899–906.

Nahman-Averbuch H, et al. Quantitative sensory testing in patients with migraine: a systematic review and meta-analysis. Pain. 2018;159(7):1202–23.

Enax-Krumova E, et al. Contralateral sensory and pain perception changes in patients with unilateral neuropathy. Neurology. 2021;97(4): e389.

Meng H, Dai J, Li Y. Quantitative sensory testing in patients with the muscle pain subtype of temporomandibular disorder: a systemic review and meta-analysis. Clin Oral Investig. 2021;25(12):6547–59.

Suokas AK, et al. Quantitative sensory testing in painful osteoarthritis: a systematic review and meta-analysis. Osteoarthr Cartil. 2012;20(10):1075–85.

Malfliet A, et al. Lack of evidence for central sensitization in idiopathic, non-traumatic neck pain: a systematic review. Pain Physician. 2015;18(3):223–36.

Blumenstiel K, et al. Quantitative sensory testing profiles in chronic back pain are distinct from those in fibromyalgia. Clin J Pain. 2011;27(8):682–90.

Fitzcharles MA, et al. Nociplastic pain: towards an understanding of prevalent pain conditions. Lancet. 2021;397(10289):2098–110.

Kosek E, et al. Do we need a third mechanistic descriptor for chronic pain states? Pain. 2016;157(7):1382–6.

Finnerup NB, et al. Neuropathic pain: an updated grading system for research and clinical practice. Pain. 2016;157(8):1599–606.

Treede RD, et al. Neuropathic pain: redefinition and a grading system for clinical and research purposes. Neurology. 2008;70(18):1630–5.

Drummond P, Finch P. pupillary reflexes in complex regional pain syndrome: asymmetry to arousal stimuli suggests an ipsilateral locus coeruleus deficit. J Pain. 2022;23(1):131–40.

de Klaver MJ, et al. Hyperacusis in patients with complex regional pain syndrome related dystonia. J Neurol Neurosurg Psychiatry. 2007;78(12):1310–3.

Nijs J, et al. Nociplastic pain criteria or recognition of central sensitization? pain phenotyping in the past, present and future. J Clin Med. 2021;10(15):3203.

Kennedy DL, et al. Responsiveness of quantitative sensory testing-derived sensory phenotype to disease-modifying intervention in patients with entrapment neuropathy: a longitudinal study. Pain. 2021;162(12):2881–93.

Matesanz L, et al. Somatosensory and psychological phenotypes associated with neuropathic pain in entrapment neuropathy. Pain. 2021;162(4):1211–20.

Szikszay TM, Adamczyk WM, Luedtke K. The Magnitude of offset analgesia as a measure of endogenous pain modulation in healthy participants and patients with chronic pain: a systematic review and meta-analysis. Clin J Pain. 2019;35(2):189–204.

Mesaroli G, et al. Clinical features of pediatric complex regional pain syndrome: a 5-year retrospective chart review. Clin J Pain. 2019;35(12):933–40.

Stanton-Hicks M, et al. Reflex sympathetic dystrophy: changing concepts and taxonomy. Pain. 1995;63(1):127–33.

Dimova V, et al. Clinical phenotypes and classification algorithm for complex regional pain syndrome. Neurology. 2020;94(4):e357–67.

Download references

Acknowledgements

This is the time to acknowledge my mom who is so brilliant, thanks for everything.

Open access funding provided by The Science, Technology & Innovation Funding Authority (STDF) in cooperation with The Egyptian Knowledge Bank (EKB). This study is self-funded.

Author information

Authors and affiliations.

Department of Physical Therapy for Musculoskeletal Disorders and its Surgeries, Faculty of Physical Therapy, Cairo University, Giza, Egypt

Mohamed Gomaa Sobeeh, Karima Abdelaty Hassan, Enas Fawzy Youssef, Nadia Abdelazim Fayaz & Maha Mostafa Mohammed

Faculty of Physical Therapy, Sinai University, Ismailia, Egypt

Mohamed Gomaa Sobeeh

CINTESIS.UA@RISE, School of Health Sciences, University of Aveiro, Aveiro, Portugal

Anabela Gonçalves da Silva

You can also search for this author in PubMed   Google Scholar

Contributions

All authors have designed the project. MS and KH participated mainly in the risk of bias assessment, while MS and AS participated mainly in data extraction. All authors participated in writing and revising the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Mohamed Gomaa Sobeeh .

Ethics declarations

Ethical approval and consent to participate.

The manuscript does not contain clinical studies or patient data.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

. Fig. S1 Pooled results of cold detection threshold (CDT) of the affected area. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 2

. Fig. S2 Funnel plot of cold detection threshold of the affected side.

Additional file 3

. Fig. S3 Pooled results of cold detection threshold (CDT) of the remote areas. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 4

. Fig. S4 Funnel plot of cold detection threshold of the remote areas.

Additional file 5.

Fig. S5 Pooled results of warm detection threshold (WDT) of the affected area. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 6

. Fig. S6 Funnel plot of warm detection threshold of the affected side.

Additional file 7

. Fig. S7 Pooled results of warm detection threshold (WDT) of the remote areas. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 8.

Fig. S8 Funnel plot of warm detection threshold of the remote areas.

Additional file 9

. Fig. S9 Pooled results of thermal sensory limen (TSL) of the affected area. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 10

. Fig. S10 Pooled results of thermal sensory limen (TSL) of the remote areas. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 11

. Fig. S11 Pooled results of mechanical detection threshold (MDT) of the affected area. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 12

. Fig. S12 Pooled results of mechanical detection threshold (MDT) of the remote areas. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 13

. Fig. S13 Pooled results of vibration detection threshold (VDT) of the affected area. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 14

. Fig. S14 Pooled results of vibration detection threshold (VDT) of the remote areas. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 15

. Fig. S15 Pooled results of cold pain threshold (CPT) of the affected area. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 16.

Fig. S16 Funnel plot of cold pain threshold of the affected side.

Additional file 17

. Fig. S17 Pooled results of cold pain threshold (CPT) of the remote areas. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 18

. Fig. S18 Funnel plot of cold pain threshold of the remote areas.

Additional file 19.

Fig. S19 Pooled results of heat pain threshold (HPT) of the affected area. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 20

. Fig. S20 Funnel plot of heat pain threshold of the affected side.

Additional file 21

. Fig. S21 Pooled results of heat pain threshold (HPT) of the remote areas. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 22

. Fig. S22 Funnel plot of heat pain threshold of the remote areas.

Additional file 23

. Fig. S23 Pooled results of mechanical pain threshold (MPT) of the affected area. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 24

. Fig. S24 Pooled results of mechanical pain threshold (MPT) of the remote areas. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 25

. Fig. S25 Pooled results of pressure pain threshold (PPT) of the affected area (deep tissue PPT). SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 26

. Fig. S26 Funnel plot of pressure pain threshold of the affected side.

Additional file 27

. Fig. S27 Pooled results of pressure pain threshold (PPT) of the remote areas (deep tissue PPT). SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 28

. Fig. S28 Funnel plot of pressure pain threshold of the remote areas.

Additional file 29

. Fig. S29 Pooled results of pressure pain threshold (PPT) of the affected area (joint PPT). SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 30

. Fig. S30 Pooled results of pressure pain threshold (PPT) of the remote areas (joint PPT). SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 31

. Fig. S31 Pooled results of mechanical pain sensitivity (MPS) of the affected area. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 32

. Fig. S32 Pooled results of mechanical pain sensitivity (MPS) of the remote areas. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 33

. Fig. S33 Pooled results of wind-up ratio (WUR) of the affected area. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 34

. Fig. S34 Pooled results of wind-up ratio (WUR) of the remote areas. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 35

. Fig. S35 Pooled results of pain ratings after noxious stimulus of the affected area. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 36

. Fig. S36 Pooled results of pain ratings after noxious stimulus of the remote areas. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 37

. Fig. S37 Pooled results of area after induced pinprick hyperalgesia of the affected area. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 38

. Fig. S38 Pooled results of pleasantness level of C-tactile perception of the affected area. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 39

. Fig. S39 Pooled results of pleasantness level of C-tactile perception of the remote areas. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 40

. Fig. S40 Pooled results of cold detection threshold (CDT) of the affected area of children and adolescent with CRPS. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 41

. Fig. S41 Pooled results of cold detection threshold (CDT) of the contralateral side of children and adolescent with CRPS. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 42.

Fig. S42 Pooled results of warm detection threshold (WDT) of the affected area of children and adolescent with CRPS. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 43

. Fig. S43 Pooled results of warm detection threshold (WDT) of the contralateral side of children and adolescent with CRPS. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 44

. Fig. S44 Pooled results of cold pain threshold (CPT) of the affected area of children and adolescent with CRPS. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 45

. Fig. S45 Pooled results of cold pain threshold (CPT) of the contralateral side of children and adolescent with CRPS. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Additional file 46

. Fig. S46 Pooled results of heat pain threshold (HPT) of the affected area of children and adolescent with CRPS. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference

Additional file 47

. Fig. S47 Pooled results of heat pain threshold (HPT) of the contralateral side of children and adolescent with CRPS. SD: standard deviation, CRPS: complex regional pain syndrome, and Std Mean Difference: standardized mean difference.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Sobeeh, M.G., Hassan, K.A., da Silva, A.G. et al. Pain mechanisms in complex regional pain syndrome: a systematic review and meta-analysis of quantitative sensory testing outcomes. J Orthop Surg Res 18 , 2 (2023). https://doi.org/10.1186/s13018-022-03461-2

Download citation

Received : 18 October 2022

Accepted : 19 December 2022

Published : 02 January 2023

DOI : https://doi.org/10.1186/s13018-022-03461-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Complex regional pain syndrome
• Sensory profile
• Pain mechanisms
• Quantitative sensory testing

Journal of Orthopaedic Surgery and Research

ISSN: 1749-799X

• Submission enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• BMJ Glob Health
• v.4(Suppl 1); 2019

Synthesising quantitative and qualitative evidence to inform guidelines on complex interventions: clarifying the purposes, designs and outlining some methods

1 School of Social Sciences, Bangor University, Wales, UK

Andrew Booth

2 School of Health and Related Research (ScHARR), University of Sheffield, Sheffield, UK

Graham Moore

3 School of Social Sciences, Cardiff University, Wales, UK

Kate Flemming

4 Department of Health Sciences, The University of York, York, UK

Özge Tunçalp

5 Department of Reproductive Health and Research including UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP), World Health Organization, Geneva, Switzerland

Elham Shakibazadeh

6 Department of Health Education and Promotion, School of Public Health, Tehran University of Medical Sciences, Tehran, Iran

Associated Data

bmjgh-2018-000893supp001.pdf

bmjgh-2018-000893supp002.pdf

bmjgh-2018-000893supp003.pdf

bmjgh-2018-000893supp005.pdf

bmjgh-2018-000893supp004.pdf

Guideline developers are increasingly dealing with more difficult decisions concerning whether to recommend complex interventions in complex and highly variable health systems. There is greater recognition that both quantitative and qualitative evidence can be combined in a mixed-method synthesis and that this can be helpful in understanding how complexity impacts on interventions in specific contexts. This paper aims to clarify the different purposes, review designs, questions, synthesis methods and opportunities to combine quantitative and qualitative evidence to explore the complexity of complex interventions and health systems. Three case studies of guidelines developed by WHO, which incorporated quantitative and qualitative evidence, are used to illustrate possible uses of mixed-method reviews and evidence. Additional examples of methods that can be used or may have potential for use in a guideline process are outlined. Consideration is given to the opportunities for potential integration of quantitative and qualitative evidence at different stages of the review and guideline process. Encouragement is given to guideline commissioners and developers and review authors to consider including quantitative and qualitative evidence. Recommendations are made concerning the future development of methods to better address questions in systematic reviews and guidelines that adopt a complexity perspective.

Summary box

• When combined in a mixed-method synthesis, quantitative and qualitative evidence can potentially contribute to understanding how complex interventions work and for whom, and how the complex health systems into which they are implemented respond and adapt.
• The different purposes and designs for combining quantitative and qualitative evidence in a mixed-method synthesis for a guideline process are described.
• Questions relevant to gaining an understanding of the complexity of complex interventions and the wider health systems within which they are implemented that can be addressed by mixed-method syntheses are presented.
• The practical methodological guidance in this paper is intended to help guideline producers and review authors commission and conduct mixed-method syntheses where appropriate.
• If more mixed-method syntheses are conducted, guideline developers will have greater opportunities to access this evidence to inform decision-making.

Introduction

Recognition has grown that while quantitative methods remain vital, they are usually insufficient to address complex health systems related research questions. 1 Quantitative methods rely on an ability to anticipate what must be measured in advance. Introducing change into a complex health system gives rise to emergent reactions, which cannot be fully predicted in advance. Emergent reactions can often only be understood through combining quantitative methods with a more flexible qualitative lens. 2 Adopting a more pluralist position enables a diverse range of research options to the researcher depending on the research question being investigated. 3–5 As a consequence, where a research study sits within the multitude of methods available is driven by the question being asked, rather than any particular methodological or philosophical stance. 6

Publication of guidance on designing complex intervention process evaluations and other works advocating mixed-methods approaches to intervention research have stimulated better quality evidence for synthesis. 1 7–13 Methods for synthesising qualitative 14 and mixed-method evidence have been developed or are in development. Mixed-method research and review definitions are outlined in box 1 .

Defining mixed-method research and reviews

Pluye and Hong 52 define mixed-methods research as “a research approach in which a researcher integrates (a) qualitative and quantitative research questions, (b) qualitative research methods* and quantitative research designs, (c) techniques for collecting and analyzing qualitative and quantitative evidence, and (d) qualitative findings and quantitative results”.A mixed-method synthesis can integrate quantitative, qualitative and mixed-method evidence or data from primary studies.† Mixed-method primary studies are usually disaggregated into quantitative and qualitative evidence and data for the purposes of synthesis. Thomas and Harden further define three ways in which reviews are mixed. 53

• The types of studies included and hence the type of findings to be synthesised (ie, qualitative/textual and quantitative/numerical).
• The types of synthesis method used (eg, statistical meta-analysis and qualitative synthesis).
• The mode of analysis: theory testing AND theory building.

*A qualitative study is one that uses qualitative methods of data collection and analysis to produce a narrative understanding of the phenomena of interest. Qualitative methods of data collection may include, for example, interviews, focus groups, observations and analysis of documents.

†The Cochrane Qualitative and Implementation Methods group coined the term ‘qualitative evidence synthesis’ to mean that the synthesis could also include qualitative data. For example, qualitative data from case studies, grey literature reports and open-ended questions from surveys. ‘Evidence’ and ‘data’ are used interchangeably in this paper.

This paper is one of a series that aims to explore the implications of complexity for systematic reviews and guideline development, commissioned by WHO. This paper is concerned with the methodological implications of including quantitative and qualitative evidence in mixed-method systematic reviews and guideline development for complex interventions. The guidance was developed through a process of bringing together experts in the field, literature searching and consensus building with end users (guideline developers, clinicians and reviewers). We clarify the different purposes, review designs, questions and synthesis methods that may be applicable to combine quantitative and qualitative evidence to explore the complexity of complex interventions and health systems. Three case studies of WHO guidelines that incorporated quantitative and qualitative evidence are used to illustrate possible uses of mixed-method reviews and mechanisms of integration ( table 1 , online supplementary files 1–3 ). Additional examples of methods that can be used or may have potential for use in a guideline process are outlined. Opportunities for potential integration of quantitative and qualitative evidence at different stages of the review and guideline process are presented. Specific considerations when using an evidence to decision framework such as the Developing and Evaluating Communication strategies to support Informed Decisions and practice based on Evidence (DECIDE) framework 15 or the new WHO-INTEGRATE evidence to decision framework 16 at the review design and evidence to decision stage are outlined. See online supplementary file 4 for an example of a health systems DECIDE framework and Rehfuess et al 16 for the new WHO-INTEGRATE framework. Encouragement is given to guideline commissioners and developers and review authors to consider including quantitative and qualitative evidence in guidelines of complex interventions that take a complexity perspective and health systems focus.

Designs and methods and their use or applicability in guidelines and systematic reviews taking a complexity perspective

Supplementary data

Taking a complexity perspective.

The first paper in this series 17 outlines aspects of complexity associated with complex interventions and health systems that can potentially be explored by different types of evidence, including synthesis of quantitative and qualitative evidence. Petticrew et al 17 distinguish between a complex interventions perspective and a complex systems perspective. A complex interventions perspective defines interventions as having “implicit conceptual boundaries, representing a flexible, but common set of practices, often linked by an explicit or implicit theory about how they work”. A complex systems perspective differs in that “ complexity arises from the relationships and interactions between a system’s agents (eg, people, or groups that interact with each other and their environment), and its context. A system perspective conceives the intervention as being part of the system, and emphasises changes and interconnections within the system itself”. Aspects of complexity associated with implementation of complex interventions in health systems that could potentially be addressed with a synthesis of quantitative and qualitative evidence are summarised in table 2 . Another paper in the series outlines criteria used in a new evidence to decision framework for making decisions about complex interventions implemented in complex systems, against which the need for quantitative and qualitative evidence can be mapped. 16 A further paper 18 that explores how context is dealt with in guidelines and reviews taking a complexity perspective also recommends using both quantitative and qualitative evidence to better understand context as a source of complexity. Mixed-method syntheses of quantitative and qualitative evidence can also help with understanding of whether there has been theory failure and or implementation failure. The Cochrane Qualitative and Implementation Methods Group provide additional guidance on exploring implementation and theory failure that can be adapted to address aspects of complexity of complex interventions when implemented in health systems. 19

Health-system complexity-related questions that a synthesis of quantitative and qualitative evidence could address (derived from Petticrew et al 17 )

It may not be apparent which aspects of complexity or which elements of the complex intervention or health system can be explored in a guideline process, or whether combining qualitative and quantitative evidence in a mixed-method synthesis will be useful, until the available evidence is scoped and mapped. 17 20 A more extensive lead in phase is typically required to scope the available evidence, engage with stakeholders and to refine the review parameters and questions that can then be mapped against potential review designs and methods of synthesis. 20 At the scoping stage, it is also common to decide on a theoretical perspective 21 or undertake further work to refine a theoretical perspective. 22 This is also the stage to begin articulating the programme theory of the complex intervention that may be further developed to refine an understanding of complexity and show how the intervention is implemented in and impacts on the wider health system. 17 23 24 In practice, this process can be lengthy, iterative and fluid with multiple revisions to the review scope, often developing and adapting a logic model 17 as the available evidence becomes known and the potential to incorporate different types of review designs and syntheses of quantitative and qualitative evidence becomes better understood. 25 Further questions, propositions or hypotheses may emerge as the reviews progress and therefore the protocols generally need to be developed iteratively over time rather than a priori.

Following a scoping exercise and definition of key questions, the next step in the guideline development process is to identify existing or commission new systematic reviews to locate and summarise the best available evidence in relation to each question. For example, case study 2, ‘Optimising health worker roles for maternal and newborn health through task shifting’, included quantitative reviews that did and did not take an additional complexity perspective, and qualitative evidence syntheses that were able to explain how specific elements of complexity impacted on intervention outcomes within the wider health system. Further understanding of health system complexity was facilitated through the conduct of additional country-level case studies that contributed to an overall understanding of what worked and what happened when lay health worker interventions were implemented. See table 1 online supplementary file 2 .

There are a few existing examples, which we draw on in this paper, but integrating quantitative and qualitative evidence in a mixed-method synthesis is relatively uncommon in a guideline process. Box 2 includes a set of key questions that guideline developers and review authors contemplating combining quantitative and qualitative evidence in mixed-methods design might ask. Subsequent sections provide more information and signposting to further reading to help address these key questions.

Key questions that guideline developers and review authors contemplating combining quantitative and qualitative evidence in a mixed-methods design might ask

Compound questions requiring both quantitative and qualitative evidence?

Questions requiring mixed-methods studies?

Separate quantitative and qualitative questions?

Separate quantitative and qualitative research studies?

Related quantitative and qualitative research studies?

Mixed-methods studies?

Quantitative unpublished data and/or qualitative unpublished data, eg, narrative survey data?

Throughout the review?

Following separate reviews?

At the question point?

At the synthesis point?

At the evidence to recommendations stage?

Or a combination?

Narrative synthesis or summary?

Quantitising approach, eg, frequency analysis?

Qualitising approach, eg, thematic synthesis?

Tabulation?

Logic model?

Conceptual model/framework?

Graphical approach?

• WHICH: Which mixed-method designs, methodologies and methods best fit into a guideline process to inform recommendations?

Complexity-related questions that a synthesis of quantitative and qualitative evidence can potentially address

Petticrew et al 17 define the different aspects of complexity and examples of complexity-related questions that can potentially be explored in guidelines and systematic reviews taking a complexity perspective. Relevant aspects of complexity outlined by Petticrew et al 17 are summarised in table 2 below, together with the corresponding questions that could be addressed in a synthesis combining qualitative and quantitative evidence. Importantly, the aspects of complexity and their associated concepts of interest have however yet to be translated fully in primary health research or systematic reviews. There are few known examples where selected complexity concepts have been used to analyse or reanalyse a primary intervention study. Most notable is Chandler et al 26 who specifically set out to identify and translate a set of relevant complexity theory concepts for application in health systems research. Chandler then reanalysed a trial process evaluation using selected complexity theory concepts to better understand the complex causal pathway in the health system that explains some aspects of complexity in table 2 .

Rehfeuss et al 16 also recommends upfront consideration of the WHO-INTEGRATE evidence to decision criteria when planning a guideline and formulating questions. The criteria reflect WHO norms and values and take account of a complexity perspective. The framework can be used by guideline development groups as a menu to decide which criteria to prioritise, and which study types and synthesis methods can be used to collect evidence for each criterion. Many of the criteria and their related questions can be addressed using a synthesis of quantitative and qualitative evidence: the balance of benefits and harms, human rights and sociocultural acceptability, health equity, societal implications and feasibility (see table 3 ). Similar aspects in the DECIDE framework 15 could also be addressed using synthesis of qualitative and quantitative evidence.

Integrate evidence to decision framework criteria, example questions and types of studies to potentially address these questions (derived from Rehfeuss et al 16 )

GIS, Geographical Information System; RCT, randomised controlled trial.

Questions as anchors or compasses

Questions can serve as an ‘anchor’ by articulating the specific aspects of complexity to be explored (eg, Is successful implementation of the intervention context dependent?). 27 Anchor questions such as “How does intervention x impact on socioeconomic inequalities in health behaviour/outcome x” are the kind of health system question that requires a synthesis of both quantitative and qualitative evidence and hence a mixed-method synthesis. Quantitative evidence can quantify the difference in effect, but does not answer the question of how . The ‘how’ question can be partly answered with quantitative and qualitative evidence. For example, quantitative evidence may reveal where socioeconomic status and inequality emerges in the health system (an emergent property) by exploring questions such as “ Does patterning emerge during uptake because fewer people from certain groups come into contact with an intervention in the first place? ” or “ are people from certain backgrounds more likely to drop out, or to maintain effects beyond an intervention differently? ” Qualitative evidence may help understand the reasons behind all of these mechanisms. Alternatively, questions can act as ‘compasses’ where a question sets out a starting point from which to explore further and to potentially ask further questions or develop propositions or hypotheses to explore through a complexity perspective (eg, What factors enhance or hinder implementation?). 27 Other papers in this series provide further guidance on developing questions for qualitative evidence syntheses and guidance on question formulation. 14 28

For anchor and compass questions, additional application of a theory (eg, complexity theory) can help focus evidence synthesis and presentation to explore and explain complexity issues. 17 21 Development of a review specific logic model(s) can help to further refine an initial understanding of any complexity-related issues of interest associated with a specific intervention, and if appropriate the health system or section of the health system within which to contextualise the review question and analyse data. 17 23–25 Specific tools are available to help clarify context and complex interventions. 17 18

If a complexity perspective, and certain criteria within evidence to decision frameworks, is deemed relevant and desirable by guideline developers, it is only possible to pursue a complexity perspective if the evidence is available. Careful scoping using knowledge maps or scoping reviews will help inform development of questions that are answerable with available evidence. 20 If evidence of effect is not available, then a different approach to develop questions leading to a more general narrative understanding of what happened when complex interventions were implemented in a health system will be required (such as in case study 3—risk communication guideline). This should not mean that the original questions developed for which no evidence was found when scoping the literature were not important. An important function of creating a knowledge map is also to identify gaps to inform a future research agenda.

Table 2 and online supplementary files 1–3 outline examples of questions in the three case studies, which were all ‘COMPASS’ questions for the qualitative evidence syntheses.

Types of integration and synthesis designs in mixed-method reviews

The shift towards integration of qualitative and quantitative evidence in primary research has, in recent years, begun to be mirrored within research synthesis. 29–31 The natural extension to undertaking quantitative or qualitative reviews has been the development of methods for integrating qualitative and quantitative evidence within reviews, and within the guideline process using evidence to decision-frameworks. Advocating the integration of quantitative and qualitative evidence assumes a complementarity between research methodologies, and a need for both types of evidence to inform policy and practice. Below, we briefly outline the current designs for integrating qualitative and quantitative evidence within a mixed-method review or synthesis.

One of the early approaches to integrating qualitative and quantitative evidence detailed by Sandelowski et al 32 advocated three basic review designs: segregated, integrated and contingent designs, which have been further developed by Heyvaert et al 33 ( box 3 ).

Segregated, integrated and contingent designs 32 33

Segregated design.

Conventional separate distinction between quantitative and qualitative approaches based on the assumption they are different entities and should be treated separately; can be distinguished from each other; their findings warrant separate analyses and syntheses. Ultimately, the separate synthesis results can themselves be synthesised.

Integrated design

The methodological differences between qualitative and quantitative studies are minimised as both are viewed as producing findings that can be readily synthesised into one another because they address the same research purposed and questions. Transformation involves either turning qualitative data into quantitative (quantitising) or quantitative findings are turned into qualitative (qualitising) to facilitate their integration.

Contingent design

Takes a cyclical approach to synthesis, with the findings from one synthesis informing the focus of the next synthesis, until all the research objectives have been addressed. Studies are not necessarily grouped and categorised as qualitative or quantitative.

A recent review of more than 400 systematic reviews 34 combining quantitative and qualitative evidence identified two main synthesis designs—convergent and sequential. In a convergent design, qualitative and quantitative evidence is collated and analysed in a parallel or complementary manner, whereas in a sequential synthesis, the collation and analysis of quantitative and qualitative evidence takes place in a sequence with one synthesis informing the other ( box 4 ). 6 These designs can be seen to build on the work of Sandelowski et al , 32 35 particularly in relation to the transformation of data from qualitative to quantitative (and vice versa) and the sequential synthesis design, with a cyclical approach to reviewing that evokes Sandelowski’s contingent design.

Convergent and sequential synthesis designs 34

Convergent synthesis design.

Qualitative and quantitative research is collected and analysed at the same time in a parallel or complementary manner. Integration can occur at three points:

a. Data-based convergent synthesis design

All included studies are analysed using the same methods and results presented together. As only one synthesis method is used, data transformation occurs (qualitised or quantised). Usually addressed one review question.

b. Results-based convergent synthesis design

Qualitative and quantitative data are analysed and presented separately but integrated using a further synthesis method; eg, narratively, tables, matrices or reanalysing evidence. The results of both syntheses are combined in a third synthesis. Usually addresses an overall review question with subquestions.

c. Parallel-results convergent synthesis design

Qualitative and quantitative data are analysed and presented separately with integration occurring in the interpretation of results in the discussion section. Usually addresses two or more complimentary review questions.

Sequential synthesis design

A two-phase approach, data collection and analysis of one type of evidence (eg, qualitative), occurs after and is informed by the collection and analysis of the other type (eg, quantitative). Usually addresses an overall question with subquestions with both syntheses complementing each other.

The three case studies ( table 1 , online supplementary files 1–3 ) illustrate the diverse combination of review designs and synthesis methods that were considered the most appropriate for specific guidelines.

Methods for conducting mixed-method reviews in the context of guidelines for complex interventions

In this section, we draw on examples where specific review designs and methods have been or can be used to explore selected aspects of complexity in guidelines or systematic reviews. We also identify other review methods that could potentially be used to explore aspects of complexity. Of particular note, we could not find any specific examples of systematic methods to synthesise highly diverse research designs as advocated by Petticrew et al 17 and summarised in tables 2 and 3 . For example, we could not find examples of methods to synthesise qualitative studies, case studies, quantitative longitudinal data, possibly historical data, effectiveness studies providing evidence of differential effects across different contexts, and system modelling studies (eg, agent-based modelling) to explore system adaptivity.

There are different ways that quantitative and qualitative evidence can be integrated into a review and then into a guideline development process. In practice, some methods enable integration of different types of evidence in a single synthesis, while in other methods, the single systematic review may include a series of stand-alone reviews or syntheses that are then combined in a cross-study synthesis. Table 1 provides an overview of the characteristics of different review designs and methods and guidance on their applicability for a guideline process. Designs and methods that have already been used in WHO guideline development are described in part A of the table. Part B outlines a design and method that can be used in a guideline process, and part C covers those that have the potential to integrate quantitative, qualitative and mixed-method evidence in a single review design (such as meta-narrative reviews and Bayesian syntheses), but their application in a guideline context has yet to be demonstrated.

Points of integration when integrating quantitative and qualitative evidence in guideline development

Depending on the review design (see boxes 3 and 4 ), integration can potentially take place at a review team and design level, and more commonly at several key points of the review or guideline process. The following sections outline potential points of integration and associated practical considerations when integrating quantitative and qualitative evidence in guideline development.

Review team level

In a guideline process, it is common for syntheses of quantitative and qualitative evidence to be done separately by different teams and then to integrate the evidence. A practical consideration relates to the organisation, composition and expertise of the review teams and ways of working. If the quantitative and qualitative reviews are being conducted separately and then brought together by the same team members, who are equally comfortable operating within both paradigms, then a consistent approach across both paradigms becomes possible. If, however, a team is being split between the quantitative and qualitative reviews, then the strengths of specialisation can be harnessed, for example, in quality assessment or synthesis. Optimally, at least one, if not more, of the team members should be involved in both quantitative and qualitative reviews to offer the possibility of making connexions throughout the review and not simply at re-agreed junctures. This mirrors O’Cathain’s conclusion that mixed-methods primary research tends to work only when there is a principal investigator who values and is able to oversee integration. 9 10 While the above decisions have been articulated in the context of two types of evidence, variously quantitative and qualitative, they equally apply when considering how to handle studies reporting a mixed-method study design, where data are usually disaggregated into quantitative and qualitative for the purposes of synthesis (see case study 3—risk communication in humanitarian disasters).

Question formulation

Clearly specified key question(s), derived from a scoping or consultation exercise, will make it clear if quantitative and qualitative evidence is required in a guideline development process and which aspects will be addressed by which types of evidence. For the remaining stages of the process, as documented below, a review team faces challenges as to whether to handle each type of evidence separately, regardless of whether sequentially or in parallel, with a view to joining the two products on completion or to attempt integration throughout the review process. In each case, the underlying choice is of efficiencies and potential comparability vs sensitivity to the underlying paradigm.

Once key questions are clearly defined, the guideline development group typically needs to consider whether to conduct a single sensitive search to address all potential subtopics (lumping) or whether to conduct specific searches for each subtopic (splitting). 36 A related consideration is whether to search separately for qualitative, quantitative and mixed-method evidence ‘streams’ or whether to conduct a single search and then identify specific study types at the subsequent sifting stage. These two considerations often mean a trade-off between a single search process involving very large numbers of records or a more protracted search process retrieving smaller numbers of records. Both approaches have advantages and choice may depend on the respective availability of resources for searching and sifting.

Screening and selecting studies

Closely related to decisions around searching are considerations relating to screening and selecting studies for inclusion in a systematic review. An important consideration here is whether the review team will screen records for all review types, regardless of their subsequent involvement (‘altruistic sifting’), or specialise in screening for the study type with which they are most familiar. The risk of missing relevant reports might be minimised by whole team screening for empirical reports in the first instance and then coding them for a specific quantitative, qualitative or mixed-methods report at a subsequent stage.

Assessment of methodological limitations in primary studies

Within a guideline process, review teams may be more limited in their choice of instruments to assess methodological limitations of primary studies as there are mandatory requirements to use the Cochrane risk of bias tool 37 to feed into Grading of Recommendations Assessment, Development and Evaluation (GRADE) 38 or to select from a small pool of qualitative appraisal instruments in order to apply GRADE; Confidence in the Evidence from Reviews of Qualitative Research (GRADE-CERQual) 39 to assess the overall certainty or confidence in findings. The Cochrane Qualitative and Implementation Methods Group has recently issued guidance on the selection of appraisal instruments and core assessment criteria. 40 The Mixed-Methods Appraisal Tool, which is currently undergoing further development, offers a single quality assessment instrument for quantitative, qualitative and mixed-methods studies. 41 Other options include using corresponding instruments from within the same ‘stable’, for example, using different Critical Appraisal Skills Programme instruments. 42 While using instruments developed by the same team or organisation may achieve a degree of epistemological consonance, benefits may come more from consistency of approach and reporting rather than from a shared view of quality. Alternatively, a more paradigm-sensitive approach would involve selecting the best instrument for each respective review while deferring challenges from later heterogeneity of reporting.

Data extraction

The way in which data and evidence are extracted from primary research studies for review will be influenced by the type of integrated synthesis being undertaken and the review purpose. Initially, decisions need to be made regarding the nature and type of data and evidence that are to be extracted from the included studies. Method-specific reporting guidelines 43 44 provide a good template as to what quantitative and qualitative data it is potentially possible to extract from different types of method-specific study reports, although in practice reporting quality varies. Online supplementary file 5 provides a hypothetical example of the different types of studies from which quantitative and qualitative evidence could potentially be extracted for synthesis.

The decisions around what data or evidence to extract will be guided by how ‘integrated’ the mixed-method review will be. For those reviews where the quantitative and qualitative findings of studies are synthesised separately and integrated at the point of findings (eg, segregated or contingent approaches or sequential synthesis design), separate data extraction approaches will likely be used.

Where integration occurs during the process of the review (eg, integrated approach or convergent synthesis design), an integrated approach to data extraction may be considered, depending on the purpose of the review. This may involve the use of a data extraction framework, the choice of which needs to be congruent with the approach to synthesis chosen for the review. 40 45 The integrative or theoretical framework may be decided on a priori if a pre-developed theoretical or conceptual framework is available in the literature. 27 The development of a framework may alternatively arise from the reading of the included studies, in relation to the purpose of the review, early in the process. The Cochrane Qualitative and Implementation Methods Group provide further guidance on extraction of qualitative data, including use of software. 40

Synthesis and integration

Relatively few synthesis methods start off being integrated from the beginning, and these methods have generally been subject to less testing and evaluation particularly in a guideline context (see table 1 ). A review design that started off being integrated from the beginning may be suitable for some guideline contexts (such as in case study 3—risk communication in humanitarian disasters—where there was little evidence of effect), but in general if there are sufficient trials then a separate systematic review and meta-analysis will be required for a guideline. Other papers in this series offer guidance on methods for synthesising quantitative 46 and qualitative evidence 14 in reviews that take a complexity perspective. Further guidance on integrating quantitative and qualitative evidence in a systematic review is provided by the Cochrane Qualitative and Implementation Methods Group. 19 27 29 40 47

Types of findings produced by specific methods

It is highly likely (unless there are well-designed process evaluations) that the primary studies may not themselves seek to address the complexity-related questions required for a guideline process. In which case, review authors will need to configure the available evidence and transform the evidence through the synthesis process to produce explanations, propositions and hypotheses (ie, findings) that were not obvious at primary study level. It is important that guideline commissioners, developers and review authors are aware that specific methods are intended to produce a type of finding with a specific purpose (such as developing new theory in the case of meta-ethnography). 48 Case study 1 (antenatal care guideline) provides an example of how a meta-ethnography was used to develop a new theory as an end product, 48 49 as well as framework synthesis which produced descriptive and explanatory findings that were more easily incorporated into the guideline process. 27 The definitions ( box 5 ) may be helpful when defining the different types of findings.

Different levels of findings

Descriptive findings —qualitative evidence-driven translated descriptive themes that do not move beyond the primary studies.

Explanatory findings —may either be at a descriptive or theoretical level. At the descriptive level, qualitative evidence is used to explain phenomena observed in quantitative results, such as why implementation failed in specific circumstances. At the theoretical level, the transformed and interpreted findings that go beyond the primary studies can be used to explain the descriptive findings. The latter description is generally the accepted definition in the wider qualitative community.

Hypothetical or theoretical finding —qualitative evidence-driven transformed themes (or lines of argument) that go beyond the primary studies. Although similar, Thomas and Harden 56 make a distinction in the purposes between two types of theoretical findings: analytical themes and the product of meta-ethnographies, third-order interpretations. 48

Analytical themes are a product of interrogating descriptive themes by placing the synthesis within an external theoretical framework (such as the review question and subquestions) and are considered more appropriate when a specific review question is being addressed (eg, in a guideline or to inform policy). 56

Third-order interpretations come from translating studies into one another while preserving the original context and are more appropriate when a body of literature is being explored in and of itself with broader or emergent review questions. 48

Bringing mixed-method evidence together in evidence to decision (EtD) frameworks

A critical element of guideline development is the formulation of recommendations by the Guideline Development Group, and EtD frameworks help to facilitate this process. 16 The EtD framework can also be used as a mechanism to integrate and display quantitative and qualitative evidence and findings mapped against the EtD framework domains with hyperlinks to more detailed evidence summaries from contributing reviews (see table 1 ). It is commonly the EtD framework that enables the findings of the separate quantitative and qualitative reviews to be brought together in a guideline process. Specific challenges when populating the DECIDE evidence to decision framework 15 were noted in case study 3 (risk communication in humanitarian disasters) as there was an absence of intervention effect data and the interventions to communicate public health risks were context specific and varied. These problems would not, however, have been addressed by substitution of the DECIDE framework with the new INTEGRATE 16 evidence to decision framework. A d ifferent type of EtD framework needs to be developed for reviews that do not include sufficient evidence of intervention effect.

Mixed-method review and synthesis methods are generally the least developed of all systematic review methods. It is acknowledged that methods for combining quantitative and qualitative evidence are generally poorly articulated. 29 50 There are however some fairly well-established methods for using qualitative evidence to explore aspects of complexity (such as contextual, implementation and outcome complexity), which can be combined with evidence of effect (see sections A and B of table 1 ). 14 There are good examples of systematic reviews that use these methods to combine quantitative and qualitative evidence, and examples of guideline recommendations that were informed by evidence from both quantitative and qualitative reviews (eg, case studies 1–3). With the exception of case study 3 (risk communication), the quantitative and qualitative reviews for these specific guidelines have been conducted separately, and the findings subsequently brought together in an EtD framework to inform recommendations.

Other mixed-method review designs have potential to contribute to understanding of complex interventions and to explore aspects of wider health systems complexity but have not been sufficiently developed and tested for this specific purpose, or used in a guideline process (section C of table 1 ). Some methods such as meta-narrative reviews also explore different questions to those usually asked in a guideline process. Methods for processing (eg, quality appraisal) and synthesising the highly diverse evidence suggested in tables 2 and 3 that are required to explore specific aspects of health systems complexity (such as system adaptivity) and to populate some sections of the INTEGRATE EtD framework remain underdeveloped or in need of development.

In addition to the required methodological development mentioned above, there is no GRADE approach 38 for assessing confidence in findings developed from combined quantitative and qualitative evidence. Another paper in this series outlines how to deal with complexity and grading different types of quantitative evidence, 51 and the GRADE CERQual approach for qualitative findings is described elsewhere, 39 but both these approaches are applied to method-specific and not mixed-method findings. An unofficial adaptation of GRADE was used in the risk communication guideline that reported mixed-method findings. Nor is there a reporting guideline for mixed-method reviews, 47 and for now reports will need to conform to the relevant reporting requirements of the respective method-specific guideline. There is a need to further adapt and test DECIDE, 15 WHO-INTEGRATE 16 and other types of evidence to decision frameworks to accommodate evidence from mixed-method syntheses which do not set out to determine the statistical effects of interventions and in circumstances where there are no trials.

When conducting quantitative and qualitative reviews that will subsequently be combined, there are specific considerations for managing and integrating the different types of evidence throughout the review process. We have summarised different options for combining qualitative and quantitative evidence in mixed-method syntheses that guideline developers and systematic reviewers can choose from, as well as outlining the opportunities to integrate evidence at different stages of the review and guideline development process.

Review commissioners, authors and guideline developers generally have less experience of combining qualitative and evidence in mixed-methods reviews. In particular, there is a relatively small group of reviewers who are skilled at undertaking fully integrated mixed-method reviews. Commissioning additional qualitative and mixed-method reviews creates an additional cost. Large complex mixed-method reviews generally take more time to complete. Careful consideration needs to be given as to which guidelines would benefit most from additional qualitative and mixed-method syntheses. More training is required to develop capacity and there is a need to develop processes for preparing the guideline panel to consider and use mixed-method evidence in their decision-making.

This paper has presented how qualitative and quantitative evidence, combined in mixed-method reviews, can help understand aspects of complex interventions and the systems within which they are implemented. There are further opportunities to use these methods, and to further develop the methods, to look more widely at additional aspects of complexity. There is a range of review designs and synthesis methods to choose from depending on the question being asked or the questions that may emerge during the conduct of the synthesis. Additional methods need to be developed (or existing methods further adapted) in order to synthesise the full range of diverse evidence that is desirable to explore the complexity-related questions when complex interventions are implemented into health systems. We encourage review commissioners and authors, and guideline developers to consider using mixed-methods reviews and synthesis in guidelines and to report on their usefulness in the guideline development process.

Handling editor: Soumyadeep Bhaumik

Contributors: JN, AB, GM, KF, ÖT and ES drafted the manuscript. All authors contributed to paper development and writing and agreed the final manuscript. Anayda Portela and Susan Norris from WHO managed the series. Helen Smith was series Editor. We thank all those who provided feedback on various iterations.

Funding: Funding provided by the World Health Organization Department of Maternal, Newborn, Child and Adolescent Health through grants received from the United States Agency for International Development and the Norwegian Agency for Development Cooperation.

Disclaimer: ÖT is a staff member of WHO. The author alone is responsible for the views expressed in this publication and they do not necessarily represent the decisions or policies of WHO.

Competing interests: No financial interests declared. JN, AB and ÖT have an intellectual interest in GRADE CERQual; and JN has an intellectual interest in the iCAT_SR tool.

Patient consent: Not required.

Provenance and peer review: Not commissioned; externally peer reviewed.

Data sharing statement: No additional data are available.

Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.