The PRISMA Extension Statement for Reporting of Systematic Reviews Incorporating Network Meta-analyses of Health Care Interventions: Checklist and ExplanationsFREE
Submit a Comment
Contributors must reveal any conflict of interest. Comments are moderated. Please see our information for authorsregarding comments on an Annals publication.
Abstract
Development of the PRISMA Network Meta-analysis Extension Statement
Scope of This Extension Statement
How to Use This Document
What Is a Treatment Network?
Discussion
Appendix: The PRISMA Network Meta-analysis Extension Statement
Title and Abstract
Item 1: Title
Different combined oral contraceptives and the risk of venous thrombosis: systematic review and network meta-analysis. (61)
Network meta-analysis on randomized trials focusing on the preventive effect of statins on contrast-induced nephropathy. (62)
Item 2: Structured Summary
Objective. To determine the comparative effectiveness and safety of current maintenance strategies in preventing exacerbations of asthma.Design. Systematic reviewand network meta-analysisusing Bayesian statistics.Data Sources. Cochrane systematic reviewson chronic asthma, complemented by an updated search when appropriate.Eligibility Criteria. Trials of adults with asthma randomised to maintenance treatments of at least 24 weeks duration and that reported on asthma exacerbations in full text. Low dose inhaled corticosteroid treatment was the comparator strategy. The primary effectiveness outcome was the rate of severe exacerbations. The secondary outcome was the composite of moderate or severe exacerbations. The rate of withdrawal was analysed as a safety outcome.Results. 64 trials with 59,622 patient years of follow-up comparing 15 strategies and placebo were included. For prevention of severe exacerbations, combined inhaled corticosteroids and long acting β-agonists as maintenance and reliever treatment and combined inhaled corticosteroids and long acting β-agonists in a fixed daily dose performed equally well and were ranked first for effectiveness. The rate ratios compared with low dose inhaled corticosteroids were 0.44 (95% CrI 0.29 to 0.66) and 0.51 (0.35 to 0.77), respectively. Other combined strategies were not superior to inhaled corticosteroids and all single drug treatments were inferior to single low dose inhaled corticosteroids. Safety was best for conventional best (guideline based) practice and combined maintenance and reliever therapy.Conclusions. Strategies with combined inhaled corticosteroids and long acting β-agonists are most effective and safe in preventing severe exacerbations of asthma, although some heterogeneity was observed in this network meta-analysisof full text reports.
Introduction
Item 3: Rationale
Although progress has been achieved in the field and patients live longer, the relative merits of the many different chemotherapy and targeted treatment regimens are not well understood. Hundreds of trials have been conducted to compare treatments for advanced breast cancer, but because each has compared only two or a few treatments, it is difficult to integrate information on the relative efficacy of all tested regimens. This integration is important because different regimens vary both in cost and in toxicity. Therefore, we performed a comprehensive systematic review of chemotherapy and targeted treatment regimens in advanced breast cancer and evaluated through a multiple-treatments meta-analysis the relative merits of the many different regimens used to prolong survival in advanced breast cancer patients. (67)
Item 4: Objectives
Methods
Item 5: Protocol and Registration
Item 6: Eligibility Criteria
Our analysis classified fluids as crystalloids (divided into balanced and unbalanced solutions) and colloids (divided into albumin, gelatin, and low- and high-molecular weight hydroxyethyl starch [HES] [threshold molecular weight, 150 000 kDa]). We considered fluid balanced if it contained an anion of a weak acid (buffer) and its chloride content was correspondingly less than in 0.9% sodium chloride. The relevant analyses were a 4-node NMA [network meta-analysis] (crystalloids vs. albumin vs. HES vs. gelatin), a 6-node NMA (crystalloids vs. albumin vs. HES vs. gelatin, with crystalloids divided into balanced or unbalanced and HES divided into low or high molecular weight), and a conventional direct frequentist fixed effects meta-analytic comparison of crystalloids versus colloids. (68)
Item 7: Information Sources
Item 8: Search
Item 9: Study Selection
Item 10: Data Collection Process
Item 11: Data Items
Item S1 (New Item): Review of Network Geometry
We analyzed published and unpublished randomized trials performed in patients with pulmonary hypertension. At the level of drug classes, we examined whether head-to-head comparisons are between agents in the same class or between agents in different classes. At the level of companies, we examined whether trials involve only agents (as active comparators or backbones) owned by the same company, or include treatments by different companies. In the networks of drug comparisons, each drug is drawn by a node and randomized comparisons between drugs are shown by links between the nodes. When a drug is compared against the same agent in different dose or formulation, this is represented by an auto-loop. In the networks of companies, nodes stand for companies and auto-loops around these nodes represent trials involving agents of a single company. Links between different nodes characterize trials comparing agents that belong to different companies. ( 70)
Item 12: Risk of Bias in Individual Studies
Item 13: Summary Measures
For each pairwise comparison and each outcome at each time point, we used odds ratios (OR) with 95% confidence intervals (95% CIs) as a measure of the association between the treatment used and efficacy. As the outcomes are negative, ORs >1 correspond to beneficial treatment effects of the first treatment compared with the second treatment.
As a measure that reflects ranking and the uncertainty, we used the Surface Under the Cumulative RAnking curve (SUCRA) as described in Salanti 2011. This measure, expressed as percentage, showed the relative probability of an intervention being among the best options. (82)
Item 14: Planned Methods of Analysis
The network meta-analysis was based on a bayesian random effects Poisson regression model, which preserves randomised treatment comparisons within trials. The model uses numbers of patients experiencing an event and accumulated patient years to estimate rate ratios. The specification of nodes in the network was based on the randomised intervention or in case of strategy trials, such as COURAGE [Clinical Outcomes Utilizing Revascularization and Aggressive Drug Evaluation] or FAME-2 [Fractional flow reserve versus Angiography for Multi-Vessel Evaluation], on the intervention received by the majority of patients in a trial arm. Analyses were performed using Markov-Chain Monte-Carlo methods. The prior distribution for treatment effects was minimally informative: a normal distribution with a mean of 1 and a 95% reference range from 0.01 to 100 on a rate ratio scale. The prior for the between trial variance τ2, which we assumed to be equal across comparisons, was based on empirical evidence derived from semi-objective outcomes of head to head comparisons: a log normal distribution with a geometric mean of τ2 of 0.04 and a 95% reference range from 0.001 to 1.58. Rate ratios were estimated from the median and corresponding 95% credibility intervals from the 2.5th and 97.5th centiles of the posterior distribution. Convergence was deemed to be achieved if plots of the Gelman-Rubin statistics indicated that widths of pooled runs and individual runs stabilised around the same value and their ratio was around 1. (83)
Item S2 (New Item): Assessment of Inconsistency
Consistency was mainly assessed by the comparison of the conventional network meta-analysis model, for which consistency is assumed, with a model that does not assume consistency (a series of pairwise meta-analyses analysed jointly). If the trade-off between model fit and complexity favoured the model with assumed consistency, this model was preferred. Moreover, we calculated the difference between direct and indirect evidence in all closed loops in the network; inconsistent loops were identified with a significant (95% CrI that excludes 0) disagreement between direct and indirect evidence. A loop of evidence is a collection of studies that links treatments to allow for indirect comparisons; the simplest loop is a triangle formed by three direct comparison studies with shared comparators. (88)
Item 15: Risk of Bias Across Studies
Item 16: Additional Analyses
We considered how decisions to group glaucoma treatments could affect the transitivity assumption and interpretation of the analysis. (27) [See Appendix Figure 1.]
We a priori had selected allocation concealment, assessor blinding, treatment fidelity and imputation of numbers of responders as potentially important effect modifiers to be examined in sensitivity analyses to limit the included studies to those at low risk of bias. We conducted additional meta-regression analyses using random effects network meta-regression models to examine potential effect moderators such as the mean age of participants, the type of rating scales (clinician-rated versus self-rated), publication status (published versus dissertation), and therapy format (individual vs group). (94)
Random effects network meta-analyses with informative priors for heterogeneity variances were conducted for the analyses. We also conducted fixed and random effects models with vague priors. (95)
Results
Item 17: Study Selection
Item S3 (New Item): Presentation of Network Geometry
Appendix Figure 2 shows a network graph comparing antipsychotic agents for prevention of schizophrenia relapse (12).
Item S4 (New Item): Summary of Network Geometry
A total of 2,545 pulmonary hypertension patients received active pulmonary hypertension medication. The studied agents were more commonly bosentan (n = 13 trials; patients receiving treatment = 633) and sildenafil (n = 13 trials; patients receiving treatment = 593). Placebo was used as the comparator arm in 38 studies (patients receiving placebo = 1,643). Of the patients that received placebo, 52 participants were part of crossover studies with sildenafil. The most frequently used comparisons were bosentan versus placebo (n = 11) and sildenafil versus placebo (n = 11). Studies that used placebo as the comparator arm (n = 38) were for the most part sponsored by the pharmaceutical company that owned the product (n = 28 studies [74%]). The only two published head-to-head comparisons of different medications (sildenafil against bosentan) were not sponsored by pharmaceutical companies, but by the British Heart Foundation and the Italian Health Authority. (70)
Item 18: Study Characteristics
Item 19: Risk of Bias Within Studies
Item 20: Results of Individual Studies
The Appendix Table presents an example of one possible approach to provision of data on mortality observed with five different interventions for treatment of left ventricular dysfunction (medical resynchronisation, cardiac resynchronisation, implantable defibrillator, combined resynchronisation and defibrillator, and amiodarone) as described elsewhere (98).
Item 21: Synthesis of Results
Item S5 (New Item): Exploration for Inconsistency
The assumption of consistency was generally supported by a better trade-off between model fit and complexity when consistency was assumed than when it was not. Significant disagreement between direct and indirect estimates (inconsistency) was identified in only very few cases: for efficacy seven of 80 loops; for all-cause discontinuation three of 80 loops; for weight gain one of 62 loops; for extrapyramidal side-effects one of 56 loops; for prolactin increase three of 44 loops; for QTc prolongation two of 35 loops; and for sedation none of 49 loops were inconsistent (appendix pp 105-14). Data were double-checked and we could not identify any important variable that differed across comparisons in these loops. The number of included studies in the inconsistent loops was typically small, so the extent of inconsistency was not substantial enough to change the results. (88)
Item 22: Risk of Bias Across Studies
Item 23: Results of Additional Analyses
Standard adjusted dose vitamin K agonist (VKA) (odds ratio 0.11 (95% credible interval 0.04 to 0.27)), dabigatran, apixaban 5 mg, apixaban 2.5 mg, and rivaroxaban decreased the risk of recurrent venous thromboembolism, compared with ASA [acetylsalicylic acid]. Compared with low dose VKA, standard adjusted dose VKA reduced the risk of recurrent venous thromboembolism (0.25 (0.10 to 0.58)).
An appendix presents a detailed explanation for the potential discrepancy between ASA and placebo results. Results for most class level analyses also aligned with those reported previously in the treatment level analysis. Subgroup analyses, performed to account for heterogeneity due to study duration, yielded results that were more favourable for ASA than those obtained from the primary analysis. However, results for ASA were still less pronounced than those reported for other treatments (standard adjusted dose VKA, low intensity VKA, and dabigatran) that remained in the evidence network. Sensitivity analysis excluding ximelagatran from the analysis did not change the results reported. (95 )
Table 2 presents an investigation into potential sources of variation in people with diabetes in the network. Estimates of relative risk comparing sirolimus eluting stents with paclitaxel eluting stents depended to some extent on the quality of the trials, the length of followup, and the time of completion of patient recruitment (table 2), but 95% credibility intervals were wide and tests for interaction negative (P for interaction ≥0.16). The estimated relative risk of death when sirolimus eluting stents were compared with bare metal stents was greater when the specified duration of dual antiplatelet therapy was less than six months (2.37, 95% credibility interval 1.18 to 5.12) compared with six months or longer (0.89, 0.58 to 1.40, P for interaction 0.02), however. (106)
None of the regression coefficients of the meta-regression examining possible effect moderators turned out to be statistically significant [-0.024 (95% CI -0.056 to 0.006) for age, -0.899 (95% CI -1.843 to 0.024 for rating scale), -0.442 (95% CI -1.399 to 0.520) for publication status, and 0.004 (95% CI -0.798 to 0.762) for therapy format]. (94)
Discussion
Item 24: Summary of Evidence
Item 25: Limitations
Our study has several limitations. The network could be expanded to old drugs such as perphenazine and sulpiride, which have had good results in effectiveness studies, but only a few relevant perphenazine trials have been done.
Reporting of side-effects is unsatisfactory in randomised controlled trials in patients with psychiatric disorders, and some side-effects were not recorded at all for some drugs. The meta-regression with percentage of withdrawals as a moderator could not rule out all potential bias associated with high attrition in schizophrenia trials.
Our findings cannot be generalised to young people with schizophrenia, patients with predominant negative symptoms, refractory patients, or stable patients, all of whom were excluded to enhance homogeneity as required by multiple-treatments meta-analysis. A funnel plot asymmetry was seen, which is not necessarily the expression of publication bias, but rather of higher efficacy in small trials than in larger ones, for various reasons. For example, sample size estimates for drugs with low efficacy might have needed higher numbers of participants to attain statistical significance than in trials with more effective drugs. However, accounting for trial size did not substantially change the rankings. Finally, because multiple-treatments meta-analysis requires reasonably homogeneous studies, we had to restrict ourselves to short-term trials. Because schizophrenia is often a chronic disorder, future multiple-treatments meta-analyses could focus on long-term trials, but these remain scarce. In any case, for clinicians to know to which drugs patients are most likely to respond within a reasonable duration such as 6 weeks is important. (88)
Item 26: Conclusions
Item 27: Funding
Use of Supplemental Appendices for Complete Reporting of Network Meta-analyses
Software for Implementing Network Meta-analysis
Example Wording for Endorsing This PRISMA Extension
References
Comments
Sign In to Submit A CommentInformation & Authors
Information
Published In
History
Keywords
Copyright
Authors
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. For an editable text file, please select Medlars format which will download as a .txt file. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
The PRISMA Extension Statement for Reporting of Systematic Reviews Incorporating Network Meta-analyses of Health Care Interventions: Checklist and Explanations. Ann Intern Med.2015;162:777-784. [Epub 2 June 2015]. doi:10.7326/M14-2385
View More
Get Access
Login Options:
Purchase
You will be redirected to acponline.org to sign-in to Annals to complete your purchase.
Access to EPUBs and PDFs for FREE Annals content requires users to be registered and logged in. A subscription is not required. You can create a free account below or from the following link. You will be redirected to acponline.org to create an account that will provide access to Annals. If you are accessing the Free Annals content via your institution's access, registration is not required.
Create your Free Account
You will be redirected to acponline.org to create an account that will provide access to Annals.
PRISMA Considerations and Searching the Literature
While we agree with the practices described Drs. Tian, Ge and Li describe regarding searching, we feel they are entirely applicable to traditional systematic reviews and are addressed in the PRISMA Statement’s explanation and elaboration article.(4) The main intent of the PRISMA extension statement for reporting of network meta-analysis is to focus on items that were not addressed in PRISMA and differ substantially from practices for traditional systematic reviews and meta-analyses.
Regarding their first suggestion, we feel ensuring the need for a proposed review is equally important for traditional reviews, and in all cases searching for existing literature should be preceded by in-depth consideration of the clinical importance of the research question in a PICOS (Population-Intervention-Comparator(s)-Outcome(s)-Study design) framework.
Regarding their second suggestion, we believe it has long been common for researchers undertaking reviews of multiple forms to inspect bibliographies of past reviews and included studies as a source for potentially relevant studies; the PRISMA Statement’s Explanation and Elaboration document addresses this issue in Item 7, suggesting ‘In addition to searching databases, authors should report the use of supplementary approaches to identify studies, such as hand searching of journals, checking reference lists, searching trials registries or regulatory agency Web sites, contacting manufacturers, or contacting authors.’(4)
Lastly, we agree that the increased number of interventions in a network meta-analysis can heighten the challenge of completing the systematic search strategy for a review. However, this may also often be true of other reviews not involving a multi-treatment question; for example, reviews involving more than one indication of relevance, or reviews involving complex interventions. In these and other scenarios, we support the practice of peer review of literature searches to maximize their quality. This is addressed in Item 8 in the PRISMA Explanations and Elaborations Statement: ‘We encourage authors to state whether search strategies were peer reviewed as part of the systematic review process.’(4)
Therefore, we support these practices noted by Tian, Ge, and Li. However we feel their importance and existing practice amongst researchers extend to many additional types of reviews, and that guidance from the PRISMA Statement remains highly relevant.
The authors also suggest a potential need for a second guidance document addressing reporting for Bayesian network meta-analyses; we disagree with this perspective at this time. The examples and elaborations provided in our guidance address reporting considerations for the key items that were suggested, while we do not foresee a need for certain suggested components such as specification of starting values or the number of iterations used (we recommend provision of details for convergence assessment already). We are unclear as to the authors’ intended meaning of suggesting sample size, however we hypothesize this is a reference to statistical power in network meta-analyses. We agree this can be of interest, and may be especially so for outcomes with few events. Some research has been conducted in this area,(5) although additional research is needed to inform considerations for reporting guidance.
We believe the current guidance provides a strong set of minimum reporting items for Frequentist and Bayesian NMAs, while authors are certainly encouraged to provide additional information of relevance to readers to support their reviews. As methodologies continue to evolve in this rapidly developing area, we will continue to gather materials for a possible future update of this extension statement which may include guidance that additional statistical considerations be reported.
Brian Hutton, PhD; David Moher, PhD
Ottawa Hospital Research Institute, Ottawa, Canada;
University of Ottawa School of Epidemiology, Public Health and Preventive Medicine, Ottawa, Canada
Chris Cameron, PhD
Ottawa Hospital Research Institute, Ottawa, Canada;
Cornerstone Research Group Inc., Burlington, Canada
Reference List
(1) Tian J, Ge L, and Li L. Searching for previous published and unpublished or ongoing systematic reviews/meta-analyses is very important (Commentary). Annals of Internal Medicine. 2015.
(2) Ge L, Tian J, Li L, and Yang K. The PRISMA Extension Statement for Statistical Analysis Reporting of Network Meta-Analysis is Needed (Commentary). Annals of Internal Medicine. 14-6-2015.
(3) Hutton B, Salanti G, Caldwell D, Schmid C, Chaimani A, Cameron C, Ioannidis J, and et al. The PRISMA Extension Statement for Reporting of Systematic Reviews Incorporating Network Meta-Analyses of Healthcare Interventions: Checklist and Explanations. Annals of Internal Medicine 162(11), 777-784. 2015.
(4) Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche P, Ioannidis J, Clarke M, Devereau PJ, Kleijnen J, and Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Annals of Internal Medicine 151(4), W65-W94. 2009.
(5) Thorlund K and Mills E. Sample size and power considerations in network meta-analysis. Systematic Reviews 1(41. doi: 10.1186/2046-4053-1-41). 2012.
Searching for previous published and unpublished or ongoing systematic reviews/meta-analyses is very important.
We declare that we have no conflicts of interest.
Jin-hui Tian, PhD
Evidence-Based Medicine Center of Lanzhou University, Lanzhou 730000, China.
Key Laboratory of Evidence-based Medicine and Clinical Translational Research of Gansu Province, Lanzhou 730000.
Long Ge,MD; Lun Li, PhD
The First Clinical Medicine College of Lanzhou University, Lanzhou 730000, China;
Evidence-Based Medicine Center of Lanzhou University, Lanzhou 730000, China;
Key Laboratory of Evidence-based Medicine and Clinical Translational Research of Gansu Province, Lanzhou 730000.
(1)Hutton B, Salanti G, Caldwell DM, Chaimani A, Schmid CH, Cameron C, et al. The PRISMA Extension Statement for Reporting of Systematic Reviews Incorporating Network Meta-analyses of Health Care Interventions: Checklist and Explanations.Ann Intern Med. 2015;162(11):777-84.
(2)Golger S, Wright K. Searching for evidence.In: Biondi-Zoccai G, editor. Network meta-analysis: evidence synthesis with mixed treatment comparison. New York: Nova Science, 2014. P. 63-76.
(3)Li L, Tian J, Tian H, Moher D, Liang F, Jiang T, et al. Network meta-analyses could be improved by searching more sources and by involving a librarian. J Clin Epidemiol. 2014; 67(9):1001-7.
The PRISMA Extension Statement for Statistical Analysis Reporting of Network Meta-Analysis is Needed
Methods section:
The detail of sample size calculation
Direct comparison: Assessment of heterogeneity, model of pooling data,summary measure, assessment of publication bias, sensitivity analysis, other analysis, software applied.
Network meta-analysis: Assessment of heterogeneity, adjustment for covariates, adjustment of multiple arms, code applied, selection of prior distribution, selection of fixed or random effect model, selection of consistency or inconsistency, assessment of inconsistency, assessment of convergence, summary measure (including treatment ranking), assessment of publication bias, sensitivity analysis (based on prior distribution or other), other analysis, software applied.
Results section:
The results of sample size calculation
Direct comparison: Results of heterogeneity assessment, model applied, results of direct comparison, publication bias assessment, sensitivity analysis and other analysis.
Network meta-analysis: Methods and results of heterogeneity, results of model fit tested, number of chains, the staring values for sampling, number of iterations per chain, number of iteration used for final results, results of convergence assessment, prior distributions used, results of indirect comparison, results of network meta-analysis, results of inconsistency assessment, results of ranking, publication bias assessment, sensitivity analysis, other analysis.
The details for each item should also be described in published papers. However, those items are based on published literature. Delphi survey, and face-to-face discussion and consensus meeting should be established to develop another PRISMA extension statement for reporting of statistical analysis and assumptions of NMA (PRISMA-S). We strongly believe that it will play very important parts to improve the quality of NMAs.
We declare that we have no conflicts of interest.
Long Ge
The First Clinical Medicine College of Lanzhou University, Lanzhou 730000, China;
Evidence-Based Medicine Center of Lanzhou University, Lanzhou 730000, China.
Jin-hui Tian, Lun Li, Ke-hu Yang*
Evidence-Based Medicine Center of Lanzhou University, Lanzhou 730000, China;
Key Laboratory of Evidence-based Medicine and Clinical Translational Research of Gansu Province, Lanzhou 730000.
*[email protected]
(1)Hutton B, Salanti G, Caldwell DM, Chaimani A, Schmid CH, Cameron C, et al. The PRISMA Extension Statement for Reporting of Systematic Reviews Incorporating Network Meta-analyses of Health Care Interventions: Checklist and Explanations.Ann Intern Med. 2015;162(11):777-84.
(2)Li L, Tian JH, Yang KH. Current situation of reporting statement for network meta-analysis.Chin J Evid Based Pediatr.2014;9(6):467-71.
(3)Tan SH, Bujkiewicz S, Sutton A, Dequen P, Cooper N.Presentational approaches used in the UK for reporting evidence synthesis using indirect and mixed treatment comparisons.J Health Serv Res Policy. 2013;18(4):224-32.