Yi Han, PhD, Cello Health, Yardley, PA, USA; Jason McDonough, PhD, ISMPP CMPP™, Cello Health Communications, Yardley, PA, USA
Evidence-based healthcare decision making is gaining global acceptance in both healthcare policy and clinical practice. Randomized controlled trials (RCTs) are the “gold standard” of evidence, but these studies are usually limited to comparison of two treatments, and thus cannot represent the entirety of treatment options or patient populations. Furthermore, there is often more than one trial comparing two treatment options, producing outcomes that may vary in magnitude or direction.
Creating a complete picture of the evidence requires consideration of available data through a systematic review of the literature and then combining the results in a complex network. The statistical tools used to create these comparisons are meta-analysis and mixed treatment comparison (MTC) analysis, which can be incredibly powerful, but there are many points in the process where biases can be introduced that influence the outcomes of the analysis.
There are two key circumstances in which meta-analysis (with or without MTC) is commonly used (Table 1):
- to aggregate data comparing two treatments across multiple studies, thus increasing the power of the findings; and
- to make indirect comparisons among three or more possible treatments to compare effect sizes among treatments for which direct comparisons are lacking.
When applied in the context of marketed pharmaceuticals, meta-analysis and MTC are most commonly used to support health economic evaluations and health technology assessments (HTAs; the systematic assessment of health interventions/technologies to inform policy decision making, particularly reimbursement).
Table 1. Prototypical Situations in Which Meta-analysis and Mixed Treatment Comparisons Are Applied
| Included studies | Rationale or outcome | |
| Meta-analysis |
|
|
| Mixed treatment comparison |
|
|
Process of Meta-analysis and MTC, and Publishing of Results
Although the terminology of “meta-analysis” originated in the 1970s, the conceptual underpinnings were applied in the early 1900s, to examine outcomes across studies of typhoid inoculation (Pearson K. BMJ. 1904;2:1243-1246). Meta-analysis begins with identifying appropriate studies for inclusion via a systematic literature review. Processes and procedures for systematic literature analysis have been described by the Cochrane Group and in guidance provided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) group (Figure 1).
A systematic review must be designed in much the same manner as a clinical trial, with prespecified populations, treatments, endpoints/outcomes, and analyses. Each aspect of the search and subsequent analysis must be documented.
Figure 1. The PRISMA Statement: flow of information through the different phases of a systematic review. Moher D, et al. PLoS Med. 2009;6(7): e1000097.
[This image is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Quality graphics are available here: http://www.prisma-statement.org/PRISMAStatement/FlowDiagram.aspx.]
Several good narrative reviews (eg, Crowther M, et al. Blood. 2010;116(17):3140-3146, Haidich AB. Hippokratia. 2010;14(Suppl 1):29-37) summarize approaches to meta-analysis and MTC, but key elements include evaluation of study quality (using tools such as the Jadad scale), and extracting data from each study in a blinded and prespecified manner. The correct statistical framework must be selected (eg, fixed-effects or random-effects models), as well as analyses for heterogeneity, sensitivity, and publication bias. Depending on the quality and number of studies on a given treatment, a true meta-analysis or MTC may not be statistically appropriate because of study quality, heterogeneity, or publication bias.
When publishing results from systematic reviews and meta-analyses, a rigorous set of reporting guidelines should be followed (eg, the PRISMA Statement). Individual journals, academic institutions (eg, Duke), and the Cochrane Collaboration will also have guidance/requirements. Following these recommendations is essential to ensuring that the analysis is reproducible, transparent, and an unbiased assessment of the medical literature.
Benefits and Challenges of Meta-analysis and MTC
The most important benefit of meta-analysis and MTC is utilizing all available data to guide decision making, rather than relying on the results of one study (generally the largest RCT). The underlying statistical framework applies an objective weight to each trial based on size and quality. MTC can fill data gaps when head-to-head trial data are lacking, and can facilitate decision making.
Both meta-analysis and MTC are highly affected by publication bias. Although many are working for change, positive data tend to be more likely to get published, and negative or neutral data may be less likely to be considered sufficiently medically interesting to warrant publication (by authors or journals). Published data are the core inputs to meta- and MTC analysis; therefore, analysis outcome can inherit this bias.
Meta-analysis and MTC methods are based on the assumptions of similarity, homogeneity, and consistency across studies or within a population. The assumption of homogeneity requires that included trials are measuring the same treatment effect for the same population. The assumption of similarity for indirect comparisons in MTC requires that response to treatment is similar across trials. The consistency assumption provides that indirect effects extrapolated in MTC will be consistent with direct head-to-head trial data.
It is difficult to ensure that all these assumptions are valid, and there is no universally accepted standard for measuring how well these assumptions hold. However, the best way to ensure a high-quality analysis is to conduct a high-quality systematic literature review.
Importance to Medical Publication Professionals
Meta-analysis is an important tool of evidence-based medicine that relies on clear and transparent publication of clinical trial data. It is, therefore, important for the community of medical publication professionals to ensure effective data dissemination, and understand how to appropriately utilize and interpret findings from meta-analyses and MTCs.