Chris Carswell, Adis Journals, Springer Nature, Auckland, New Zealand; Michael Drummond, Centre for Health Economics, University of York, York, UK; Don Husereau, School of Epidemiology and Public Health, University of Ottawa, Ontario, Canada; Institute of Health Economics, Alberta, Canada
Conflicts of Interest:
CC, DH and MD are co-authors of the CHEERS 2022 statement. CC is co-Editor in Chief of PharmacoEconomics. MD is co-Editor in Chief of Value in Health.
Introducing a new way to reach the authors! Email your questions and comments to TheMAP@ismpp.org.
The transparent reporting of economic evaluations is growing in importance, now that an increasing number of healthcare payers require health sciences companies to demonstrate that their products are not only clinically effective, but also cost-effective. Members of ISMPP will be very familiar with the need for adequate reporting of clinical studies, but economic evaluations pose additional challenges. First, the methodological standards for economic evaluation are not as universally agreed as those for clinical trials. Secondly, there are often suspicions of potential bias in the reporting of economic evaluations, as they have a more obvious commercial objective.[1] Therefore, it is especially important that companies know how to publish the studies they sponsor in a fair and transparent manner.
The update of the Consolidated Health Economic Evaluation Reporting Standards (CHEERS 2022) was published one year ago, in January 2022 (ISPOR – CHEERS). The CHEERS guidelines are part of the EQUATOR Network of guidelines, alongside CONSORT, PRISMA and STROBE, and have been endorsed by 15 biomedical and health services research journals.[2,3] In this article, we use CHEERS to give a few top tips on how to achieve transparent reporting of economic evaluations and to make sure the studies are accepted for publication in peer reviewed journals. The CHEERS checklist (See Table 1) consists of 28 reporting items. All are important, but we have selected 6 that we feel are particularly important in the quest for transparency and credibility.
Item 3: Give the context for the study, the study question, and its practical relevance for decision making in policy or practice
There has been increasing focus in recent years on measuring and maximizing the impact of research.[4] For economic evaluations to have impact, it needs to be clear to the readership what specific policy, reimbursement decision or individual prescribing decision is being informed; we recommend this is done in the Introduction. Commonly, economic evaluations are used to inform reimbursement decisions, and it is desirable that such decisions have yet to take place at the time of submission of an economic evaluation to a journal.
It is important that the decision problem is fully characterised, including the setting and location, study perspective (a viewpoint on the types of costs and benefits to be included in an economic evaluation) and the interventions being compared. Importantly, particular attention should be paid to accurately describing the study perspective in the US given that there are many different payers. Studies that do not have clear practical relevance to real world decision making are unlikely to be prioritised for review at high impact journals.
Top Tip: Make the practical relevance of your paper to future real-world decision making clear.
Item 4: Indicate whether a health economic analysis plan was developed and where available
Health economic analysis plans (HEAPs) are an emerging concept, particularly in the UK, [5] and intended to be used similarly to clinical trial or systematic review protocols. While there are no internationally recognized registries or templates for HEAPs, there are ongoing initiatives that aim for their regular development as a desirable goal. HEAPs can be published as a separate report and referenced afterwards (see [6] for example) or provided as a supplement to an article (e.g. [7]).
The existence of a pre-specified HEAP may give the reader some assurance that the published economic evaluation was well planned and that the end results are not a fishing expedition. However, don’t panic if you don’t have a HEAP or if the HEAP may be difficult to share. Currently, high impact journals are unlikely to require a HEAP, and CHEERS simply wants authors to report whether or not one exists and whether it can be made available to readers.
Top Tip: HEAPs are new so don’t panic if there’s no HEAP. Just say so. Similarly, if there is a HEAP but it can’t be shared, just say so. A good place to report these facts is at the beginning of the Methods section.
Item 7: Describe the interventions or strategies being compared and why chosen
Economic evaluation always involves a comparison of treatments, health interventions, or strategies. Therefore, if we say that ‘drug X is cost-effective’, this is always assessed in comparison with an alternative treatment (for a given category of patients in a given setting). Thus, a major source of potential bias in economic evaluations is to compare the treatment of interest (ie, the company’s new drug) with an inappropriate alternative. For example, a new drug may be cost-effective in comparison with an inferior, outmoded therapy, but not cost-effective when compared with another new therapy for the given health condition.
Therefore, it is extremely important to be explicit about the comparator(s) used in the study and why these were chosen. The most relevant comparator is normally the current standard of care for the patients concerned in the setting in which the study is being conducted. If the current standard of care involves a range of therapies and only one is included in the study, the reasons for this need to be given. If reviewers of the paper believe that a relevant treatment option has been excluded without good reason, they are likely to recommend rejection of the paper.
It is also important to give a precise description of the alternatives considered in the study. Typically, the new intervention (of interest) is described in detail, but ‘current standard of care’ is often inadequately defined. An adequate description is important, as it helps readers of the study to form a judgment on whether the results apply in their setting.
Top Tip: Give an adequate justification for the comparator(s) used in the study.
Item 14: Describe how costs were valued
Compared with the measurement and valuation of the benefits of treatments, the assessment of costs is often viewed as being a bit boring. Consequently, the measurement and valuation of costs is often not described in a very transparent way, and the sources of cost estimates are sometimes not given. Remember that the validity of a cost-effectiveness estimate is equally dependent on the estimates of both costs and effects, so they should be treated with the same respect.
It is important to recognize that the valuation of costs consists of two distinct elements: the estimation of the quantities of resources used (eg, number of hospital admissions of a given type, days of hospital stay or number of physician visits) and their monetary valuation using the relevant unit costs or prices. In practice, ‘minding your Ps and Qs’ in the reporting of economic evaluations involves having two tables in the paper:
- one table reporting the estimates of quantities of resources used and their sources
- a second table reporting the estimates of the costs or prices used, along with their sources
If there is any variability around the estimates, or a range of sources is used, this can be reported in the same tables. Failure to do this adequately will almost certainly increase the risk of the paper being rejected. In addition, this level of transparency helps the readers of studies to form a judgment on whether the results of the study apply in their setting. It’s one thing to be told that the costs of intensive care were $30,000, but more informative to be told that it consisted of 10 days at the local daily rate of $3,000.
Top Tip: Include two tables in the paper: one reporting the estimates of resource quantities and their sources; the other reporting the financial estimates of prices/costs and their sources.
Item 16: If modelling is used, describe in detail and why used; report if the model is publicly available and where it can be accessed
Mathematical modelling has become a common practice in economic evaluation, as there is a need to understand longer-term benefits (and costs) that are associated with healthcare decisions. As there are many different approaches to modelling, describing a model structure is important for the interpretation of results. Descriptions are often easiest when done through figures with some accompanying text; although there are no widely accepted standards, there are available taxonomies to describe types of models and additional reporting guidance available for how best to do this.[8-10]
The goal of describing the model is to allow readers and reviewers to interpret findings and an interested researcher the ability to replicate the model. Despite available conventions for model reporting, the sophistication of some modelling approaches used in economic evaluation may still make an adequate description difficult. The best approach in these situations is to share the model outright. As many models are proprietary, CHEERS suggests at minimum that unlocked models be shared with editors and peer reviewers confidentially (indicated in CHEERS Item 17). Authors may also want to take advantage of recent trends in open data and sharing platforms to share models directly with readers.
Top Tip: If you can’t describe the model in a way that others can replicate the analysis, it isn’t useful information for publication. At minimum, there should be a figure with accompanying text to describe how the model works. If necessary, consider how the model can be shared with editors and peer reviewers confidentially.
Item 21: Describe any approaches to engage patients or service recipients, the general public, communities or stakeholders (such as clinicians and payers) in the design of the study
Engaging with patients and other relevant stakeholders in the design of an economic evaluation can help improve its relevance and impact.[11] Clinicians and payers are routinely engaged in the design of an economic evaluation. However, although patient and public involvement and engagement (PPIE) is relatively common within clinical research, that is not the case for economic evaluations. A number of examples of patient and caregiver engagement in the design of economic evaluations are starting to appear in the literature.[12,13] We encourage authors to provide a detailed and transparent account of any stakeholder engagement. Given PPIE is in its infancy, we recommend taking care to especially highlight any PPIE. Authors may wish to consult the GRIPP2 guidance on how to report such PPIE, “Nothing about us without us”[14]
Top Tip: Take care to fully report any stakeholder engagement, particularly when that engagement is used to elicit model inputs. Ability to replicate is key.
CHEERS 2022, GPP 2022, and Complying with Current Standards
Following the CHEERS 2022 guidelines is a critical step in communicating industry-sponsored economic evaluations according to the principles of Good Publication Practice (GPP 2022). Three key themes that occur throughout GPP 2022 are the need for transparency, the importance of replication and the value of data sharing.[15] These are standards that are particularly challenging to meet for authors, medical writers and industry sponsors when preparing economic evaluations for journal submission. CHEERS 2022 provides guidance on the necessary detail to aid transparency and potential replication. It also provides guidance on the challenging issue of model sharing and prospective registration of analyses. Given the amount of detail that is often needed to adequately report an economic evaluation, we encourage the use of supplementary material as this allows the main manuscript to remain readable and transparent, whilst providing the interested reader with sufficient detail to replicate and critically appraise an analysis.
We encourage you to use CHEERS 2022 whenever you are preparing an economic evaluation for journal submission. It will help you comply with the standards set out in GPP 2022, will lead to a timely and smooth publication process in the journal of choice, and most importantly will result in an impactful patient-centered timely piece of research.
Table 1. CHEERS 2022 Checklist
References
[1] Xie F, Zhou T. Industry sponsorship bias in cost effectiveness analysis: registry based analysis. BMJ. 2022 Jun 22;377:e069573. doi: 10.1136/bmj-2021-069573. PMID: 35732297; PMCID: PMC9214880.
[2] Husereau D, Drummond M, Augustovski F, et al.: Consolidated health economic evaluation reporting standards 2022 (CHEERS 2022) explanation and elaboration: a report of the ISPOR CHEERS II good practices task force. Value Health.2022;25:10–31. 10.1016/j.jval.2021.10.008
[3] EQUATOR Network https://www.equator-network.org/reporting-guidelines/ [accessed Oct 2022]
[4] How Research England Supports Research Excellence https://www.ukri.org/about-us/research-england/research-excellence/ref-impact/#contents-list [accessed Oct 2022]
[5] Dritsaki M, Gray A, Petrou S, Dutton S, Lamb SE, Thorn JC. Current UK practices on health economics analysis plans (HEAPs): are we using heaps of them? Pharmacoeconomics. 2018;36(2):253–257.
[6] Orzechowska, I., Sadique, M.Z., Thomas, K. et al. First-line support for assistance in breathing in children: statistical and health economic analysis plan for the FIRST-ABC trial. Trials 21, 903 (2020). https://doi.org/10.1186/s13063-020-04818-w
[7] Png ME, Petrou S, Achten J, Ooms A, Lamb SE, Hedley H, et al. Cost-utility analysis of surgical fixation with Kirschner wire versus casting after fracture of the distal radius: a health economic evaluation of the DRAFFT2 trial. Bone Joint J. 2022 Nov;104-B(11):1225–33.
[8] Brennan A, Chick SE, Davies R. A taxonomy of model structures for economic evaluation of health technologies. Health Econ. 2006;15(12):1295–1310.
[9] Stahl JE. Modelling methods for pharmacoeconomics and health technology assessment: an overview and guide. Pharmacoeconomics. 2008;26(2):131–148.
[10] Dahabreh IJ, Trikalinos TA, Balk EM, Wong JB. Recommendations for the conduct and reporting of modeling and simulation studies in health technology assessment. Ann Intern Med. 2016;165(8):575–581.
[11] Xie, R.Z., Malik, E.d., Linthicum, M.T. et al. Putting Stakeholder Engagement at the Center of Health Economic Modeling for Health Technology Assessment in the United States. PharmacoEconomics 39, 631–638 (2021). https://doi.org/10.1007/s40273-021-01036-3
[12] Wilson, M., Thavorn, K., Hawrysh, T. et al. Engaging Patients and Caregivers in an Early Health Economic Evaluation: Discerning Treatment Value Based on Lived Experience. PharmacoEconomics 40, 1119–1130 (2022). https://doi.org/10.1007/s40273-022-01180-4
[13] Staniszewska, S., Hill, E.M., Grant, R. et al. Developing a Framework for Public Involvement in Mathematical and Economic Modelling: Bringing New Dynamism to Vaccination Policy Recommendations. Patient 14, 435–445 (2021). https://doi.org/10.1007/s40271-020-00476-x
[14] Staniszewska S, Brett J, Simera I, Seers K, Mockford C, Goodlad S, et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. BMJ. 2017;358: j3453.
[15] DeTora LM, Toroser D, Sykes A, Vanderlinden C, Plunkett FJ, Lane T, Hanekamp E, Dormer L, DiBiasi F, Bridges D, Baltzer L, Citrome L. Good Publication Practice (GPP) Guidelines for Company-Sponsored Biomedical Research: 2022 Update. Ann Intern Med. 2022 Sep;175(9):1298-1304. doi: 10.7326/M22-1460. Epub 2022 Aug 30. PMID: 36037471.