Purpose:
Cost-utility analysis is a type of cost-effectiveness analysis in which health effects are measured in terms of quality-adjusted life-years [QALYs] gained. Such analyses have become popular for examining the health and economic consequences of health and medical interventions, and they have been recommended by leaders in the field. These recommendations emphasize the importance of good reporting practices. This study determined 1) the quality of reporting in published cost–utility analyses through 1997 and 2) whether reporting practices have improved over time. We examined quality of reporting by journal type and number of cost–utility analyses a journal has published.
Data Sources:
Computerized databases were searched through 1997 for the Medical Subject Headings or text keywords quality-adjusted, QALY, and cost–utility analysis. Published bibliographies of the field were also searched.
Study Selection:
Original cost–utility analyses written in English were included. Cost-effectiveness analyses that measured health effects in units other than QALYs and review, editorial, or methodologic articles were excluded.
Data Extraction:
Each of the 228 articles found was audited independently by two trained readers who used a standard data collection form to determine the quality of reporting in several categories: disclosure of funding, framing, reporting of costs, reporting of preference weights, reporting of results, and discussion.
Results:
The number of cost–utility analyses in the medical literature increased greatly between 1976 and 1997. Analyses covered a wide range of diseases and interventions. Most studies listed modeling assumptions (82%), described the comparator intervention (83%), reported sensitivity analysis (89%), and noted study limitations (84%). Only 52% clearly stated the study perspective; 34% did not disclose the funding source. Methods of reporting costs and preference weights varied widely. The quality of published analyses improved slightly over time and was higher in general clinical journals and in journals that published more of these analyses.
Conclusions:
The study results reveal an active and evolving field but also underscore the need for more consistency and clarity in reporting. Better peer review and independent, third-party audits may help in this regard. Future investigations should examine the quality of clinical and economic assumptions used in cost–utility analyses, in addition to whether analysts followed recommended protocols for performance and reporting.
References
- 1.
Elixhauser A ,Halpern M ,Schmier J ,Luce BR . Health care CBA and CEA from 1991 to 1996: an updated bibliography. Med Care. 1998;36 5Suppl 1 MS1-9, MS18-147. MedlineGoogle Scholar - 2.
Elixhauser A ,Luce BR ,Taylor WR ,Reblando J . Health care CBA/CEA: an update on the growth and composition of the literature. Med Care. 1993;31 7Suppl JS1-11, JS18-149. MedlineGoogle Scholar - 3.
Russell LB ,Gold MR ,Siegel JE ,Daniels N . The role of cost-effectiveness analysis in health and medicine. Panel on Cost-Effectiveness Analysis in Health and Medicine. JAMA. 1996;276:1172-7. CrossrefMedlineGoogle Scholar - 4.
Weinstein MC ,Siegel JE ,Gold MR ,Kamlet MS ,Russell LB . Recommendations of the Panel on Cost-Effectiveness in Health and Medicine. JAMA. 1996;276:1253-8. CrossrefMedlineGoogle Scholar - 5.
Siegel JE ,Weinstein MC ,Russell LB ,Gold MR . Recommendations for reporting cost-effectiveness analyses. Panel on Cost-Effectiveness in Health and Medicine. JAMA. 1996;276:1339-41. CrossrefMedlineGoogle Scholar - 6.
Drummond MF ,Richardson WS ,O'Brien BJ ,Levine M ,Heyland D . Users' guides to the medical literature. XIII. How to use an article on economic analysis of clinical practice. A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA. 1997;277:1552-7. CrossrefMedlineGoogle Scholar - 7.
O'Brien BJ ,Heyland D ,Richardson WS ,Levine M ,Drummond MF . Users' guides to the medical literature. XIII. How to use an article on economic analysis in clinical practice. B. What are the results and will they help me in caring for my patients? Evidence-Based Medicine Working Group. JAMA. 1997;277:1802-6. CrossrefMedlineGoogle Scholar - 8.
Udvarhelyi IS ,Colditz GA ,Rai A ,Epstein AM . Cost-effectiveness and cost–benefit analyses in the medical literature. Are they being used correctly? Ann Intern Med. 1992;116:238-44. LinkGoogle Scholar - 9.
Kassirer J ,Angell M . The journal's policy on cost-effectiveness analyses. N Engl J Med. 1994;331:669-70. CrossrefMedlineGoogle Scholar - 10.
Evans RG . Manufacturing consensus, marketing truth: guidelines for economic evaluation [Editorial]. Ann Intern Med. 1995;123:59-60. LinkGoogle Scholar - 11.
Neumann PJ ,Zinner DE ,Wright JC . Are methods for estimating QALYs in cost-effectiveness analyses improving? Med Decis Making. 1997;17:402-8. CrossrefMedlineGoogle Scholar - 12.
Luce BR ,Lyles AC ,Rentz AM . The view from managed care pharmacy. Health Aff Millwood. 1996;15:168-76. CrossrefMedlineGoogle Scholar - 13.
Gold MR ,Siegel JE ,Russell LB ,Weinstein MC . Cost-effectiveness in Health and Medicine. New York: Oxford Univ Pr; 1996. Google Scholar - 14.
Drummond MF ,Stoddart GL ,Torrance GW . Methods for the Economic Evaluation of Health Care Programmes. New York: Oxford Univ Pr; 1987. Google Scholar - 15.
Jefferson TO ,Smith R ,Yee Y ,Drummond MJ ,Pratt M ,Gale R . Evaluating the BMJ guidelines for economic submissions. JAMA. 1998;280:275-7. CrossrefMedlineGoogle Scholar - 16.
Mehrez A ,Gafni A . Healthy-years equivalents versus quality-adjusted life years: in pursuit of progress. Med Decis Making. 1993;13:287-92. CrossrefMedlineGoogle Scholar - 17.
Cox D ,Fitzpatrick R ,Fletcher A ,Gore S ,Spiegelhalter D ,Jones D . Quality-of-life assessment: can we keep it simple? J R Statist Soc A. 1992;155:353-93. CrossrefGoogle Scholar - 18.
Ubel PA ,Loewenstein G ,Scanlon D ,Kamlet M . Individual utilities are inconsistent with rationing choices. A partial list of why Oregon's cost-effectiveness list failed. Med Decis Making. 1996;16:108-16. CrossrefMedlineGoogle Scholar - 19.
Menon D ,Schubert F ,Torrance GW . Canada's new guidelines for the economic evaluation of pharmaceuticals. Med Care. 1996;34:DS77-86. MedlineGoogle Scholar - 20. Drummond MF, O'Brien B, Stoddart GL, Torrance GW. Methods for the Economic Evaluation of Health Care Programmes. 2d ed. New York: Oxford Univ Pr; 1997. Google Scholar
- 21.
Task Force on Principles for Economic Analysis of Health Care Technology . Economic analysis of health care technology. A report on principles. Ann Intern Med. 1995;123:61-70. LinkGoogle Scholar - 22.
Drummond MF ,Jefferson TO . Guidelines for authors and peer reviewers of economic submissions to the BMJ. The BMJ Economic Evaluation Working Party. BMJ. 1996;313:275-83. CrossrefMedlineGoogle Scholar - 23.
Landis RJ ,Koch GG . The measurement of observer agreement for categorical data. Biometrics. 1977;33:159-74. CrossrefMedlineGoogle Scholar - 24.
Gerard K ,Smoker I ,Seymour J . Raising the quality of cost–utility analyses: lessons learnt and still to learn. Health Policy. 1999;46:217-38. CrossrefMedlineGoogle Scholar - 25. Guide to Clinical Preventive Services: Report of the U.S. Preventive Services Task Force. 2d ed. Baltimore: Williams & Wilkins; 1996. Google Scholar
- 26.
Tengs TO ,Adams ME ,Pliskin JS ,Safran DG ,Siegel JE ,Weinstein MC ,et al . Five-hundred life-saving interventions and their cost-effectiveness. Risk Anal. 1995;15:369-90. CrossrefMedlineGoogle Scholar - 27.
Gerard K . Cost-utility in practice: a policy maker's guide to the state of the art. Health Policy. 1992;21:249-79. CrossrefMedlineGoogle Scholar - 28.
Brown ML ,Fintor L . Cost-effectiveness of breast cancer screening: preliminary results of a systematic review of the literature. Breast Cancer Res Treatment. 1993;25:113-8. CrossrefMedlineGoogle Scholar - 29.
Nord E . Toward quality assurance in QALY calculations. Int J Technol Assess Health Care. 1993;9:37-45. CrossrefMedlineGoogle Scholar - 30.
Mason JM . Cost-per-QALY league tables: their role in pharmacoeconomic analysis. Pharmacoeconomics. 1994;5:472-81. CrossrefMedlineGoogle Scholar - 31.
Adams ME ,McCall NT ,Gray DT ,Orza MJ ,Chalmers TC . Economic analysis in randomized, controlled trials. Med Care. 1992;30:231-43. CrossrefMedlineGoogle Scholar - 32.
Briggs A ,Sculpher M . Sensitivity analysis in economic evaluation: a review of published studies. Health Econ. 1995;4:355-71. CrossrefMedlineGoogle Scholar - 33.
Blackmore CC ,Magid DJ . Methodologic evaluation of the radiology cost-effectiveness literature. Radiology. 1997;203:87-91. CrossrefMedlineGoogle Scholar - 34.
Balas EA ,Kretschmer RA ,Gnann W ,West DA ,Boren SA ,Centor RM ,et al . Interpreting cost analyses of clinical interventions. JAMA. 1998;279:54-7. CrossrefMedlineGoogle Scholar - 35.
Weinstein MC ,Stason WB . Foundations of cost-effectiveness analysis for health and medical practice. N Engl J Med. 1977;296:716-21. CrossrefMedlineGoogle Scholar - 36.
Graham JD ,Corso PS ,Morris JM ,Segui-Gomez M ,Weinstein MC . Evaluating the cost-effectiveness of clinical and public health measures. Annu Rev Public Health. 1998;19:125-52. CrossrefMedlineGoogle Scholar - 37.
Reinhardt UE . Making economic evaluations respectable. Soc Sci Med. 1997;45:555-62. CrossrefMedlineGoogle Scholar - 38.
Elstein AS . MDM policy regarding financial support of authors. Med Decis Making. 1997;17:497-8. CrossrefMedlineGoogle Scholar - 39.
Paltiel AD ,Neumann PJ . Why training is the key to successful guideline implementation. Pharmacoeconomics. 1997;12:297-302. CrossrefMedlineGoogle Scholar - 40.
Neumann PJ . Paying the piper for pharmacoeconomic studies. Med Decis Making. 1998;18 2 Suppl S23-6. CrossrefMedlineGoogle Scholar - 41.
Neumann PJ ,Zinner DE ,Paltiel AD . The FDA's regulation of cost-effectiveness claims. Health Aff Millwood. 1996;15:54-71. CrossrefMedlineGoogle Scholar - 42.
Mitchell A . Update and evaluation of Australian guidelines. Government perspective. Med Care. 1996;34 12 Suppl DS216-25. MedlineGoogle Scholar - 43. OHE Briefing. The Health Economic Evaluations Database. Trends in Economic Evaluation. London: OHE-IFPMA; 1998. Google Scholar
Author, Article, and Disclosure Information
From Harvard School of Public Health, Boston, Massachusetts; University of Rochester, Rochester, New York; and University of Toronto, Toronto, Ontario, Canada.
For definitions of terms, see Glossary at end of text.
Acknowledgments: The authors thank Vijay Ramakrishnan for research assistance and Marc Berger, John Graham, Jim Hammitt, Steven Teutsch, Milton Weinstein, and Albert Wertheimer for helpful comments on earlier drafts of this manuscript.
Grant Support: By a joint award from the National Science Foundation and Merck & Co., Inc. under the Joint NSF/Private Research Opportunity Initiative (SBR-9730448). Dr. Bell is funded through a fellowship from the Medical Research Council of Canada.
Corresponding Author: Peter J. Neumann, ScD, Harvard School of Public Health, 718 Huntington Avenue, 2nd Floor, Boston, MA 02115; e-mail, [email protected]
Current Author Addresses: Dr. Neumann, Mr. Chapman, and Ms. Sandberg: Harvard School of Public Health, 718 Huntington Avenue, 2nd Floor, Boston, MA 02115.
Dr. Stone: University of Rochester, School of Nursing, and Department of Community and Preventive Medicine, 601 Elmwood Avenue, Box SON, Rochester, NY 14642-8404.
Dr. Bell: Clinical Epidemiology Unit, Sunnybrook and Women's Health Science Centre, Toronto, Ontario M4N 3M5, Canada.
Submit a Comment
Contributors must reveal any conflict of interest. Comments are moderated. Please see our information for authorsregarding comments on an Annals publication.
*All comments submitted after October 1, 2021 and selected for publication will be published online only.