Systematic Review: The Evidence That Publishing Patient Care Performance Data Improves Quality of Care
Publication: Annals of Internal Medicine
Volume 148, Number 2
Submit a Comment
Contributors must reveal any conflict of interest. Comments are moderated. Please see our information for authorsregarding comments on an Annals publication.
Abstract
Background:
Previous reviews have shown inconsistent effects of publicly reported performance data on quality of care, but many new studies have become available in the 7 years since the last systematic review.
Purpose:
To synthesize the evidence for using publicly reported performance data to improve quality.
Data Sources:
Web of Science, MEDLINE, EconLit, and Wilson Business Periodicals (1999–2006) and independent review of articles (1986–1999) identified in a previous systematic review. Only sources published in English were included.
Study Selection:
Peer-reviewed articles assessing the effects of public release of performance data on selection of providers, quality improvement activity, clinical outcomes (effectiveness, patient safety, and patient-centeredness), and unintended consequences.
Data Extraction:
Data on study participants, reporting system or level, study design, selection of providers, quality improvement activity, outcomes, and unintended consequences were extracted.
Data Synthesis:
Forty-five articles published since 1986 (27 of which were published since 1999) evaluated the impact of public reporting on quality. Many focus on a select few reporting systems. Synthesis of data from 8 health plan–level studies suggests modest association between public reporting and plan selection. Synthesis of 11 studies, all hospital-level, suggests stimulation of quality improvement activity. Review of 9 hospital-level and 7 individual provider–level studies shows inconsistent association between public reporting and selection of hospitals and individual providers. Synthesis of 11 studies, primarily hospital-level, indicates inconsistent association between public reporting and improved effectiveness. Evidence on the impact of public reporting on patient safety and patient-centeredness is scant.
Limitations:
Heterogeneity made comparisons across studies challenging. Only peer-reviewed, English-language articles were included.
Conclusion:
Evidence is scant, particularly about individual providers and practices. Rigorous evaluation of many major public reporting systems is lacking. Evidence suggests that publicly releasing performance data stimulates quality improvement activity at the hospital level. The effect of public reporting on effectiveness, safety, and patient-centeredness remains uncertain.
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
Agency for Healthcare Research and Quality. National healthcare quality report. Washington, DC: U.S. Department of Health and Human Services; 2004.
2.
National Committee for Quality Assurance. The state of healthcare quality. Washington, DC: National Committee for Quality Assurance; 2005.
3.
McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348:2635-45. [PMID: 12826639]
4.
Asch SM, McGlynn EA, Hogan MM, Hayward RA, Shekelle P, Rubenstein L, et al. Comparison of quality of care for patients in the Veterans Health Administration and patients in a national sample. Ann Intern Med. 2004;141:938-45. [PMID: 15611491]
5.
Asch SM, Kerr EA, Keesey J, Adams JL, Setodji CM, Malik S, et al. Who is at greatest risk for receiving poor-quality health care? N Engl J Med. 2006;354:1147-56. [PMID: 16540615]
6.
Kuperman GJ, Teich JM, Gandhi TK, Bates DW. Patient safety and computerized medication ordering at Brigham and Women's Hospital. Jt Comm J Qual Improv. 2001;27:509-21. [PMID: 11593885]
7.
Bates DW, Cohen M, Leape LL, Overhage JM, Shabot MM, Sheridan T. Reducing the frequency of errors in medicine using information technology. J Am Med Inform Assoc. 2001;8:299-308. [PMID: 11418536]
8.
Lannon CM, Coven BJ, LaneFrance F, Hickson GB, Miles PV, Swanson JT, et al; National Initiative for Children's Health Care Quality Project Advisory Committee. Principles of patient safety in pediatrics. Pediatrics. 2001;107:1473-5. [PMID: 11389280]
9.
Handler JA, Gillam M, Sanders AB, Klasco R. Defining, identifying, and measuring error in emergency medicine. Acad Emerg Med. 2000;7:1183-8. [PMID: 11073465]
10.
Barry CA, Bradley CP, Britten N, Stevenson FA, Barber N. Patients' unvoiced agendas in general practice consultations: qualitative study. BMJ. 2000;320:1246-50. [PMID: 10797036]
11.
Marvel MK, Epstein RM, Flowers K, Beckman HB. Soliciting the patient's agenda: have we improved? JAMA. 1999;281:283-7. [PMID: 9918487]
12.
Institute of Medicine. Crossing the Quality Chasm. Accessed at www.nap.edu/catalog.php?record_id=10027#toc on 7 November 2007.
13.
Lansky D. Improving quality through public disclosure of performance information. Health Aff (Millwood). 2002;21:52-62. [PMID: 12117153]
14.
Lohr K, ed.; Institute of Medicine. Medicare: A Strategy for Quality Assurance. Washington DC: National Academies Pr; 1990.
15.
USNews.com. America's Best Colleges 2006. Accessed at www.usnews.com/usnews/edu/college/rankings/rankindex_brief.php on 9 May 2006.
16.
Zagat Survey. Accessed at zagat.com on 9 May 2006.
17.
Sofaer S, Crofton C, Goldstein E, Hoy E, Crabb J. What do consumers want to know about the quality of care in hospitals? Health Serv Res. 2005;40:2018-36. [PMID: 16316436]
18.
RAND Health. Consumers and Health Care Quality Information: Need, Availability, Utility. Oakland, CA: California Healthcare Foundation; 2001.
19.
The Kaiser Family Foundation, Agency for Healthcare Research and Quality, Harvard School of Public Health. The National Survey on Consumers' Experiences with Patient Safety and Quality Information. Rockville, MD: Kaiser Family Foundation; 2004.
20.
Leapfrog Group. Accessed at www.leapfroggroup.org/cp on 10 May 2006.
21.
Pacific Business Group on Health. Value based purchasing. Accessed at www.pbgh.org/programs/value_based_purchasing.asp on 5 May 2006.
22.
U.S. Department of Health and Human Services. Hospital Compare. Accessed at www.hospitalcompare.hhs.gov/ on 5 May 2006.
23.
Berwick DM, James B, Coye MJ. Connections between quality measurement and improvement. Med Care. 2003;41:I30-8.
24.
Marshall MN, Shekelle PG, Davies HT, Smith PC. Public reporting on quality in the United States and the United Kingdom. Health Aff (Millwood). 2003;22:134-48. [PMID: 12757278]
25.
Office of Inspector General June Gibbs Brown. The External Review of Hospital Quality State Initiatives. Accessed at oig.hhs.gov/oei/reports/oei-01-97-00054.pdf on 5 May 2006.
26.
Centers for Medicare & Medicaid Services. Hospital Compare. Accessed at www.hospitalcompare.hhs.gov/Hospital/Static/About-Overview.asp?dest=NAV|Home|About|Overview on 3 July 2007.
27.
Marshall MN, Shekelle PG, Leatherman S, Brook RH. The public release of performance data: what do we expect to gain? A review of the evidence. JAMA. 2000;283:1866-74. [PMID: 10770149]
28.
Schauffler HH, Mordavsky JK. Consumer reports in health care: do they make a difference? Annu Rev Public Health. 2001;22:69-89. [PMID: 11274512]
29.
Schneider EC, Lieberman T. Publicly disclosed information about the quality of health care: response of the US public. Qual Health Care. 2001;10:96-103. [PMID: 11389318]
30.
Werner RM, Asch DA. The unintended consequences of publicly reporting quality information. JAMA. 2005;293:1239-44. [PMID: 15755946]
31.
Hibbard JH, Stockard J, Tusler M. Does publicizing hospital performance stimulate quality improvement efforts? Health Aff (Millwood). 2003;22:84-94. [PMID: 12674410]
32.
Marshall MN, Shekelle PG, Davies HT, Smith PC. Public reporting on quality in the United States and the United Kingdom. Health Aff (Millwood). 2003;22:134-48. [PMID: 12757278]
33.
Cochrane Effective Practice and Organisation of Care Review Group. The Data Collection Checklist. Ottawa: Cochrane Effective Practice and Organisation of Care Group; 2002.
34.
Grimshaw J, McAuley LM, Bero LA, Grilli R, Oxman AD, Ramsay C, et al. Systematic reviews of the effectiveness of quality improvement strategies and programmes. Qual Saf Health Care. 2003;12:298-303. [PMID: 12897365]
35.
GRADE Working Group. Grading of Recommendations, Assessment, Development, and Evaluation. Accessed at www.gradeworkinggroup.org/FAQ/index.htm on 3 July 2007.
36.
Agency for Healthcare Research and Quality. CAHPS Overview. Accessed at www.cahps.ahrq.gov/content/cahpsOverview/OVER_Intro.asp on 15 May 2006.
37.
Farley DO, Short PF, Elliott MN, Kanouse DE, Brown JA, Hays RD. Effects of CAHPS health plan performance information on plan choices by New Jersey Medicaid beneficiaries. Health Serv Res. 2002;37:985-1007. [PMID: 12236394]
38.
Farley DO, Elliott MN, Short PF, Damiano P, Kanouse DE, Hays RD. Effect of CAHPS performance information on health plan choices by Iowa Medicaid beneficiaries. Med Care Res Rev. 2002;59:319-36. [PMID: 12205831]
39.
Spranca M, Kanouse DE, Elliott M, Short PF, Farley DO, Hays RD. Do consumer reports of health plan quality affect health plan selection? Health Serv Res. 2000;35:933-47. [PMID: 11130805]
40.
Harris KM. Can high quality overcome consumer resistance to restricted provider access? Evidence from a health plan choice experiment. Health Serv Res. 2002;37:551-71. [PMID: 12132595]
41.
Beaulieu ND. Quality information and consumer health plan choices. J Health Econ. 2002;21:43-63. [PMID: 11845925]
42.
Wedig GJ, Tai-Seale M. The effect of report cards on consumer choice in the health insurance market. J Health Econ. 2002;21:1031-48. [PMID: 12475124]
43.
Jin GZ, Sorensen AT. Information and consumer choice: the value of publicized health plan ratings. J Health Econ. 2006;25:248-75. [PMID: 16107284]
44.
Scanlon DP, Chernew M, Mclaughlin C, Solon G. The impact of health plan report cards on managed care enrollment. J Health Econ. 2002;21:19-41.
45.
Bost JE. Managed care organizations publicly reporting three years of HEDIS measures. Manag Care Interface. 2001;14:50-4. [PMID: 11569305]
46.
McCormick D, Himmelstein DU, Woolhandler S, Wolfe SM, Bor DH. Relationship between low quality-of-care scores and HMOs' subsequent public disclosure of quality-of-care scores. JAMA. 2002;288:1484-90. [PMID: 12243635]
47.
Mennemeyer ST, Morrisey MA, Howard LZ. Death and reputation: how consumers acted upon HCFA mortality information. Inquiry. 1997;34:117-28. [PMID: 9256817]
48.
Vladeck BC, Goodwin EJ, Myers LP, Sinisi M. Consumers and hospital use: the HCFA “death list”. Health Aff (Millwood). 1988;7:122-5. [PMID: 3360387]
49.
Mukamel DB, Mushlin AI. Quality of care information makes a difference: an analysis of market share and price changes after publication of the New York State Cardiac Surgery Mortality Reports. Med Care. 1998;36:945-54. [PMID: 9674613]
50.
Hannan EL, Kumar D, Racz M, Siu AL, Chassin MR. New York State's Cardiac Surgery Reporting System: four years later. Ann Thorac Surg. 1994;58:1852-7. [PMID: 7979781]
51.
Chassin MR. Achieving and sustaining improved quality: lessons from New York State and cardiac surgery. Health Aff (Millwood). 2002;21:40-51. [PMID: 12117152]
52.
Jha AK, Epstein AM. The predictive accuracy of the New York State coronary artery bypass surgery report-card system. Health Aff (Millwood). 2006;25:844-55. [PMID: 16684751]
53.
Baker DW, Einstadter D, Thomas C, Husak S, Gordon NH, Cebul RD. The effect of publicly reporting hospital performance on market share and risk-adjusted mortality at high-mortality hospitals. Med Care. 2003;41:729-40. [PMID: 12773839]
54.
Romano PS, Zhou H. Do well-publicized risk-adjusted outcomes reports affect hospital volume? Med Care. 2004;42:367-77. [PMID: 15076814]
55.
Hibbard JH, Stockard J, Tusler M. Hospital performance reports: impact on quality, market share, and reputation. Health Aff (Millwood). 2005;24:1150-60. [PMID: 16012155]
56.
Dziuban SW Jr, McIlduff JB, Miller SJ, DalCol RH. How a New York cardiac surgery program uses outcomes data. Ann Thorac Surg. 1994;58:1871-6. [PMID: 7979785]
57.
Bentley JM, Nash DB. How Pennsylvania hospitals have responded to publicly released reports on coronary artery bypass graft surgery. Jt Comm J Qual Improv. 1998;24:40-9. [PMID: 9494873]
58.
Rosenthal GE, Hammar PJ, Way LE, Shipley SA, Doner D, Wojtala B, et al. Using hospital performance data in quality improvement: the Cleveland Health Quality Choice experience. Jt Comm J Qual Improv. 1998;24:347-60. [PMID: 9689568]
59.
Tu JV, Cameron C. Impact of an acute myocardial infarction report card in Ontario, Canada. Int J Qual Health Care. 2003;15:131-7. [PMID: 12705706]
60.
Longo DR, Land G, Schramm W, Fraas J, Hoskins B, Howell V. Consumer reports in health care. Do they make a difference in patient care? JAMA. 1997;278:1579-84. [PMID: 9370503]
61.
Luce JM, Thiel GD, Holland MR, Swig L, Currin SA, Luft HS. Use of risk-adjusted outcome data for quality improvement by public hospitals. West J Med. 1996;164:410-4. [PMID: 8686297]
62.
Rainwater JA, Romano PS, Antonius DM. The California Hospital Outcomes Project: how useful is California's report card for quality improvement? Jt Comm J Qual Improv. 1998;24:31-9. [PMID: 9494872]
63.
Mannion R, Davies H, Marshall M. Impact of star performance ratings in English acute hospital trusts. J Health Serv Res Policy. 2005;10:18-24. [PMID: 15667700]
64.
Hannan EL, Kilburn H Jr, Racz M, Shields E, Chassin MR. Improving the outcomes of coronary artery bypass surgery in New York State. JAMA. 1994;271:761-6. [PMID: 8114213]
65.
Ghali WA, Ash AS, Hall RE, Moskowitz MA. Statewide quality improvement initiatives and mortality after cardiac surgery. JAMA. 1997;277:379-82. [PMID: 9010169]
66.
Peterson ED, DeLong ER, Jollis JG, Muhlbaier LH, Mark DB. The effects of New York's bypass surgery provider profiling on access to care and patient outcomes in the elderly. J Am Coll Cardiol. 1998;32:993-9. [PMID: 9768723]
67.
Moscucci M, Eagle KA, Share D, Smith D, DeFranco AC, O'Donnell M, et al. Public reporting and case selection for percutaneous coronary interventions: an analysis from two large multicenter percutaneous coronary intervention databases. J Am Coll Cardiol. 2005;45:1759-65. [PMID: 15936602]
68.
Omoigui NA, Miller DP, Brown KJ, Annan K, Cosgrove D 3rd, Lytle B, et al. Outmigration for coronary bypass surgery in an era of public dissemination of clinical outcomes. Circulation. 1996;93:27-33. [PMID: 8616936]
69.
Dranove D, Kessler D, McClellan M, Satterthwaithe M. Is more information better? The effects of “report cards” on health care providers. Journal of Political Economy. 2003;111:555-88.
70.
Rosenthal GE, Quinn L, Harper DL. Declines in hospital mortality associated with a regional initiative to measure hospital performance. Am J Med Qual. 1997;12:103-12. [PMID: 9161057]
71.
Clough JD, Engler D, Snow R, Canuto PE. Lack of relationship between the Cleveland Health Quality Choice project and decreased inpatient mortality in Cleveland. Am J Med Qual. 2002;17:47-55. [PMID: 11941994]
72.
Baker DW, Einstadter D, Thomas CL, Husak SS, Gordon NH, Cebul RD. Mortality trends during a program that publicly reported hospital performance. Med Care. 2002;40:879-90. [PMID: 12395022]
73.
Mukamel DB, Weimer DL, Zwanziger J, Gorthy SF, Mushlin AI. Quality report cards, selection of cardiac surgeons, and racial disparities: a study of the publication of the New York State Cardiac Surgery Reports. Inquiry. 2004;41:435-46. [PMID: 15835601]
74.
Hannan EL, Siu AL, Kumar D, Kilburn H Jr, Chassin MR. The decline in coronary artery bypass graft surgery mortality in New York State. The role of surgeon volume. JAMA. 1995;273:209-13. [PMID: 7807659]
75.
Mukamel DB, Mushlin AI, Weimer D, Zwanziger J, Parker T, Indridason I. Do quality report cards play a role in HMOs' contracting practices? Evidence from New York State. Health Serv Res. 2000;35:319-32. [PMID: 10778818]
76.
Mukamel DB, Weimer DL, Zwanziger J, Mushlin AI. Quality of cardiac surgeons and managed care contracting practices. Health Serv Res. 2002;37:1129-44. [PMID: 12479489]
77.
Schneider EC, Epstein AM. Influence of cardiac-surgery performance reports on referral practices and access to care. A survey of cardiovascular specialists. N Engl J Med. 1996;335:251-6. [PMID: 8657242]
78.
Burack JH, Impellizzeri P, Homel P, Cunningham JN Jr. Public reporting of surgical mortality: a survey of New York State cardiothoracic surgeons. Ann Thorac Surg. 1999;68:1195-200; discussion 1201-2. [ 10543479]
79.
Narins CR, Dozier AM, Ling FS, Zareba W. The influence of public reporting of outcome data on medical decision making by physicians. Arch Intern Med. 2005;165:83-7. [PMID: 15642879]
80.
Werner RM, Asch DA, Polsky D. Racial profiling: the unintended consequences of coronary artery bypass graft report cards. Circulation. 2005;111:1257-63. [PMID: 15769766]
81.
HealthGrades. Accessed at www.healthgrades.com on 10 May 2006.
82.
Pacific Business Group on Health. HealthScope.org. Accessed at http://www.pbgh.org/pbgh_sites/healthscope.asp on 10 July 2007.
83.
U.S. Congress. The next generation of health information tools for consumers. Washington, DC: Joint Economic Committees; 2006.
84.
Hibbard JH, Harris-Kojetin L, Mullin P, Lubalin J, Garfinkel S. Increasing the impact of health plan report cards by addressing consumers' concerns. Health Aff (Millwood). 2000;19:138-43. [PMID: 10992661]
85.
Hibbard JH, Jewett JJ, Legnini MW, Tusler M. Choosing a health plan: do large employers use the data? Health Aff (Millwood). 1997;16:172-80. [PMID: 9444825]
86.
Hannan EL, Stone CC, Biddle TL, DeBuono BA. Public release of cardiac surgery outcomes data in New York: what do New York state cardiologists think of it? Am Heart J. 1997;134:55-61. [PMID: 9266783]
87.
Berwick DM, Wald DL. Hospital leaders' opinions of the HCFA mortality data. JAMA. 1990;263:247-9. [PMID: 2403602]
88.
Robinson S, Brodie M. Understanding the quality challenge for health consumers: the Kaiser/AHCPR Survey. Jt Comm J Qual Improv. 1997;23:239-44. [PMID: 9179715]
89.
Grumbach K, Osmond D, Vranizan K, Jaffe D, Bindman AB. Primary care physicians' experience of financial incentives in managed-care systems. N Engl J Med. 1998;339:1516-21. [PMID: 9819451]
90.
Henley E. Pay-for-performance: what can you expect? J Fam Pract. 2005;54:609-12. [PMID: 16009089]
91.
Rosenthal MB, Frank RG, Li Z, Epstein AM. Early experience with pay-for-performance: from concept to practice. JAMA. 2005;294:1788-93. [PMID: 16219882]
92.
NCQA Health Plan Report Card. Accessed at hprc.ncqa.org/ on 5 May 2006.
93.
New York State Department of Health. Heart disease. Accessed at www.health.state.ny.us/nysdoh/heart/heart_disease.htm#cardiovascular on 5 May 2006.
94.
New Jersey Department of Health and Senior Services. 2005 HMO New Jersey Performance Report. Accessed at web.doh.state.nj.us/hpr/ on 5 May 2006.
Information & Authors
Information
Published In
Annals of Internal Medicine
Volume 148 • Number 2 • 15 January 2008
Pages: 111 - 123
History
Published online: 15 January 2008
Published in issue: 15 January 2008
Keywords
Copyright
© 2008 American College of Physicians.
Authors
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. For an editable text file, please select Medlars format which will download as a .txt file. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
Download article citation data for:
Systematic Review: The Evidence That Publishing Patient Care Performance Data Improves Quality of Care. Ann Intern Med.2008;148:111-123. [Epub 15 January 2008]. doi:10.7326/0003-4819-148-2-200801150-00006
Systematic Review: The Evidence That Publishing Patient Care Performance Data Improves Quality of Care. Ann Intern Med.2008;148:111-123. [Epub 15 January 2008]. doi:10.7326/0003-4819-148-2-200801150-00006
View More
Login Options:
Purchase
You will be redirected to acponline.org to sign-in to Annals to complete your purchase.
Access to EPUBs and PDFs for FREE Annals content requires users to be registered and logged in. A subscription is not required. You can create a free account below or from the following link. You will be redirected to acponline.org to create an account that will provide access to Annals. If you are accessing the Free Annals content via your institution's access, registration is not required.
Create your Free Account
You will be redirected to acponline.org to create an account that will provide access to Annals.
Comments
0 Comments