Background:
Accountable care organizations (ACOs) in the Medicare Shared Savings Program (MSSP) are associated with modest savings. However, prior research may overstate this effect if high-cost clinicians exit ACOs.
Objective:
To evaluate the effect of the MSSP on spending and quality while accounting for clinicians' nonrandom exit.
Design:
Similar to prior MSSP analyses, this study compared MSSP ACO participants versus control beneficiaries using adjusted longitudinal models that accounted for secular trends, market factors, and beneficiary characteristics. To further account for selection effects, the share of nearby clinicians in the MSSP was used as an instrumental variable. Hip fracture served as a falsification outcome. The authors also tested for compositional changes among MSSP participants.
Setting:
Fee-for-service Medicare, 2008 through 2014.
Patients:
A 20% sample (97 204 192 beneficiary-quarters).
Measurements:
Total spending, 4 quality indicators, and hospitalization for hip fracture.
Results:
In adjusted longitudinal models, the MSSP was associated with spending reductions (change, −$118 [95% CI, −$151 to −$85] per beneficiary-quarter) and improvements in all 4 quality indicators. In instrumental variable models, the MSSP was not associated with spending (change, $5 [CI, −$51 to $62] per beneficiary-quarter) or quality. In falsification tests, the MSSP was associated with hip fracture in the adjusted model (−0.24 hospitalizations for hip fracture [CI, −0.32 to −0.16 hospitalizations] per 1000 beneficiary-quarters) but not in the instrumental variable model (0.05 hospitalizations [CI, −0.10 to 0.20 hospitalizations] per 1000 beneficiary-quarters). Compositional changes were driven by high-cost clinicians exiting ACOs: High-cost clinicians (99th percentile) had a 30.4% chance of exiting the MSSP, compared with a 13.8% chance among median-cost clinicians (50th percentile).
Limitation:
The study used an observational design and administrative data.
Conclusion:
After adjustment for clinicians' nonrandom exit, the MSSP was not associated with improvements in spending or quality. Selection effects—including exit of high-cost clinicians—may drive estimates of savings in the MSSP.
Primary Funding Source:
Horowitz Foundation for Social Policy, Agency for Healthcare Research and Quality, and National Institute on Aging.
References
- 1. Muhlestein D, Saunders RS, McClellan MB. Growth of ACOs and alternative payment models in 2017. Health Affairs Blog. 28 June 2017. Accessed at www.healthaffairs.org/do/10.1377/hblog20170628.060719/full on 1 April 2019. Google Scholar
- 2.
McWilliams JM ,Hatfield LA ,Chernew ME ,Landon BE ,Schwartz AL . Early performance of accountable care organizations in Medicare. N Engl J Med. 2016;374:2357-66. [PMID: 27075832] doi:10.1056/NEJMsa1600142 CrossrefMedlineGoogle Scholar - 3.
McWilliams JM . Changes in Medicare Shared Savings Program savings from 2013 to 2014 [Letter]. JAMA. 2016;316:1711-3. [PMID: 27612292] doi:10.1001/jama.2016.12049 CrossrefMedlineGoogle Scholar - 4.
Colla CH ,Lewis VA ,Kao LS ,O'Malley AJ ,Chang CH ,Fisher ES . Association between Medicare accountable care organization implementation and spending among clinically vulnerable beneficiaries. JAMA Intern Med. 2016;176:1167-75. [PMID: 27322485] doi:10.1001/jamainternmed.2016.2827 CrossrefMedlineGoogle Scholar - 5.
Schur CL ,Sutton JP . Physicians in Medicare ACOs offer mixed views of model for health care cost and quality. Health Aff (Millwood). 2017;36:649-54. [PMID: 28373330] doi:10.1377/hlthaff.2016.1427 CrossrefMedlineGoogle Scholar - 6.
Markovitz AA ,Ramsay PP ,Shortell SM ,Ryan AM . Financial incentives and physician practice participation in Medicare's value-based reforms. Health Serv Res. 2018;53 Suppl 1:3052-69. [PMID: 28748535] doi:10.1111/1475-6773.12743 CrossrefMedlineGoogle Scholar - 7.
Lewis VA ,Colla CH ,Carluzzo KL ,Kler SE ,Fisher ES . Accountable care organizations in the United States: market and demographic factors associated with formation. Health Serv Res. 2013;48:1840-58. [PMID: 24117222] doi:10.1111/1475-6773.12102 CrossrefMedlineGoogle Scholar - 8.
Hsu J ,Vogeli C ,Price M ,Brand R ,Chernew ME ,Mohta N ,et al . Substantial physician turnover and beneficiary ‘churn' in a large Medicare Pioneer ACO. Health Aff (Millwood). 2017;36:640-8. [PMID: 28373329] doi:10.1377/hlthaff.2016.1107 CrossrefMedlineGoogle Scholar - 9.
Markovitz AA ,Hollingsworth JM ,Ayanian JZ ,Norton EC ,Moloci NM ,Yan PL ,et al . Risk adjustment in Medicare ACO program deters coding increases but may lead ACOs to drop high-risk beneficiaries. Health Aff (Millwood). 2019;38:253-61. [PMID: 30715995] doi:10.1377/hlthaff.2018.05407 CrossrefMedlineGoogle Scholar - 10. Centers for Medicare & Medicaid Services. Shared savings and losses and assignment methodology: specifications (version 3). 2014. Accessed at www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/sharedsavingsprogram/Downloads/Shared-Savings-Losses-Assignment-Spec.pdf on 6 January 2019. Google Scholar
- 11.
Nyweide DJ ,Lee W ,Cuerdon TT ,Pham HH ,Cox M ,Rajkumar R ,et al . Association of Pioneer Accountable Care Organizations vs traditional Medicare fee for service with spending, utilization, and patient experience. JAMA. 2015;313:2152-61. [PMID: 25938875] doi:10.1001/jama.2015.4930 CrossrefMedlineGoogle Scholar - 12.
Gottlieb DJ ,Zhou W ,Song Y ,Andrews KG ,Skinner JS ,Sutherland JM . Prices don't drive regional Medicare spending variations. Health Aff (Millwood). 2010;29:537-43. [PMID: 20110290] doi:10.1377/hlthaff.2009.0609 CrossrefMedlineGoogle Scholar - 13. Agency for Healthcare Research and Quality. Prevention quality indicators technical specifications - version 6.0. 2016. Accessed at www.qualityindicators.ahrq.gov/modules/PQI_TechSpec.aspx on 6 January 2019. Google Scholar
- 14. RTI International. Accountable care organization 2014 program analysis quality performance standards narrative measure specifications. 2014. Accessed at www.healthreform.ct.gov/ohri/lib/ohri/work_groups/quality/2014-09-03/aco_narrative_measures_specs_1.pdf on 6 January 2019 Google Scholar
- 15.
Prasad V ,Jena AB . Prespecified falsification end points: can they validate true observational associations? JAMA. 2013;309:241-2. [PMID: 23321761] doi:10.1001/jama.2012.96867 CrossrefMedlineGoogle Scholar - 16.
Colla CH ,Wennberg DE ,Meara E ,Skinner JS ,Gottlieb D ,Lewis VA ,et al . Spending differences associated with the Medicare Physician Group Practice Demonstration. JAMA. 2012;308:1015-23. [PMID: 22968890] doi:10.1001/2012.jama.10812 CrossrefMedlineGoogle Scholar - 17.
Fisher ES ,Wennberg JE ,Stukel TA ,Sharp SM . Hospital readmission rates for cohorts of Medicare beneficiaries in Boston and New Haven. N Engl J Med. 1994;331:989-95. [PMID: 8084356] CrossrefMedlineGoogle Scholar - 18.
Wennberg JE ,Freeman JL ,Shelton RM ,Bubolz TA . Hospital use and mortality among Medicare beneficiaries in Boston and New Haven. N Engl J Med. 1989;321:1168-73. [PMID: 2677726] CrossrefMedlineGoogle Scholar - 19.
McClellan M ,McNeil BJ ,Newhouse JP . Does more intensive treatment of acute myocardial infarction in the elderly reduce mortality? Analysis using instrumental variables. JAMA. 1994;272:859-66. [PMID: 8078163] CrossrefMedlineGoogle Scholar - 20.
Valley TS ,Sjoding MW ,Ryan AM ,Iwashyna TJ ,Cooke CR . Association of intensive care unit admission with mortality among older patients with pneumonia. JAMA. 2015;314:1272-9. [PMID: 26393850] doi:10.1001/jama.2015.11068 CrossrefMedlineGoogle Scholar - 21.
Bekelis K ,Marth NJ ,Wong K ,Zhou W ,Birkmeyer JD ,Skinner J . Primary stroke center hospitalization for elderly patients with stroke: implications for case fatality and travel times. JAMA Intern Med. 2016;176:1361-8. [PMID: 27455403] doi:10.1001/jamainternmed.2016.3919 CrossrefMedlineGoogle Scholar - 22.
Crooks VA ,Schuurman N . Interpreting the results of a modified gravity model: examining access to primary health care physicians in five Canadian provinces and territories. BMC Health Serv Res. 2012;12:230. [PMID: 22852816] doi:10.1186/1472-6963-12-230 CrossrefMedlineGoogle Scholar - 23.
Davidson R ,MacKinnon JG . Estimation and Inference in Econometrics. New York: Oxford Univ Pr; 1993. Google Scholar - 24.
Luo W ,Wang F . Measures of spatial accessibility to health care in a GIS environment: synthesis and a case study in the Chicago region. Environ Plan B Urban Anal City Sci. 2003;30:865-84. Google Scholar - 25.
McGrail MR ,Humphreys JS . Measuring spatial accessibility to primary care in rural areas: improving the effectiveness of the two-step floating catchment area method. Appl Geogr. 2009;29:533-41. CrossrefGoogle Scholar - 26.
WWAMI Rural Health Research Center . Rural Urban Commuting Area Data: ZIP Code RUCA Approximation. Version 2.0. Seattle: WWAMI Rural Health Research Center; 2006. Google Scholar - 27.
Winblad U ,Mor V ,McHugh JP ,Rahman M . ACO-affiliated hospitals reduced rehospitalizations from skilled nursing facilities faster than other hospitals. Health Aff (Millwood). 2017;36:67-73. [PMID: 28069848] doi:10.1377/hlthaff.2016.0759 CrossrefMedlineGoogle Scholar - 28.
McWilliams JM ,Chernew ME ,Landon BE ,Schwartz AL . Performance differences in year 1 of Pioneer Accountable Care Organizations. N Engl J Med. 2015;372:1927-36. [PMID: 25875195] doi:10.1056/NEJMsa1414929 CrossrefMedlineGoogle Scholar - 29. United States Census Bureau. American FactFinder. 2017. Accessed at https://factfinder.census.gov/faces/nav/jsf/pages/index.xhtml on 6 January 2019. Google Scholar
- 30.
McWilliams JM ,Gilstrap LG ,Stevenson DG ,Chernew ME ,Huskamp HA ,Grabowski DC . Changes in postacute care in the Medicare Shared Savings Program. JAMA Intern Med. 2017;177:518-26. [PMID: 28192556] doi:10.1001/jamainternmed.2016.9115 CrossrefMedlineGoogle Scholar - 31.
Baiocchi M ,Cheng J ,Small DS . Instrumental variable methods for causal inference. Stat Med. 2014;33:2297-340. [PMID: 24599889] doi:10.1002/sim.6128 CrossrefMedlineGoogle Scholar - 32.
Staiger D ,Stock J . Instrumental variables regression with weak instruments. Econometrica. 1997;65:557-86. CrossrefGoogle Scholar - 33.
Baum CF ,Schaffer ME ,Stillman S . Instrumental variables and GMM: estimation and testing. Stata J. 2003;3:1-31. CrossrefGoogle Scholar - 34.
Clough JD ,McClellan M . Implementing MACRA: implications for physicians and for physician leadership. JAMA. 2016;315:2397-8. [PMID: 27213914] doi:10.1001/jama.2016.7041 CrossrefMedlineGoogle Scholar
Author, Article, and Disclosure Information
University of Michigan Medical School and School of Public Health, Ann Arbor, Michigan (A.A.M.)
University of Michigan Medical School and Institute for Healthcare Policy and Innovation, Ann Arbor, Michigan (J.M.H.)
University of Michigan Medical School, School of Public Health, Gerald R. Ford School of Public Policy, and Institute for Healthcare Policy and Innovation, Ann Arbor, Michigan (J.Z.A.)
University of Michigan School of Public Health, and Institute for Healthcare Policy and Innovation, Ann Arbor, Michigan, and National Bureau of Economic Research, Cambridge, Massachusetts (E.C.N.)
University of Michigan Medical School, Ann Arbor, Michigan (P.L.Y.)
University of Michigan School of Public Health, Center for Evaluating Health Reform, Institute for Healthcare Policy and Innovation, Ann Arbor, Michigan (A.M.R.)
Note: Dr. Ryan had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Financial Support: Mr. Markovitz is supported by the Horowitz Foundation for Social Policy and by grant R36HS025615 from the Agency for Healthcare Research and Quality. Dr. Hollingsworth is supported by grants R01HS024728 and 1R01HS024525-01A1 from the Agency for Healthcare Research and Quality. Dr. Ryan is supported by grant R01AG047932 from the National Institute on Aging.
Disclosures: Dr. Hollingsworth reports other support from University of California, San Francisco, that had no effect on the analysis presented in this manuscript. Authors not named here have disclosed no conflicts of interest. Disclosures can also be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M18-2539.
Editors' Disclosures: Christine Laine, MD, MPH, Editor in Chief, reports that her spouse has stock options/holdings with Targeted Diagnostics and Therapeutics. Darren B. Taichman, MD, PhD, Executive Editor, reports that he has no financial relationships or interests to disclose. Cynthia D. Mulrow, MD, MSc, Senior Deputy Editor, reports that she has no relationships or interests to disclose. Jaya K. Rao, MD, MHS, Deputy Editor, reports that she has stock holdings/options in Eli Lilly and Pfizer. Catharine B. Stack, PhD, MS, Deputy Editor, Statistics, reports that she has stock holdings in Pfizer, Johnson & Johnson, and Colgate-Palmolive. Christina C. Wee, MD, MPH, Deputy Editor, reports employment with Beth Israel Deaconess Medical Center. Sankey V. Williams, MD, Deputy Editor, reports that he has no financial relationships or interests to disclose. Yu-Xiao Yang, MD, MSCE, Deputy Editor, reports that he has no financial relationships or interest to disclose.
Reproducible Research Statement:Study protocol: Available in the Supplement. Statistical code: Available at https://github.com/adammarkovitz. Data set: Available for purchase from ResDAC.
Corresponding Author: Andrew M. Ryan, PhD, Department of Health Management and Policy, University of Michigan School of Public Health, M3124 SPH II, 1415 Washington Heights, Ann Arbor, MI 48109-2029; e-mail, [email protected]
Current Author Addresses: Mr. Markovitz: UM Institute for Health Policy and Innovation, Center for Evaluating Health Reform, North Campus Research Complex, 2800 Plymouth Road, Building 16, 016-100N-28, Ann Arbor, MI 48109.
Dr. Hollingsworth: University of Michigan Medical School, Urology, North Campus Research Complex, 2800 Plymouth Road, Building 16, 016-112W, Ann Arbor, MI 48109-2800.
Dr. Ayanian: University of Michigan Medical School, UM Institute for Health Policy and Innovation, North Campus Research Complex, 2800 Plymouth Road, Building 16, 440E, Ann Arbor, MI 48109-2800.
Dr. Norton: University of Michigan School of Public Health, M3108 SPH II, 1415 Washington Heights, Ann Arbor, MI 48109-2029.
Ms. Yan: University of Michigan Medical School, Urology, North Campus Research Complex, 2800 Plymouth Road, Building 16, 016-100S-14, Ann Arbor, MI 48109-2800.
Dr. Ryan: University of Michigan School of Public Health, M3124 SPH II, 1415 Washington Heights, Ann Arbor, MI 48109-2029.
Author Contributions: Conception and design: A.A. Markovitz, J.M. Hollingsworth, J.Z. Ayanian, E.C. Norton, A.M. Ryan.
Analysis and interpretation of the data: A.A. Markovitz, J.M. Hollingsworth, J.Z. Ayanian, E.C. Norton, A.M. Ryan.
Drafting of the article: A.A. Markovitz.
Critical revision of the article for important intellectual content: A.A. Markovitz, J.M. Hollingsworth, J.Z. Ayanian, A.M. Ryan.
Final approval of the article: A.A. Markovitz, J.M. Hollingsworth, J.Z. Ayanian, E.C. Norton, P.L. Yan, A.M. Ryan.
Statistical expertise: A.A. Markovitz, E.C. Norton, A.M. Ryan.
Obtaining of funding: A.A. Markovitz, J.M. Hollingsworth.
Administrative, technical, or logistic support: A.M. Ryan.
Collection and assembly of data: J.M. Hollingsworth, P.L. Yan.
This article was published at Annals.org on 18 June 2019.
Submit a Comment
Contributors must reveal any conflict of interest. Comments are moderated. Please see our information for authorsregarding comments on an Annals publication.
*All comments submitted after October 1, 2021 and selected for publication will be published online only.