Original Research
17 February 2015

An Analysis of Calibration and Discrimination Among Multiple Cardiovascular Risk Scores in a Modern Multiethnic Cohort

Publication: Annals of Internal Medicine
Volume 162, Number 4

Abstract

Background:

Accurate risk assessment of atherosclerotic cardiovascular disease (ASCVD) is essential to effectively balance the risks and benefits of therapy for primary prevention.

Objective:

To compare the calibration and discrimination of the new American Heart Association (AHA) and American College of Cardiology (ACC) ASCVD risk score with alternative risk scores and to explore preventive therapy as a cause of the reported risk overestimation using the AHA-ACC-ASCVD score.

Design:

Prospective epidemiologic study of ASCVD.

Setting:

MESA (Multi-Ethnic Study of Atherosclerosis), a community-based, sex-balanced, multiethnic cohort.

Patients:

4227 MESA participants aged 50 to 74 years and without diabetes at baseline.

Measurements:

Observed and expected events for the AHA-ACC-ASCVD score were compared with 4 commonly used risk scores—and their respective end points—in MESA after a 10.2-year follow-up.

Results:

The new AHA-ACC-ASCVD and 3 older Framingham-based risk scores overestimated cardiovascular events by 37% to 154% in men and 8% to 67% in women. Overestimation was noted throughout the continuum of risk. In contrast, the Reynolds Risk Score overestimated risk by 9% in men but underestimated risk by 21% in women. Aspirin, lipid-lowering or antihypertensive therapy, and interim revascularization did not explain the overestimation.

Limitation:

Comparability of MESA with target populations for primary prevention and possibility of missed events in MESA.

Conclusion:

Of the 5 risk scores, 4, including the new AHA-ACC-ASCVD score, showed overestimation of risk (25% to 115%) in a modern, multiethnic cohort without baseline clinical ASCVD. If validated, overestimation of ASCVD risk may have substantial implications for individual patients and the health care system.

Primary Funding Source:

National Heart, Lung, and Blood Institute.

Get full access to this article

View all available purchase options and get full access to this article.

References

1.
Fletcher BBerra KAdes PBraun LTBurke LEDurstine JLet alCouncil on Cardiovascular Nursing. Managing abnormal blood lipids: a collaborative approach. Circulation. 2005;112:3184-209. [PMID: 16286609]
2.
Goldstein LBBushnell CDAdams RJAppel LJBraun LTChaturvedi Set alAmerican Heart Association Stroke Council. Guidelines for the primary prevention of stroke: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. 2011;42:517-84. [PMID: 21127304]  doi: 10.1161/STR.0b013e3181fcb238
3.
Mosca LBenjamin EJBerra KBezanson JLDolor RJLloyd-Jones DMet al. Effectiveness-based guidelines for the prevention of cardiovascular disease in women—2011 update: a guideline from the American Heart Association. Circulation. 2011;123:1243-62. [PMID: 21325087]  doi: 10.1161/CIR.0b013e31820faaf8
4.
Wilson PWD'Agostino RBLevy DBelanger AMSilbershatz HKannel WB. Prediction of coronary heart disease using risk factor categories. Circulation. 1998;97:1837-47. [PMID: 9603539]
5.
National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III). Third Report of the National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III) final report. Circulation. 2002;106:3143-421. [PMID: 12485966]
6.
Bastuji-Garin SDeverly AMoyse DCastaigne AMancia Gde Leeuw PWet alIntervention as a Goal in Hypertension Treatment Study Group. The Framingham prediction rule is not valid in a European population of treated hypertensive patients. J Hypertens. 2002;20:1973-80. [PMID: 12359975]
7.
Brindle PEmberson JLampe FWalker MWhincup PFahey Tet al. Predictive accuracy of the Framingham coronary risk score in British men: prospective cohort study. BMJ. 2003;327:1267. [PMID: 14644971]
8.
Cook NRPaynter NPEaton CBManson JEMartin LWRobinson JGet al. Comparison of the Framingham and Reynolds risk scores for global cardiovascular risk prediction in the multiethnic Women's Health Initiative. Circulation. 2012;125:1748-56, S1-11. [PMID: 22399535]  doi: 10.1161/CIRCULATIONAHA.111.075929
9.
Liu JHong YD'Agostino RB SrWu ZWang WSun Jet al. Predictive value for the Chinese population of the Framingham CHD risk assessment tool compared with the Chinese Multi-Provincial Cohort Study. JAMA. 2004;291:2591-9. [PMID: 15173150]
10.
Ridker PMCook NR. Statins: new American guidelines for prevention of cardiovascular disease. Lancet. 2013;382:1762-5. [PMID: 24268611]  doi: 10.1016/S0140-6736(13)62388-0
11.
D'Agostino RB SrVasan RSPencina MJWolf PACobain MMassaro JMet al. General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation. 2008;117:743-53. [PMID: 18212285]  doi: 10.1161/CIRCULATIONAHA.107.699579
12.
Ridker PMBuring JERifai NCook NR. Development and validation of improved algorithms for the assessment of global cardiovascular risk in women: the Reynolds Risk Score. JAMA. 2007;297:611-9. [PMID: 17299196]
13.
Goff DC JrLloyd-Jones DMBennett GCoady SD'Agostino RBGibbons Ret alAmerican College of Cardiology/American Heart Association Task Force on Practice Guidelines. 2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation. 2014;129:S49-73. [PMID: 24222018]  doi: 10.1161/01.cir.0000437741.48606.98
14.
Fried LPBorhani NOEnright PFurberg CDGardin JMKronmal RAet al. The Cardiovascular Health Study: design and rationale. Ann Epidemiol. 1991;1:263-76. [PMID: 1669507]
15.
Friedman GDCutter GRDonahue RPHughes GHHulley SBJacobs DR Jret al. CARDIA: study design, recruitment, and some characteristics of the examined subjects. J Clin Epidemiol. 1988;41:1105-16. [PMID: 3204420]
16.
The ARIC investigators. The Atherosclerosis Risk in Communities (ARIC) Study: design and objectives. Am J Epidemiol. 1989;129:687-702. [PMID: 2646917]
17.
Kavousi MLeening MJNanchen DGreenland PGraham IMSteyerberg EWet al. Comparison of application of the ACC/AHA guidelines, Adult Treatment Panel III guidelines, and European Society of Cardiology guidelines for cardiovascular disease prevention in a European cohort. JAMA. 2014;311:1416-23. [PMID: 24681960]  doi: 10.1001/jama.2014.2632
18.
Muntner PColantonio LDCushman MGoff DC JrHoward GHoward VJet al. Validation of the atherosclerotic cardiovascular disease Pooled Cohort risk equations. JAMA. 2014;311:1406-15. [PMID: 24682252]  doi: 10.1001/jama.2014.2630
19.
Stone NJRobinson JGLichtenstein AHBaireyMerz CNBlum CBEckel RHet alAmerican College of Cardiology/American Heart Association Task Force on Practice Guidelines. 2013 ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation. 2014;129:S1-45. [PMID: 24222016]  doi: 10.1161/01.cir.0000437738.63853.7a
20.
Bild DEBluemke DABurke GLDetrano RDiez Roux AVFolsom ARet al. Multi-ethnic study of atherosclerosis: objectives and design. Am J Epidemiol. 2002;156:871-81. [PMID: 12397006]
21.
Gordon WJPolansky JMBoscardin WJFung KZSteinman MA. Coronary risk assessment by point-based vs. equation-based Framingham models: significant implications for clinical care. J Gen Intern Med. 2010;25:1145-51. [PMID: 20824362]  doi: 10.1007/s11606-010-1454-2
22.
Ridker PMPaynter NPRifai NGaziano JMCook NR. C-reactive protein and parental history improve global cardiovascular risk prediction: the Reynolds Risk Score for men. Circulation. 2008;118:2243-51. [PMID: 18997194]  doi: 10.1161/CIRCULATIONAHA.108.814251
23.
Crane PKWalker RHubbard RALi GNathan DMZheng Het al. Glucose levels and risk of dementia. N Engl J Med. 2013;369:540-8. [PMID: 23924004]  doi: 10.1056/NEJMoa1215740
24.
Sacks DB. Translating hemoglobin A1c into average blood glucose: implications for clinical chemistry. Clin Chem. 2008;54:1756-8. [PMID: 18931015]  doi: 10.1373/clinchem.2008.113282
25.
Harrell FE JrLee KLMark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15:361-87. [PMID: 8668867]
26.
U.S. Preventive Services Task Force. Aspirin for the prevention of cardiovascular disease: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med. 2009;150:396-404. [PMID: 19293072]
27.
Kannel WBFeinleib MMcNamara PMGarrison RJCastelli WP. An investigation of coronary heart disease in families. The Framingham offspring study. Am J Epidemiol. 1979;110:281-90. [PMID: 474565]
28.
Hoffmann DHoffmann I. The changing cigarette, 1950-1995. J Toxicol Environ Health. 1997;50:307-64. [PMID: 9120872]
29.
Thun MJCarter BDFeskanich DFreedman NDPrentice RLopez ADet al. 50-year trends in smoking-related mortality in the United States. N Engl J Med. 2013;368:351-64. [PMID: 23343064]  doi: 10.1056/NEJMsa1211127
30.
Ford ESAjani UACroft JBCritchley JALabarthe DRKottke TEet al. Explaining the decrease in U.S. deaths from coronary disease, 1980-2000. N Engl J Med. 2007;356:2388-98. [PMID: 17554120]
31.
Erbel RMöhlenkamp SMoebus SSchmermund ALehmann NStang Aet alHeinz Nixdorf Recall Study Investigative Group. Coronary risk stratification, discrimination, and reclassification improvement based on quantification of subclinical coronary atherosclerosis: the Heinz Nixdorf Recall study. J Am Coll Cardiol. 2010;56:1397-406. [PMID: 20946997]  doi: 10.1016/j.jacc.2010.06.030
32.
Curb JDMcTiernan AHeckbert SRKooperberg CStanford JNevitt Met alWHI Morbidity and Mortality Committee. Outcomes ascertainment and adjudication methods in the Women's Health Initiative. Ann Epidemiol. 2003;13:S122-8. [PMID: 14575944]
33.
Ives DGFitzpatrick ALBild DEPsaty BMKuller LHCrowley PMet al. Surveillance and ascertainment of cardiovascular events. The Cardiovascular Health Study. Ann Epidemiol. 1995;5:278-85. [PMID: 8520709]
34.
Brandle MDavidson MBSchriger DLLorber BHerman WH. Cost effectiveness of statin therapy for the primary prevention of major coronary events in individuals with type 2 diabetes. Diabetes Care. 2003;26:1796-801. [PMID: 12766112]
35.
Mitchell APSimpson RJ. Statin cost effectiveness in primary prevention: a systematic review of the recent cost-effectiveness literature in the United States. BMC Res Notes. 2012;5:373. [PMID: 22828389]  doi: 10.1186/1756-0500-5-373

Information & Authors

Information

Published In

cover image Annals of Internal Medicine
Annals of Internal Medicine
Volume 162Number 417 February 2015
Pages: 266 - 275

History

Published online: 17 February 2015
Published in issue: 17 February 2015

Keywords

Authors

Affiliations

Andrew P. DeFilippis, MD, MSc
From the University of Louisville, Louisville, Kentucky; University of Washington, Seattle, Washington; University of Colorado, Boulder, Colorado; Johns Hopkins Medicine Ciccarone Center for the Prevention of Heart Disease, Baltimore, Maryland; Los Angeles Biomedical Research Institute at Harbor–UCLA Medical Center, Torrance, California; and Baptist Medical Group, Miami, Florida.
Rebekah Young, PhD
From the University of Louisville, Louisville, Kentucky; University of Washington, Seattle, Washington; University of Colorado, Boulder, Colorado; Johns Hopkins Medicine Ciccarone Center for the Prevention of Heart Disease, Baltimore, Maryland; Los Angeles Biomedical Research Institute at Harbor–UCLA Medical Center, Torrance, California; and Baptist Medical Group, Miami, Florida.
Christopher J. Carrubba, MD
From the University of Louisville, Louisville, Kentucky; University of Washington, Seattle, Washington; University of Colorado, Boulder, Colorado; Johns Hopkins Medicine Ciccarone Center for the Prevention of Heart Disease, Baltimore, Maryland; Los Angeles Biomedical Research Institute at Harbor–UCLA Medical Center, Torrance, California; and Baptist Medical Group, Miami, Florida.
John W. McEvoy, MB, BCh, BAO
From the University of Louisville, Louisville, Kentucky; University of Washington, Seattle, Washington; University of Colorado, Boulder, Colorado; Johns Hopkins Medicine Ciccarone Center for the Prevention of Heart Disease, Baltimore, Maryland; Los Angeles Biomedical Research Institute at Harbor–UCLA Medical Center, Torrance, California; and Baptist Medical Group, Miami, Florida.
Matthew J. Budoff, MD
From the University of Louisville, Louisville, Kentucky; University of Washington, Seattle, Washington; University of Colorado, Boulder, Colorado; Johns Hopkins Medicine Ciccarone Center for the Prevention of Heart Disease, Baltimore, Maryland; Los Angeles Biomedical Research Institute at Harbor–UCLA Medical Center, Torrance, California; and Baptist Medical Group, Miami, Florida.
Roger S. Blumenthal, MD
From the University of Louisville, Louisville, Kentucky; University of Washington, Seattle, Washington; University of Colorado, Boulder, Colorado; Johns Hopkins Medicine Ciccarone Center for the Prevention of Heart Disease, Baltimore, Maryland; Los Angeles Biomedical Research Institute at Harbor–UCLA Medical Center, Torrance, California; and Baptist Medical Group, Miami, Florida.
Richard A. Kronmal, PhD
From the University of Louisville, Louisville, Kentucky; University of Washington, Seattle, Washington; University of Colorado, Boulder, Colorado; Johns Hopkins Medicine Ciccarone Center for the Prevention of Heart Disease, Baltimore, Maryland; Los Angeles Biomedical Research Institute at Harbor–UCLA Medical Center, Torrance, California; and Baptist Medical Group, Miami, Florida.
Robyn L. McClelland, PhD
From the University of Louisville, Louisville, Kentucky; University of Washington, Seattle, Washington; University of Colorado, Boulder, Colorado; Johns Hopkins Medicine Ciccarone Center for the Prevention of Heart Disease, Baltimore, Maryland; Los Angeles Biomedical Research Institute at Harbor–UCLA Medical Center, Torrance, California; and Baptist Medical Group, Miami, Florida.
Khurram Nasir, MD, MPH
From the University of Louisville, Louisville, Kentucky; University of Washington, Seattle, Washington; University of Colorado, Boulder, Colorado; Johns Hopkins Medicine Ciccarone Center for the Prevention of Heart Disease, Baltimore, Maryland; Los Angeles Biomedical Research Institute at Harbor–UCLA Medical Center, Torrance, California; and Baptist Medical Group, Miami, Florida.
Michael J. Blaha, MD, MPH
From the University of Louisville, Louisville, Kentucky; University of Washington, Seattle, Washington; University of Colorado, Boulder, Colorado; Johns Hopkins Medicine Ciccarone Center for the Prevention of Heart Disease, Baltimore, Maryland; Los Angeles Biomedical Research Institute at Harbor–UCLA Medical Center, Torrance, California; and Baptist Medical Group, Miami, Florida.
Acknowledgment: The authors thank the other investigators, the staff, and the participants of MESA for their valuable contributions. A full list of participating MESA investigators and institutions can be found at www.mesa-nhlbi.org.
Grant Support: This research was supported by contracts N01-HC-95159 through N01-HC-95169 from the National Heart, Lung, and Blood Institute and by grants UL1-TR-000040 and UL1-TR-001079 from the National Center for Research Resources.
Reproducible Research Statement: Study protocol: Available at www.mesa-nhlbi.org. Statistical code: Requests will be considered by Dr. Rebekah Young on a case-by-case basis. Data set: Applications for use of MESA data are available to the public at www.mesa-nhlbi.org.
Corresponding Author: Andrew P. DeFilippis, MD, MSc, University of Louisville, Division of Cardiovascular Medicine, 550 South Jackson Street, Louisville, KY 40202; e-mail, [email protected].
Current Author Addresses: Dr. DeFilippis: University of Louisville, Division of Cardiovascular Medicine, Ambulatory Care Building, Cardiology, 550 South Jackson Street, Louisville, KY 40202.
Dr. Young: Collaborative Health Studies Coordinating Center, Building 29, Suite 210, University of Washington, Box 354922, 6200 Northeast 74th Street, Seattle, WA 98115.
Dr. Carrubba: University of Colorado, 12631 East 17th Avenue, B177, Academic Office 1, Aurora, CO 80045.
Mr. McEvoy and Dr. Blumenthal: The Johns Hopkins Hospital, Blalock 524C, 1800 Orleans Street, Baltimore, MD 21287.
Dr. Budoff: Los Angeles Biomedical Research Institute at Harbor–UCLA Medical Center, 1124 West Carson Street, Torrance, CA 90502.
Drs. Kronmal and McClelland: Collaborative Health Studies Coordinating Center, Building 29, Suite 210, University of Washington, Box 354922, 6200 Northeast 74th Street, Seattle, WA 98115.
Dr. Nasir: Baptist Health Medical Group, Center for Prevention and Wellness Research, 1691 Michigan Avenue, Suite 500, Miami, FL 33139.
Dr. Blaha: Johns Hopkins Ciccarone Center for the Prevention of Heart Disease, Carnegie 568A, Johns Hopkins Hospital, 600 North Wolfe Street, Baltimore, MD 21287.
Author Contributions: Conception and design: A.P. DeFilippis, R. Young, M.J. Blaha.
Analysis and interpretation of the data: A.P. DeFilippis, R. Young, C. Carrubba, R.S. Blumenthal, R.A. Kronmal, R.L. McClelland, M.J. Blaha.
Drafting of the article: A.P. DeFilippis, R. Young, C. Carrubba, R.A. Kronmal, M.J. Blaha.
Critical revision of the article for important intellectual content: A.P. DeFilippis, R. Young, J.W. McEvoy, M.J. Budoff, R.S. Blumenthal, R.A. Kronmal, R.L. McClelland, K. Nasir, M.J. Blaha.
Final approval of the article: A.P. DeFilippis, R. Young, J.W. McEvoy, M.J. Budoff, R.S. Blumenthal, R.A. Kronmal, R.L. McClelland, K. Nasir, M.J. Blaha.
Provision of study materials or patients: M.J. Budoff.
Statistical expertise: R. Young, R.A. Kronmal, R.L. McClelland, M.J. Blaha.
Administrative, technical, or logistic support: A.P. DeFilippis, R.L. McClelland, M.J. Blaha.
Collection and assembly of data: R. Young, R.A. Kronmal, R.L. McClelland, M.J. Blaha.
* Drs. DeFilippis and Young contributed equally to this work.

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. For an editable text file, please select Medlars format which will download as a .txt file. Simply select your manager software from the list below and click Download.

For more information or tips please see 'Downloading to a citation manager' in the Help menu.

Format





Download article citation data for:
Andrew P. DeFilippis, Rebekah Young, Christopher J. Carrubba, et al. An Analysis of Calibration and Discrimination Among Multiple Cardiovascular Risk Scores in a Modern Multiethnic Cohort. Ann Intern Med.2015;162:266-275. [Epub 17 February 2015]. doi:10.7326/M14-1281

View More

Get Access

Login Options:
Purchase

You will be redirected to acponline.org to sign-in to Annals to complete your purchase.

Create your Free Account

You will be redirected to acponline.org to create an account that will provide access to Annals.

View options

PDF/ePub

View PDF/ePub

Related in ACP Journals

Full Text

View Full Text

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share on social media