Original Research
21 June 2016

Comparing Use of Low-Value Health Care Services Among U.S. Advanced Practice Clinicians and Physicians

Publication: Annals of Internal Medicine
Volume 165, Number 4



Many physicians believe that advanced practice clinicians (APCs [nurse practitioners and physician assistants]) provide care of relatively lower value.


To compare use of low-value services among U.S. APCs and physicians.


Service use after primary care visits was evaluated for 3 conditions after adjustment for patient and provider characteristics and year. Patients with guideline-based red flags were excluded and analyses stratified by office- versus hospital-based visits, acute versus nonacute presentations, and whether clinicians self-identified as the patient's primary care provider (PCP).


National Ambulatory Medical Care Survey (NAMCS) and National Hospital Ambulatory Medical Care Survey (NHAMCS), 1997 to 2011.


Patients presenting with upper respiratory infections (URIs), back pain, or headache.


Use of guideline-discordant antibiotics (for URIs), radiography (for URIs and back pain), computed tomography (CT) or magnetic resonance imaging (MRI) (for headache and back pain), and referrals to other physicians (for all 3 conditions).


12 170 physician and 473 APC office-based visits and 13 359 physician and 2947 APC hospital-based visits were identified. Although office-based clinicians saw similar patients, hospital-based APCs saw younger patients (mean age, 42.6 vs. 45.0 years; P  < 0.001), and practiced in urban settings less frequently (49.7% vs. 81.7% of visits; P < 0.001) than hospital-based physicians. Unadjusted and adjusted results revealed that APCs ordered antibiotics, CT or MRI, radiography, and referrals as often as physicians in both settings. Stratification suggested that self-identified PCP APCs ordered more services than PCP physicians in the hospital-based setting.


NHAMCS reflects hospital-based APC care; NAMCS samples physician practices and likely underrepresents office-based APCs.


APCs and physicians provided an equivalent amount of low-value health services, dispelling physicians' perceptions that APCs provide lower-value care than physicians for these common conditions.

Primary Funding Source:

U.S. Health Services and Research Administration, Ryoichi Sasakawa Fellowship Fund, and National Institutes of Health.

Get full access to this article

View all available purchase options and get full access to this article.


Schwartz MD. Health care reform and the primary care workforce bottleneck. J Gen Intern Med. 2012;27:469-72. [PMID: 22042605]  doi: 10.1007/s11606-011-1921-4
Colwill JMCultice JMKruse RL. Will generalist physician supply meet demands of an increasing and aging population? Health Aff (Millwood). 2008;27:w232-41. [PMID: 18445642]  doi: 10.1377/hlthaff.27.3.w232
U.S. Department of Health and Human Services, Health Resources and Services Administration, National Center for Health Workforce Analysis. Projecting the Supply and Demand for Primary Care Practitioners Through 2020. Rockville, MD: U.S. Department of Health and Human Services; 2013. Accessed at http://bhpr.hrsa.gov/healthworkforce/supplydemand/usworkforce/primarycare/projectingprimarycare.pdf? on 20 January 2016.
Petterson SMLiaw WRPhillips RL JrRabin DLMeyers DSBazemore AW. Projecting US primary care physician workforce needs: 2010-2025. Ann Fam Med. 2012;10:503-9. [PMID: 23149526]  doi: 10.1370/afm.1431
Lipner RSBylsma WHArnold GKFortna GSTooker JCassel CK. Who is maintaining certification in internal medicine—and why? A national survey 10 years after initial certification. Ann Intern Med. 2006;144:29-36. [PMID: 16389252]
Landon BEReschovsky JDPham HHBlumenthal D. Leaving medicine: the consequences of physician dissatisfaction. Med Care. 2006;44:234-42. [PMID: 16501394]
Edwards STMafi JNLandon BE. Trends and quality of care in outpatient visits to generalist and specialist physicians delivering primary care in the United States, 1997-2010. J Gen Intern Med. 2014;29:947-55. [PMID: 24567201]  doi: 10.1007/s11606-014-2808-y
Bodenheimer TChen EBennett HD. Confronting the growing burden of chronic disease: can the U.S. health care workforce do the job? Health Aff (Millwood). 2009;28:64-74. [PMID: 19124856]  doi: 10.1377/hlthaff.28.1.64
Bodenheimer TSSmith MD. Primary care: proposed solutions to the physician shortage without training more physicians. Health Aff (Millwood). 2013;32:1881-6. [PMID: 24191075]  doi: 10.1377/hlthaff.2013.0234
Mehrotra ALiu HAdams JLWang MCLave JRThygeson NMet al. Comparing costs and quality of care at retail clinics with that of other medical settings for 3 common illnesses. Ann Intern Med. 2009;151:321-8. [PMID: 19721020]
Roblin DWHoward DHBecker ERKathleenAdams ERoberts MH. Use of midlevel practitioners to achieve labor cost savings in the primary care practice of an MCO. Health Serv Res. 2004;39:607-26. [PMID: 15149481]
Naylor MDKurtzman ET. The role of nurse practitioners in reinventing primary care. Health Aff (Millwood). 2010;29:893-9. [PMID: 20439877]  doi: 10.1377/hlthaff.2010.0440
Chenoweth DMartin NPankowski JRaymond LW. Nurse practitioner services: three-year impact on health care costs. J Occup Environ Med. 2008;50:1293-8. [PMID: 19001955]  doi: 10.1097/JOM.0b013e318184563a
Horrocks SAnderson ESalisbury C. Systematic review of whether nurse practitioners working in primary care can provide equivalent care to doctors. BMJ. 2002;324:819-23. [PMID: 11934775]
Martínez-González NADjalali STandjung RHuber-Geismann FMarkun SWensing Met al. Substitution of physicians by nurses in primary care: a systematic review and meta-analysis. BMC Health Serv Res. 2014;14:214. [PMID: 24884763]  doi: 10.1186/1472-6963-14-214
Wilson IBLandon BEHirschhorn LRMcInnes KDing LMarsden PVet al. Quality of HIV care provided by nurse practitioners, physician assistants, and physicians. Ann Intern Med. 2005;143:729-36. [PMID: 16287794]
Mundinger MOKane RLLenz ERTotten AMTsai WYCleary PDet al. Primary care outcomes in patients treated by nurse practitioners or physicians: a randomized trial. JAMA. 2000;283:59-68. [PMID: 10632281]
Donelan KDesRoches CMDittus RSBuerhaus P. Perspectives of physicians and nurse practitioners on primary care practice. N Engl J Med. 2013;368:1898-906. [PMID: 23675658]  doi: 10.1056/NEJMsa1212938
Fisher ESWennberg DEStukel TAGottlieb DJLucas FLPinder EL. The implications of regional variations in Medicare spending. Part 1: the content, quality, and accessibility of care. Ann Intern Med. 2003;138:273-87. [PMID: 12585825]
Mafi JNMcCarthy EPDavis RBLandon BE. Worsening trends in the management and treatment of back pain. JAMA Intern Med. 2013;173:1573-81. [PMID: 23896698]  doi: 10.1001/jamainternmed.2013.8992
Mafi JNEdwards STPedersen NPDavis RBMcCarthy EPLandon BE. Trends in the ambulatory management of headache: analysis of NAMCS and NHAMCS data 1999-2010. J Gen Intern Med. 2015;30:548-55. [PMID: 25567755]  doi: 10.1007/s11606-014-3107-3
Schwartz ALLandon BEElshaug AGChernew MEMcWilliams JM. Measuring low-value care in Medicare. JAMA Intern Med. 2014;174:1067-76. [PMID: 24819824]  doi: 10.1001/jamainternmed.2014.1541
Hsiao C. Understanding and using NAMCS and NHAMCS data: data tools and basic programming techniques. 2010. Accessed at www.cdc.gov/nchs/ppt/nchs2010/03_Hsiao.pdf on 20 January 2016.
Centers for Disease Control and Prevention. Using ultimate cluster models with NAMCS and NHAMCS public use files. 2004. Accessed at www.cdc.gov/nchs/data/ahcd/ultimatecluster.pdf on 20 January 2016.
Atwater ABednar SHassman DKhouri J. Nurse practitioners and physician assistants in primary care. Dis Mon. 2008;54:728-44. [PMID: 18940335]  doi: 10.1016/j.disamonth.2008.08.002
Barnett MLLinder JA. Antibiotic prescribing for adults with acute bronchitis in the United States, 1996-2010. JAMA. 2014;311:2020-2. [PMID: 24846041]  doi: 10.1001/jama.2013.286141
Barnett MLLinder JA. Antibiotic prescribing to adults with sore throat in the United States, 1997-2010. JAMA Intern Med. 2014;174:138-40. [PMID: 24091806]  doi: 10.1001/jamainternmed.2013.11673
Gonzales RBartlett JGBesser RECooper RJHickner JMHoffman JRet al. Principles of appropriate antibiotic use for treatment of acute respiratory tract infections in adults: background, specific aims, and methods. Ann Intern Med. 2001;134:479-86. [PMID: 11255524]
Chou RQaseem ASnow VCasey DCross JT JrShekelle Pet alClinical Efficacy Assessment Subcommittee of the American College of Physicians. Diagnosis and treatment of low back pain: a joint clinical practice guideline from the American College of Physicians and the American Pain Society. Ann Intern Med. 2007;147:478-91. [PMID: 17909209]
National Committee for Quality Assurance. Summary table of measure changes. 2015. Accessed at www.ncqa.org/Portals/0/HEDISQM/Hedis2015/HEDIS%202015%20Physician%20Measures%20List.pdf on 20 January 2016.
Goertz MThorson DBonsell JBonte BCampbell RHaake Bet alInstitute for Clinical Systems Improvement. Adult Acute and Subacute Low Back Pain. Bloomington, MN: Institute for Clinical Systems Improvement; 2012.
Hainer BLMatheson EM. Approach to acute headache in adults. Am Fam Physician. 2013;87:682-7. [PMID: 23939446]
Peek METang HAlexander GCChin MH. National prevalence of lifestyle counseling or referral among African-Americans and whites with diabetes. J Gen Intern Med. 2008;23:1858-64. [PMID: 18683005]  doi: 10.1007/s11606-008-0737-3
Hemani ARastegar DAHill Cal-Ibrahim MS. A comparison of resource utilization in nurse practitioners and physicians. Eff Clin Pract. 1999;2:258-65. [PMID: 10788023]
Martin-Misener RHarbman PDonald FReid KKilpatrick KCarter Net al. Cost-effectiveness of nurse practitioners in primary and specialised ambulatory care: systematic review. BMJ Open. 2015;5:e007167. [PMID: 26056121]  doi: 10.1136/bmjopen-2014-007167
Iglehart JK. Expanding the role of advanced nurse practitioners—risks and rewards. N Engl J Med. 2013;368:1935-41. [PMID: 23675663]  doi: 10.1056/NEJMhpr1301084
Linder JASinger DEStafford RS. Association between antibiotic prescribing and visit duration in adults with upper respiratory tract infections. Clin Ther. 2003;25:2419-30. [PMID: 14604741]
Gonzales RBarrett PH JrCrane LASteiner JF. Factors associated with antibiotic use for acute bronchitis. J Gen Intern Med. 1998;13:541-8. [PMID: 9734791]
Kuo YFLoresto FL JrRounds LRGoodwin JS. States with the least restrictive regulations experienced the largest increase in patients seen by nurse practitioners. Health Aff (Millwood). 2013;32:1236-43. [PMID: 23836739]  doi: 10.1377/hlthaff.2013.0072
Hughes DRJiang MDuszak R Jr. A comparison of diagnostic imaging ordering patterns between advanced practice clinicians and primary care physicians following office-based evaluation and management visits. JAMA Intern Med. 2015;175:101-7. [PMID: 25419763]  doi: 10.1001/jamainternmed.2014.6349
Morgan PAStrand JØstbye TAlbanese MA. Missing in action: care by physician assistants and nurse practitioners in national health surveys. Health Serv Res. 2007;42:2022-37. [PMID: 17850531]
Gadbois EAMiller EATyler DIntrator O. Trends in state regulation of nurse practitioners and physician assistants, 2001 to 2010. Med Care Res Rev. 2015;72:200-19. [PMID: 25542195]  doi: 10.1177/1077558714563763
Christian S, Dower C, O’Neil E. Overview of Nurse Practitioner Scopes of Practice in the United States—Discussion. San Francisco, CA: UCSF Center for the Health Professions; 2007. Accessed at http://enp-network.s3.amazonaws.com/NP_Alliance/PDF/NPScopes.pdf on 6 June 2016.
Stange K. How does provider supply and regulation influence health care markets? Evidence from nurse practitioners and physician assistants. J Health Econ. 2014;33:1-27. [PMID: 24240144]  doi: 10.1016/j.jhealeco.2013.10.009
McCaig LFMcLemore T. Plan and operation of the National Hospital Ambulatory Medical Survey. Series 1: programs and collection procedures. Vital Health Stat 1. 1994:1-78. [PMID: 7975355]

Information & Authors


Published In

cover image Annals of Internal Medicine
Annals of Internal Medicine
Volume 165Number 416 August 2016
Pages: 237 - 244


Published online: 21 June 2016
Published in issue: 16 August 2016




John N. Mafi, MD, MPH
From David Geffen School of Medicine at UCLA, Los Angeles, California; RAND Corporation, Santa Monica, California; and Beth Israel Deaconess Medical Center and Harvard Medical School, Boston, Massachusetts.
Christina C. Wee, MD, MPH
From David Geffen School of Medicine at UCLA, Los Angeles, California; RAND Corporation, Santa Monica, California; and Beth Israel Deaconess Medical Center and Harvard Medical School, Boston, Massachusetts.
Roger B. Davis, ScD
From David Geffen School of Medicine at UCLA, Los Angeles, California; RAND Corporation, Santa Monica, California; and Beth Israel Deaconess Medical Center and Harvard Medical School, Boston, Massachusetts.
Bruce E. Landon, MD, MBA, MSc
From David Geffen School of Medicine at UCLA, Los Angeles, California; RAND Corporation, Santa Monica, California; and Beth Israel Deaconess Medical Center and Harvard Medical School, Boston, Massachusetts.
Presented in oral abstract form at the New England Regional Meeting of the Society of General Internal Medicine (SGIM) in Boston, Massachusetts, March 2015, and at the SGIM 38th Annual Meeting, in Toronto, Ontario, Canada, April 2015, and in abstract form at the Annual Research Meeting of AcademyHealth, in Minneapolis, Minnesota, June 2015.
Acknowledgment: The authors acknowledge David Wilson, PhD, a senior research staff consultant with RTI International, SUDAAN Software Division.
Financial Support: Dr. Mafi was supported by National Research Service Award training grant T32HP12706 from the U.S. Health Services and Research Administration and by the Ryoichi Sasakawa Fellowship Fund. Dr. Wee was supported by National Institutes of Health Midcareer Mentorship Award K24DK087932. Dr. Davis was supported by Harvard Catalyst National Institutes of Health Award UL1 TR001102.
Disclosures: Dr. Davis reports grants from the National Institutes of Health during the conduct of the study and personal fees from the American Heart Association outside the submitted work. Dr. Landon reports personal fees from United Biosource and Research Triangle Institute outside the submitted work. Authors not named here have disclosed no conflicts of interest. Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M15-2152.
Editors' Disclosures: Christine Laine, MD, MPH, Editor in Chief, reports that she has no financial relationships or interests to disclose. Darren B. Taichman, MD, PhD, Executive Deputy Editor, reports that he has no financial relationships or interests to disclose. Cynthia D. Mulrow, MD, MSc, Senior Deputy Editor, reports that she has no relationships or interests to disclose. Deborah Cotton, MD, MPH, Deputy Editor, reports that she has no financial relationships or interest to disclose. Jaya K. Rao, MD, MHS, Deputy Editor, reports that she has stock holdings/options in Eli Lilly and Pfizer. Sankey V. Williams, MD, Deputy Editor, reports that he has no financial relationships or interests to disclose. Catharine B. Stack, PhD, MS, Deputy Editor for Statistics, reports that she has stock holdings in Pfizer.
Reproducible Research Statement: Study protocol: Available from Dr. Mafi (e-mail, [email protected]). Statistical code: Interested readers may contact Dr. Mafi for discussion. Data set: Available from Centers for Disease Control and Prevention, National Center for Health Statistics (www.cdc.gov/nchs/ahcd/).
Corresponding Author: John N. Mafi, MD, MPH, Division of General Internal Medicine and Health Services Research, Department of Medicine, Ronald Reagan UCLA Medical Center, 911 Broxton Avenue #301, Los Angeles, CA 90024; e-mail, [email protected].
Current Author Addresses: Dr. Mafi: Division of General Internal Medicine and Health Services Research, UCLA David Geffen School of Medicine, 911 Broxton Avenue #301, Los Angeles, CA 90024.
Drs. Wee and Davis: Division of General Medicine and Primary Care, Beth Israel Deaconess Medical Center, 330 Brookline Avenue, Boston, MA 02215.
Dr. Landon: Department of Health Care Policy, Harvard Medical School, 180 Longwood Avenue, Boston, MA 02115.
Author Contributions: Conception and design: J.N. Mafi, C.C. Wee, B.E. Landon.
Analysis and interpretation of the data: J.N. Mafi, R.B. Davis, B.E. Landon.
Drafting of the article: J.N. Mafi.
Critical revision for important intellectual content: J.N. Mafi, C.C. Wee, R.B. Davis, B.E. Landon.
Final approval of the article: J.N. Mafi, C.C. Wee, R.B. Davis, B.E. Landon.
Statistical expertise: R.B. Davis.
Obtaining of funding: C.C. Wee.
Administrative, technical, or logistic support: J.N. Mafi, B.E. Landon.
Collection and assembly of data: J.N. Mafi.
This article was published at www.annals.org on 21 June 2016.

Metrics & Citations



If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. For an editable text file, please select Medlars format which will download as a .txt file. Simply select your manager software from the list below and click Download.

For more information or tips please see 'Downloading to a citation manager' in the Help menu.


Download article citation data for:
John N. Mafi, Christina C. Wee, Roger B. Davis, et al. Comparing Use of Low-Value Health Care Services Among U.S. Advanced Practice Clinicians and Physicians. Ann Intern Med.2016;165:237-244. [Epub 21 June 2016]. doi:10.7326/M15-2152

View More

Get Access

Login Options:

You will be redirected to acponline.org to sign-in to Annals to complete your purchase.

Create your Free Account

You will be redirected to acponline.org to create an account that will provide access to Annals.

View options


View PDF/ePub

Related in ACP Journals

Full Text

View Full Text







Copy the content Link

Share on social media