Reviews21 April 2015
A Systematic Review and Meta-analysis
    Author, Article, and Disclosure Information



    One driver of increasing health care costs is the use of radiologic imaging procedures. More appropriate use could improve quality and reduce costs.


    To review interventions that use the computerized clinical decision-support (CCDS) capabilities of electronic health records to improve appropriate use of diagnostic radiologic test ordering.

    Data Sources:

    English-language articles in PubMed from 1995 to September 2014 and searches in Web of Science and PubMed of citations related to key articles.

    Study Selection:

    23 studies, including 3 randomized trials, 7 time-series studies, and 13 pre–post studies that assessed the effect of CCDS on diagnostic radiologic test ordering in adults.

    Data Extraction:

    2 independent reviewers extracted data on functionality, study outcomes, and context and assessed the quality of included studies.

    Data Synthesis:

    Thirteen studies provided moderate-level evidence that CCDS improves appropriateness (effect size, −0.49 [95% CI, −0.71 to −0.26]) and reduces use (effect size, −0.13 [CI, −0.23 to −0.04]). Interventions with a “hard stop” that prevents a clinician from overriding the CCDS without outside consultation, as well as interventions in integrated care delivery systems, may be more effective. Harms have rarely been assessed but include decreased ordering of appropriate tests and physician dissatisfaction.


    Potential for publication bias, insufficient reporting of harms, and poor description of context and implementation.


    Computerized clinical decision support integrated with the electronic health record can improve appropriate use of diagnostic radiology by a moderate amount and decrease use by a small amount. Before widespread adoption can be recommended, more data are needed on potential harms.

    Primary Funding Source:

    U.S. Department of Veterans Affairs. (PROSPERO registration number: CRD42014007469)


    • 1. Kocher KEMeurer WJFazel RScott PAKrumholz HMNallamothu BKNational trends in use of computed tomography in the emergency department. Ann Emerg Med2011;58:452-62. [PMID: 21835499] doi:10.1016/j.annemergmed.2011.05.020 CrossrefMedlineGoogle Scholar
    • 2. Korley FKPham JCKirsch TDUse of advanced radiology during visits to U.S. emergency departments for injury-related conditions, 1998-2007. JAMA2010;304:1465-71. [PMID: 20924012] doi:10.1001/jama.2010.1408 CrossrefMedlineGoogle Scholar
    • 3. Chou RFu RCarrino JADeyo RAImaging strategies for low-back pain: systematic review and meta-analysis. Lancet2009;373:463-72. [PMID: 19200918] doi:10.1016/S0140-6736(09)60172-0 CrossrefMedlineGoogle Scholar
    • 4. Moher DLiberati ATetzlaff JAltman DGPRISMA GroupPreferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med2009;151:264-9. [PMID: 19622511] LinkGoogle Scholar
    • 5. Chaudhry BWang JWu SMaglione MMojica WRoth Eet alSystematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med2006;144:742-52. [PMID: 16702590] LinkGoogle Scholar
    • 6. Buntin MBBurke MFHoaglin MCBlumenthal DThe benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Aff (Millwood)2011;30:464-71. [PMID: 21383365] doi:10.1377/hlthaff.2011.0178 CrossrefMedlineGoogle Scholar
    • 7. Goldzweig CLOrshansky GPaige NMTowfigh AAHaggstrom DAMiake-Lye Iet alElectronic patient portals: evidence on health outcomes, satisfaction, efficiency, and attitudes: a systematic review. Ann Intern Med2013;159:677-87. [PMID: 24247673] doi:10.7326/0003-4819-159-10-201311190-00006 LinkGoogle Scholar
    • 8. Jones SSRudin RSPerry TShekelle PGHealth information technology: an updated systematic review with a focus on meaningful use. Ann Intern Med2014;160:48-54. [PMID: 24573664] doi:10.7326/M13-1531 LinkGoogle Scholar
    • 9. Georgiou APrgomet MMarkewycz AAdams EWestbrook JIThe impact of computerized provider order entry systems on medical-imaging services: a systematic review. J Am Med Inform Assoc2011;18:335-40. [PMID: 21385821] doi:10.1136/amiajnl-2010-000043 CrossrefMedlineGoogle Scholar
    • 10. Roshanov PSMisra SGerstein HCGarg AXSebaldt RJMackay JAet alCCDSS Systematic Review TeamComputerized clinical decision support systems for chronic disease management: a decision maker–researcher partnership systematic review. Implement Sci2011;6:92. [PMID: 21824386] doi:10.1186/1748-5908-6-92 CrossrefMedlineGoogle Scholar
    • 11. Fillmore CLBray BEKawamoto KSystematic review of clinical decision support interventions with potential for inpatient cost reduction. BMC Med Inform Decis Mak2013;13:135. [PMID: 24344752] doi:10.1186/1472-6947-13-135 CrossrefMedlineGoogle Scholar
    • 12. Borenstein MHedges LVHiggins JPTRothstein HRChapter 7: Converting among effect sizes.. In: Borenstein M, Hedges LV, Higgins JPT, Rothstein HR, eds. Introduction to Meta-Analysis. Chichester, UK: J Wiley; 2011:45-9. Google Scholar
    • 13. Sánchez-Meca JMaríen-Martíinez FChacón-Moscoso SEffect-size indices for dichotomized outcomes in meta-analysis. Psychol Methods2003;8:448-67. [PMID: 14664682] CrossrefMedlineGoogle Scholar
    • 14. IntHout JIoannidis JPBorm GFThe Hartung–Knapp–Sidik–Jonkman method for random effects meta-analysis is straightforward and considerably outperforms the standard DerSimonian–Laird method. BMC Med Res Methodol2014;14:25. [PMID: 24548571] doi:10.1186/1471-2288-14-25 CrossrefMedlineGoogle Scholar
    • 15. Bates DWKuperman GJJha ATeich JMOrav EJMa'luf Net alDoes the computerized display of charges affect inpatient ancillary test utilization? Arch Intern Med1997;157:2501-8. [PMID: 9385303] CrossrefMedlineGoogle Scholar
    • 16. Tierney WMMcDonald CJHui SLMartin DKComputer predictions of abnormal test results. Effects on outpatient testing. JAMA1988;259:1194-8. [PMID: 3339821] CrossrefMedlineGoogle Scholar
    • 17. Harpole LHKhorasani RFiskio JKuperman GJBates DWAutomated evidence-based critiquing of orders for abdominal radiographs: impact on utilization and appropriateness. J Am Med Inform Assoc1997;4:511-21. [PMID: 9391938] CrossrefMedlineGoogle Scholar
    • 18. Blackmore CCMecklenburg RSKaplan GSEffectiveness of clinical decision support in controlling inappropriate imaging. J Am Coll Radiol2011;8:19-25. [PMID: 21211760] doi:10.1016/j.jacr.2010.07.009 CrossrefMedlineGoogle Scholar
    • 19. Carton MAuvert BGuerini HBoulard JCHeautot JFLandre MFet alAssessment of radiological referral practice and effect of computer-based guidelines on radiological requests in two emergency departments. Clin Radiol2002;57:123-8. [PMID: 11977945] CrossrefMedlineGoogle Scholar
    • 20. Chin HL, Wallace P. Embedding guidelines into direct physician order entry: simple methods, powerful results. Proceedings of the American Medical Informatics Association Annual Symposium, Washington, DC, 6–10 November 1999. Google Scholar
    • 21. Ip IKSchneider LSeltzer SSmith ADudley JMenard Aet alImpact of provider-led, technology-enabled radiology management program on imaging. Am J Med2013;126:687-92. [PMID: 23786668] doi:10.1016/j.amjmed.2012.11.034 CrossrefMedlineGoogle Scholar
    • 22. Raja ASIp IKPrevedello LMSodickson ADFarkas CZane RDet alEffect of computerized clinical decision support on the use and yield of CT pulmonary angiography in the emergency department. Radiology2012;262:468-74. [PMID: 22187633] doi:10.1148/radiol.11110951 CrossrefMedlineGoogle Scholar
    • 23. Rosenthal DIWeilburg JBSchultz TMiller JCNixon VDreyer KJet alRadiology order entry with decision support: initial clinical experience. J Am Coll Radiol2006;3:799-806. [PMID: 17412171] CrossrefMedlineGoogle Scholar
    • 24. Sistrom CLDang PAWeilburg JBDreyer KJRosenthal DIThrall JHEffect of computerized order entry with integrated decision support on the growth of outpatient procedure volumes: seven-year time series analysis. Radiology2009;251:147-55. [PMID: 19221058] doi:10.1148/radiol.2511081174 CrossrefMedlineGoogle Scholar
    • 25. Curry LReed MHElectronic decision support for diagnostic imaging in a primary care setting. J Am Med Inform Assoc2011;18:267-70. [PMID: 21486884] doi:10.1136/amiajnl-2011-000049 CrossrefMedlineGoogle Scholar
    • 26. Day FHoang LPOuk SNagda SSchriger DLThe impact of a guideline-driven computer charting system on the emergency care of patients with acute low back pain. Proc Annu Symp Comput Appl Med Care1995:576-80. [PMID: 8563351] MedlineGoogle Scholar
    • 27. Drescher FSChandrika SWeir IDWeintraub JTBerman LLee Ret alEffectiveness and acceptability of a computerized decision support system using modified Wells criteria for evaluation of suspected pulmonary embolism. Ann Emerg Med2011;57:613-21. [PMID: 21050624] doi:10.1016/j.annemergmed.2010.09.018 CrossrefMedlineGoogle Scholar
    • 28. Durand DJFeldman LSLewin JSBrotman DJProvider cost transparency alone has no impact on inpatient imaging utilization. J Am Coll Radiol2013;10:108-13. [PMID: 23273974] doi:10.1016/j.jacr.2012.06.020 CrossrefMedlineGoogle Scholar
    • 29. Flamm MFritsch GHysek MKlausner SEntacher KPanisch Set alQuality improvement in preoperative assessment by implementation of an electronic decision support tool. J Am Med Inform Assoc2013;20:e91-6. [PMID: 23599223] doi:10.1136/amiajnl-2012-001178 CrossrefMedlineGoogle Scholar
    • 30. Gupta AIp IKRaja ASAndruchow JESodickson AKhorasani REffect of clinical decision support on documented guideline adherence for head CT in emergency department patients with mild traumatic brain injury. J Am Med Inform Assoc2014;21:e347-51. [PMID: 24534635] doi:10.1136/amiajnl-2013-002536 CrossrefMedlineGoogle Scholar
    • 31. Soo Hoo GWWu CCVazirani SLi ZBarack BMDoes a clinical decision rule using D-dimer level improve the yield of pulmonary CT angiography? AJR Am J Roentgenol2011;196:1059-64. [PMID: 21512071] doi:10.2214/AJR.10.4200 CrossrefMedlineGoogle Scholar
    • 32. Ip IKGershanik EFSchneider LIRaja ASMar WSeltzer Set alImpact of IT-enabled intervention on MRI use for back pain. Am J Med2014;127:512-8. [PMID: 24513065] doi:10.1016/j.amjmed.2014.01.024 CrossrefMedlineGoogle Scholar
    • 33. Sanders DLMiller RAThe effects on clinician ordering patterns of a computerized decision support system for neuroradiology imaging studies. Proc AMIA Symp2001:583-7. [PMID: 11825254] MedlineGoogle Scholar
    • 34. Solberg LIWei FButler JCPalattao KJVinz CAMarshall MAEffects of electronic decision support on high-tech diagnostic imaging orders and patients. Am J Manag Care2010;16:102-6. [PMID: 20148614] MedlineGoogle Scholar
    • 35. Tierney WMMiller MEMcDonald CJThe effect on test ordering of informing physicians of the charges for outpatient diagnostic tests. N Engl J Med1990;322:1499-504. [PMID: 2186274] CrossrefMedlineGoogle Scholar
    • 36. Vartanians VMSistrom CLWeilburg JBRosenthal DIThrall JHIncreasing the appropriateness of outpatient imaging: effects of a barrier to ordering low-yield examinations. Radiology2010;255:842-9. [PMID: 20501721] doi:10.1148/radiol.10091228 CrossrefMedlineGoogle Scholar
    • 37. Raja ASGupta AIp IKMills AMKhorasani RThe use of decision support to measure documented adherence to a national imaging quality measure. Acad Radiol2014;21:378-83. [PMID: 24507424] doi:10.1016/j.ascra.2013.10.017 CrossrefMedlineGoogle Scholar
    • 38. Cohen JStatistical Power Analysis for the Behavioral Sciences. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Assoc; 1988. Google Scholar
    • 39. Strom BLSchinnar RAberra FBilker WHennessy SLeonard CEet alUnintended effects of a computerized physician order entry nearly hard-stop alert to prevent a drug interaction: a randomized controlled trial. Arch Intern Med2010;170:1578-83. [PMID: 20876410] doi:10.1001/archinternmed.2010.324 CrossrefMedlineGoogle Scholar
    • 40. Bowen SJohnson KReed MHZhang LCurry LThe effect of incorporating guidelines into a computerized order entry system for diagnostic imaging. J Am Coll Radiol2011;8:251-8. [PMID: 21458763] doi:10.1016/j.jacr.2010.11.020 CrossrefMedlineGoogle Scholar