Background:
Some people believe that likelihood ratios provide diagnostic information that is more useful than sensitivity and specificity estimates.
Objective:
To assess how physicians' estimates about probability of illness are affected by the presentation of a diagnostic test's value as an estimate of sensitivity and specificity versus a likelihood ratio or an inexact numerical graphic.
Design:
Random assignment of vignettes with different presentation formats of diagnostic test accuracy.
Setting:
Auditorium at a continuing medical education conference.
Participants:
183 physicians.
Intervention:
After estimating probabilities of 6 common illnesses described in patient vignettes, physicians reviewed pertinent test results presented in 1 of 3 formats.
Measurements:
Physicians' probability estimates of illness before and after receiving test information, and post-test probability estimates based on the Bayes theorem.
Results:
Absolute percentage point differences between the physicians' estimated and the Bayes-based post-test probabilities varied from −7 to 31, from −7 to 28, and from 1 to 29 for the sensitivity and specificity, likelihood ratio, and graphical groups, respectively. Mean differences of probability estimates between the sensitivity and specificity and the likelihood ratio groups were small for all vignettes (−2 to 3 percentage points; summary mean z value across the 6 vignettes, 0.04 [95% CI, −0.14 to 0.21]).
Limitations:
The small pool of participants (who were potentially selected) and the limited number of vignettes prevented a more detailed analysis of relationships between the interpreted strength of diagnostic evidence and estimations of illness probability.
Conclusions:
These findings suggest that presenting diagnostic test accuracy with likelihood ratios does not affect some physicians' estimates of illness probability compared with presenting diagnostic test results as sensitivity and specificity.
References
- 1.
Ledley RS ,Lusted LB . Reasoning foundations of medical diagnosis; symbolic logic, probability, and value theory aid our understanding of how physicians reason. Science. 1959;130:9-21. [PMID:13668531 ] CrossrefMedlineGoogle Scholar - 2.
Reid MC ,Lane DA ,Feinstein AR . Academic calculations versus clinical judgments: practicing physicians' use of quantitative measures of test accuracy. Am J Med. 1998;104:374-80. [PMID:9576412 ] CrossrefMedlineGoogle Scholar - 3.
Steurer J ,Fischer JE ,Bachmann LM ,Koller M ,ter Riet G . Communicating accuracy of tests to general practitioners: a controlled study. BMJ. 2002;324:824-6. [PMID:11934776 ] CrossrefMedlineGoogle Scholar - 4.
Hoffrage U ,Lindsey S ,Hertwig R ,Gigerenzer G . Medicine. Communicating statistical information. Science. 2000;290:2261-2. [PMID:11188724 ] CrossrefMedlineGoogle Scholar - 5.
Jaeschke R ,Guyatt GH ,Sackett DL . Users' guides to the medical literature. III. How to use an article about a diagnostic test. B. What are the results and will they help me in caring for my patients? The Evidence-Based Medicine Working Group. JAMA. 1994;271:703-7. [PMID:8309035 ] CrossrefMedlineGoogle Scholar - 6.
Sox HC ,Blatt MA ,Higgins MC ,Marton KI . Medical Decision Making. Newton, MA: Butterworth-Heinemann; 1988:67-102. Google Scholar - 7.
Hunink M ,Glasziou P . Decision Making in Health and Medicine: Integrating Evidence and Values. New York: Cambridge Univ Pr; 2001:128-76. Google Scholar - 8.
Weinstein MC ,Fineberg HV . The use of diagnostic information to revise probabilities. In: Clinical Decision Analysis. Philadelphia: WB Saunders; 1980:75-130. Google Scholar - 9.
Ginsberg JS ,Wells PS ,Kearon C ,Anderson D ,Crowther M ,Weitz JI ,et al . Sensitivity and specificity of a rapid whole-blood assay for D-dimer in the diagnosis of pulmonary embolism. Ann Intern Med. 1998;129:1006-11. [PMID:9867754 ] LinkGoogle Scholar - 10.
Jaeschke R ,Guyatt G ,Lijmer JG . Diagnostic tests.. In: Guyatt G, Rennie D, eds. User's Guide to the Medical Literature. Chicago: American Med Assoc; 2002:121-40. Google Scholar - 11.
Schouten HJ . Combined evidence from multiple outcomes in a clinical trial. J Clin Epidemiol. 2000;53:1137-44. [PMID:11106887 ] CrossrefMedlineGoogle Scholar - 12.
Bachmann LM ,Steurer J ,ter Riet G . Simple presentation of test accuracy may lead to inflated disease probabilities [Letter]. BMJ. 2003;326:393. [PMID:12586679 ] CrossrefMedlineGoogle Scholar - 13.
Goehring C ,Perrier A ,Morabia A . Spectrum bias: a quantitative and graphical analysis of the variability of medical diagnostic test performance. Stat Med. 2004;23:125-35. [PMID:14695644 ] CrossrefMedlineGoogle Scholar - 14.
Puhan MA ,Steurer J ,Bachmann LM ,ter Riet G . Variability in diagnostic probability estimates [Letter]. Ann Intern Med. 2004;141:578-9. [PMID:15466782 ] LinkGoogle Scholar - 15.
Bachmann LM ,ter Riet G ,Clark TJ ,Gupta JK ,Khan KS . Probability analysis for diagnosis of endometrial hyperplasia and cancer in postmenopausal bleeding: an approach for a rational diagnostic workup. Acta Obstet Gynecol Scand. 2003;82:564-9. [PMID:12780428 ] CrossrefMedlineGoogle Scholar - 16.
Miettinen OS . The modern scientific physician: 3. Scientific diagnosis. CMAJ. 2001;165:781-2. [PMID:11584568 ] MedlineGoogle Scholar - 17.
Moons KG ,Biesheuvel CJ ,Grobbee DE . Test research versus diagnostic research. Clin Chem. 2004;50:473-6. [PMID:14981027 ] CrossrefMedlineGoogle Scholar - 18.
Moons KG ,van Es GA ,Deckers JW ,Habbema JD ,Grobbee DE . Limitations of sensitivity, specificity, likelihood ratio, and bayes' theorem in assessing diagnostic probabilities: a clinical example. Epidemiology. 1997;8:12-7. [PMID:9116087 ] CrossrefMedlineGoogle Scholar - 19.
Irwig L ,Bossuyt P ,Glasziou P ,Gatsonis C ,Lijmer J . Designing studies to ensure that estimates of test accuracy are transferable. BMJ. 2002;324:669-71. [PMID:11895830 ] CrossrefMedlineGoogle Scholar - 20.
Davie AP ,Francis CM ,Love MP ,Caruana L ,Starkey IR ,Shaw TR ,et al . Value of the electrocardiogram in identifying heart failure due to left ventricular systolic dysfunction. BMJ. 1996;312:222. [PMID:8563589 ] CrossrefMedlineGoogle Scholar - 21.
Innes G ,Christenson J ,Weaver WD ,Liu T ,Hoekstra J ,Every N ,et al . Diagnostic parameters of CK-MB and myoglobin related to chest pain duration. Canadian Journal of Emergency Medicine. 2002;4:. CrossrefGoogle Scholar - 22.
Badgett RG ,Tanaka DJ ,Hunt DK ,Jelley MJ ,Feinberg LE ,Steiner JF ,et al . Can moderate chronic obstructive pulmonary disease be diagnosed by historical and physical findings alone? Am J Med. 1993;94:188-96. [PMID:8430714 ] CrossrefMedlineGoogle Scholar - 23.
Salvarani C ,Silingardi M ,Ghirarduzzi A ,Lo Scocco G ,Macchioni P ,Bajocchi G ,et al . Is duplex ultrasonography useful for the diagnosis of giant-cell arteritis? Ann Intern Med. 2002;137:232-8. [PMID:12186513 ] LinkGoogle Scholar - 24.
Schultze D ,Thomas Y ,Wunderli W . Evaluation of an optical immunoassay for the rapid detection of influenza A and B viral antigens. Eur J Clin Microbiol Infect Dis. 2001;20:280-3. [PMID:11399021 ] CrossrefMedlineGoogle Scholar
Author, Article, and Disclosure Information
From the University of Zurich, Zurich, Switzerland.
Acknowledgments: The authors thank Dr. Otto Brändli for the opportunity to conduct this study during the congress of the Zurich Lung League in Davos, Switzerland.
Grant Support: By the Helmut Horten Foundation. Dr. Bachmann's work was supported by the Swiss National Science Foundation (grants 3233B0-103182 and 3200B0-103183).
Disclosures: None disclosed.
Corresponding Author: Milo A. Puhan, MD, Horten Centre, University Hospital of Zurich, Postfach Nord, 8091 Zurich, Switzerland; e-mail, milo.
Current Author Addresses: Drs. Puhan and Steurer: Horten Centre, University Hospital of Zurich, Postfach Nord, 8091 Zurich, Switzerland.
Dr. Bachmann: University of Berne, Department of Social and Preventive Medicine, Finkenhubelweg 11, 3012 Berne, Switzerland.
Dr. ter Riet: Academic Medical Center, Deptment of General Practice, Room J2-118, 1105 AZ Amsterdam, the Netherlands.
Author Contributions: Conception and design: M.A. Puhan, J. Steurer, G. ter Riet.
Analysis and interpretation of the data: M.A. Puhan, L.M. Bachmann, J. Steurer, G. ter Riet.
Drafting of the article: M.A. Puhan.
Critical revision of the article for important intellectual content: L.M. Bachmann, J. Steurer, G. ter Riet.
Final approval of the article: M.A. Puhan, L.M. Bachmann, J. Steurer, G. ter Riet.
Provision of study materials or patients: M.A. Puhan.
Statistical expertise: M.A. Puhan, G. ter Riet.
Collection and assembly of data: M.A. Puhan.
Submit a Comment
Contributors must reveal any conflict of interest. Comments are moderated. Please see our information for authorsregarding comments on an Annals publication.
*All comments submitted after October 1, 2021 and selected for publication will be published online only.