Guidelines for Meta-analyses Evaluating Diagnostic Tests
Publication: Annals of Internal Medicine
Volume 120, Number 8
Submit a Comment
Contributors must reveal any conflict of interest. Comments are moderated. Please see our information for authorsregarding comments on an Annals publication.
Abstract
Objectives:
To introduce guidelines for the conduct, reporting, and critical appraisal of meta-analyses evaluating diagnostic tests and to apply these guidelines to recently published meta-analyses of diagnostic tests.
Data Sources:
Based on current concepts of how to assess diagnostic tests and conduct meta-analyses. They are applied to all meta-analyses evaluating diagnostic tests published in English-language journals from January 1990 through December 1991, identified through MEDLINE searching and by experts in the field.
Study Selection:
Meta-analyses were included if at least two of three independent readers regarded their main purpose as the evaluation of diagnostic tests against a concurrent reference standard.
Data Extraction:
By three independent readers on the extent to which meta-analyses fulfilled each guideline, with consensus defined as agreement by at least two readers.
Data Synthesis:
The guidelines are concerned with determining the objective of the meta-analysis, identifying the relevant literature and extracting the data, estimating diagnostic accuracy, and identifying the extent to which variability is explained by study design characteristics and characteristics of the patients and diagnostic test. In general, the guidelines were only partially fulfilled.
Conclusion:
Meta-analysis is potentially important in the assessment of diagnostic tests. Those reading meta-analyses evaluating diagnostic tests should critically appraise them; those doing meta-analyses should apply recently developed methods. The conduct and reporting of primary studies on which meta-analyses are based require improvement.
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
Panzer RJ, Black ER, Griner PF, eds. Diagnostic Strategies for Common Medical Problems. Philadelphia: American College of Physicians; 1991.
2.
Guyatt GH, Tugwell PX, Feeny DH, Haynes RB, Drummond M. A framework for clinical evaluation of diagnostic technologies. Can Med Assoc J. 1986; 134:587-94.
3.
L'Abbe KA, Detsky AS, O'Rourke K. Meta-analysis in clinical research. Ann Intern Med. 1987; 107:224-33.
4.
Jenicek M. Meta-analysis in medicine. Where we are and where we want to go. J Clin Epidemiol. 1989; 42:35-44.
5.
Fleiss JL, Gross AJ. Meta-analysis in epidemiology, with special reference to studies of the association between exposure to environmental tobacco smoke and lung cancer: a critique. J Clin Epidemiol. 1991; 44:127-39.
6.
Greenland S. Quantitative methods in the review of epidemiologic literature. Epidemiol Rev. 1987; 9:1-30.
7.
Oxman AD, Guyatt GH. Validation of an index of the quality of review articles. J Clin Epidemiol. 1991; 44:1271-8.
8.
Sacks HS, Berrier J, Reitman D, Ancona-Berk VA, Chalmers TC. Meta-analyses of randomized, controlled trials. N Engl J Med. 1987; 316:450-5.
9.
Berman DS, Kiat H, Van Train KF, Friedman J, Garcia EV, Maddahi J. Comparison of SPECT using technetium-99m agents and thallium-201 and PET for the assessment of myocardial perfusion and viability. Am J Cardiol. 1990; 66:72E-79E.
10.
Dales RE, Stark RM, Raman S. Computed tomography to stage lung cancer. Approaching a controversy using meta-analysis. Am Rev Respir Dis. 1990; 141:1096-101.
11.
Gianrossi R, Detrano R, Columbo A, Froehlicher V. Cardiac fluoroscopy for the diagnosis of coronary artery disease: a meta analytic review. Am Heart J. 1990; 120:1179-88.
12.
Goris ML, Basso LV, Keeling C. Parathyroid imaging. J Nucl Med. 1991; 32:887-9.
13.
Hoffman RM, Kent DL, Deyo RA. Diagnostic accuracy and clinical utility of thermography for lumbar radiculopathy. A meta-analysis. Spine. 1991; 16:623-8.
14.
Hurlbut TA 3d, Littenberg B. The diagnostic accuracy of rapid dipstick tests to predict urinary tract infection. Am J Clin Pathol. 1991; 96:582-8.
15.
Kardaun JW, Kardaun OJ. Comparative diagnostic performance of three radiological procedures for the detection of lumbar disk herniation. Methods Inf Med. 1990; 29:12-22.
16.
Mezger J, Lamerz R, Permanetter W. Diagnostic significance of carcinoembryonic antigen in the differential diagnosis of malignant mesothelioma. J Thorac Cardiovasc Surg. 1990; 100:860-6.
17.
Phillips KA. The use of meta-analysis in technology assessment: a meta-analysis of the enzyme immunosorbent assay human immunodeficiency virus antibody test. J Clin Epidemiol. 1991; 44:925-31.
18.
Pinson AG, Becker DM, Philbrick JT, Parekh JS. Technetium-99m-RBC venography in the diagnosis of deep venous thrombosis of the lower extremity: a systematic review of the literature. J Nucl Med. 1991; 32:2324-8.
19.
Reed JF 3d. Meta-analysis of the reliability of noninvasive carotid studies. Biomed Instrum Technol. 1991; 25:465-71.
20.
Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical epidemiology. A Basic Science for Clinical Medicine. 2d ed. Boston: Little Brown; 1991.
21.
Knottnerus JA, Leffers P. The influence of referral patterns on the characteristics of diagnostic tests. J Clin Epidemiol. 1992; 45:1143-54.
22.
Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research. Lancet. 1991; 337:867-72.
23.
Clermont RJ, Chalmers TC. The transaminase tests in liver disease. Medicine (Baltimore). 1967; 46:197-207.
24.
Chalmers TC, Frank CS, Reitman D. Minimizing the three stages of publication bias. JAMA. 1990; 263:1392-5.
25.
Begg CB, Berlin JA. Publication bias and dissemination of clinical research. J Natl Cancer Inst. 1989; 81:107-15.
26.
Lau J, Antman EM, Jimenez-Silva J, Kupelnick B, Mosteller F, Chalmers TC. Cumulative meta-analysis of therapeutic trials for myocardial infarction. N Engl J Med. 1992; 327:248-54.
27.
Sox HC Jr, Blatt MA, Higgins MC, Marton KI. Medical Decision Making. Boston: Butterworths; 1988.
28.
Fletcher RH, Fletcher SW, Wagner EH. Clinical Epidemiology: The Essentials. 2d ed. Baltimore: Williams & Wilkins; 1988.
29.
Hanley JA. Receiver operating characteristic (ROC) methodology: the state of the art. Crit Rev Diagn Imaging. 1989; 29:307-35.
30.
Centor RM, Schwartz JS. An evaluation of methods for estimating the area under the receiver operating characteristic (ROC) curve. Med Decis Making. 1985; 5:149-56.
31.
Moses LE, Shapiro D, Littenberg B. Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations. Stat Med. 1993; 12: 1293-316.
32.
Albert A. On the use and computation of likelihood ratios in clinical chemistry. Clin Chem. 1982; 28:1113-9.
33.
Irwig L. Modelling result-specific likelihood ratios (Letter). J. Clin Epidemiol. 1992; 45:1335-8.
34.
Tosteson AN, Begg CB. A general regression methodology for ROC curve estimation. Med Decis Making. 1988; 8:204-15.
35.
Hunink MG, Richardson DK, Doubilet PM, Begg CB. Testing for pulmonary maturity: ROC analysis involving covariates, verification bias, and combination testing. Med Decis Making. 1990; 10:201-11.
36.
McClish DK. Analyzing a portion of the ROC curve. Med Decis Making. 1989; 9:190-5.
37.
McClish DK. Determining a range of false-positive rates for which ROC curves differ. Med Decis Making. 1990; 10:283-7.
38.
Wieand S, Gail MH, James BR, James KL. A family of nonparametric statistics for comparing diagnostic markers with paired or unpaired data. Biometrika. 1989; 76:585-92.
39.
Guyatt GH, Oxman AD, Ali M, Willan A, McIlroy W, Patterson C. Laboratory diagnosis of iron-deficiency anemia: an overview. J Gen Intern Med. 1992; 7:145-53.
40.
Begg CB. Biases in the assessment of diagnostic tests. Stat Med. 1987; 6:411-23.
41.
Begg CB, Greenes RA. Assessment of diagnostic tests when disease verification is subject to selection bias. Biometrics. 1983; 39:207-15.
42.
Gray R, Begg CB, Greenes RA. Construction of receiver operating characteristic curves when disease verification is subject to selection bias. Med Decis Making. 1984; 4:151-64.
43.
Mulrow CD, Linn WD, Gaul MK, Pugh JA. Assessing quality of a diagnostic test evaluation. J Gen Intern Med. 1989; 4:288-95.
44.
Colditz GA, Miller JN, Mosteller F. How study design affects outcomes in comparisons of therapy. I: Medical. Stat Med. 1989; 8:441-54.
45.
Miller JN, Colditz GA, Mosteller F. How study design affects outcomes in comparisons of therapy. II: Surgical. Stat Med. 1989; 8: 455-66.
46.
Emerson JD, Burdick E, Hoaglin DC, Mosteller F, Chalmers TC. An empirical study of the possible relation of treatment differences to quality scores in controlled randomized clinical trials. Control Clin Trials. 1990; 11:339-52.
47.
Bernelot Moens HJ, Hirshberg AJ, Claessens AA. Data-source effects on the sensitivities and specificities of clinical features in the diagnosis of rheumatoid arthritis: the relevance of multiple sources of knowledge for a decision-support system. Med Decis Making. 1992; 12:250-8.
48.
Lachs MS, Nachamkin I, Edelstein PH, Goldman J, Feinstein AR, Schwartz JS. Spectrum bias in the evaluation of diagnostic tests: lessons from the rapid dipstick test for urinary tract infection. Ann Intern Med. 1992; 117:135-40.
49.
Ransohoff DF, Feinstein AR. Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. N Engl J Med. 1978; 299: 926-30.
50.
Hlatky MA, Pryor DB, Harrell FE Jr, Califf RM, Mark DB, Rosati RA. Factors affecting sensitivity and specificity of exercise electrocardiography. Am J Med. 1984; 77:64-71.
51.
Ng PC, Dear PR. The predictive value of a normal ultrasound scan in the preterm baby—a meta-analysis. Acta Paediatr Scand. 1990; 79:286-91.
52.
Banales JL, Pineda PR, Fitzgerald JM, Rubio H, Selman M, Salazar-Lezama M. Adenosine deaminase in the diagnosis of tuberculous pleural effusions. A report of 218 patients and review of the literature. Chest. 1991; 99:355-7.
53.
Rosen Y, Rosenblatt P, Saltzman E. Intraoperative pathologic diagnosis of thyroid neoplasms. Report on experience with 504 specimens. Cancer. 1990; 66:2001-6.
54.
Laird NM, Mosteller F. Some statistical methods for combining experimental results. Int J Technol Assess Health Care. 1990; 6:5-30.
55.
Berlin JA, Laird NM, Sacks HS, Chalmers TC. A comparison of statistical methods for combining event rates from clinical trials. Stat Med. 1989; 8:141-51.
56.
Detrano R, Janosi A, Lyons KP, Marcondes G, Abbassi N, Froelicher VF. Factors affecting sensitivity and specificity of a diagnostic test: the exercise thallium scintigram. Am J Med. 1988; 84:699-710.
57.
Oxman AD, Guyatt GH. A consumer's guide to subgroup analyses. Ann Intern Med. 1992; 116:78-84.
Information & Authors
Information
Published In
Annals of Internal Medicine
Volume 120 • Number 8 • 15 April 1994
Pages: 667 - 676
History
Published in issue: 15 April 1994
Published online: 15 August 2000
Keywords
Copyright
© 1994 American College of Physicians.
Authors
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. For an editable text file, please select Medlars format which will download as a .txt file. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
Download article citation data for:
Guidelines for Meta-analyses Evaluating Diagnostic Tests. Ann Intern Med.1994;120:667-676. doi:10.7326/0003-4819-120-8-199404150-00008
Guidelines for Meta-analyses Evaluating Diagnostic Tests. Ann Intern Med.1994;120:667-676. doi:10.7326/0003-4819-120-8-199404150-00008
View More
Login Options:
Purchase
You will be redirected to acponline.org to sign-in to Annals to complete your purchase.
Access to EPUBs and PDFs for FREE Annals content requires users to be registered and logged in. A subscription is not required. You can create a free account below or from the following link. You will be redirected to acponline.org to create an account that will provide access to Annals. If you are accessing the Free Annals content via your institution's access, registration is not required.
Create your Free Account
You will be redirected to acponline.org to create an account that will provide access to Annals.
Comments
0 Comments