Systematic Review: The Relationship between Clinical Experience and Quality of Health CareFREE
Publication: Annals of Internal Medicine
Volume 142, Number 4
Submit a Comment
Contributors must reveal any conflict of interest. Comments are moderated. Please see our information for authorsregarding comments on an Annals publication.
Abstract
Background:
Physicians with more experience are generally believed to have accumulated knowledge and skills during years in practice and therefore to deliver high-quality care. However, evidence suggests that there is an inverse relationship between the number of years that a physician has been in practice and the quality of care that the physician provides.
Purpose:
To systematically review studies relating medical knowledge and health care quality to years in practice and physician age.
Data Sources:
English-language articles in MEDLINE from 1966 to June 2004 and reference lists of retrieved articles.
Study Selection:
Studies that provided empirical results about knowledge or a quality-of-care outcome and included years since graduation or physician age as explanatory variables.
Data Extraction:
We categorized studies on the basis of the nature of the association between years in practice or age and performance.
Data Synthesis:
Overall, 32 of the 62 (52%) evaluations reported decreasing performance with increasing years in practice for all outcomes assessed; 13 (21%) reported decreasing performance with increasing experience for some outcomes but no association for others; 2 (3%) reported that performance initially increased with increasing experience, peaked, and then decreased (concave relationship); 13 (21%) reported no association; 1 (2%) reported increasing performance with increasing years in practice for some outcomes but no association for others; and 1 (2%) reported increasing performance with increasing years in practice for all outcomes. Results did not change substantially when the analysis was restricted to studies that used the most objective outcome measures.
Limitations:
Because of the lack of reliable search terms for physician experience, reports that provided relevant data may have been missed.
Conclusions:
Physicians who have been in practice longer may be at risk for providing lower-quality care. Therefore, this subgroup of physicians may need quality improvement interventions.
Context
•
While “practice makes perfect” in some situations, physicians' knowledge and performance may decline with the passage of time.
Contribution
•
Of 62 published studies that measured physician knowledge or quality of care and described time since medical school graduation or age, more than half suggested that physician performance declined over time for all outcomes measured. Only 1 study showed improved performance for all outcomes measured.
Implications
•
This review should provoke careful study of the relationship of physician experience and the quality of care. It also raises concerns about the adequacy of continuing professional education in medicine.
–The Editors
Quality assurance and performance evaluation have become central issues in medicine. Care is suboptimal in many different medical conditions and clinical settings (1-6). Although delivering high-quality care is important to all clinicians, this issue may be particularly relevant to certain subgroups, such as physicians with less specialized training and those who see a smaller volume of patients (7-10).
Physicians who have been in practice for more years may also be less likely to deliver high-quality care (11, 12). Medical advances occur frequently, and the explicit knowledge that physicians possess may easily become out of date. Therefore, although it is generally assumed that the tacit knowledge and skills accumulated by physicians during years of practice lead to superior clinical abilities (13), it is also plausible that physicians with more experience may paradoxically be less likely to provide technically appropriate care.
Few existing studies have had the specific goal of evaluating the effects of experience on the quality of medical care (11). However, length of time in clinical practice has been included as part of a set of physician characteristics that might explain variations in quality or that may be confounders of the association between quality and other factors (13-18).
The purpose of this paper is to assess the robustness of the relationship between clinical experience and quality of care by systematically reviewing empirical studies. Although we define experience as the number of years a physician has been in practice, physician age and time in practice are highly correlated (11, 19, 20); therefore, for the purposes of this paper, we consider these variables to be interchangeable.
Methods
We searched MEDLINE (Ovid Technologies, 1966 to June 2004; English language) for terms describing physician experience (keywords: physician age, clinician age, physician experience, clinician experience), physician demographic characteristics (keywords: physician characteristics, clinician characteristics), practice variation (subject heading: physician's practice patterns), and performance in various domains (subject headings: clinical competence, health knowledge, attitudes and practice, outcomes assessment [health care]; keywords: knowledge, guideline adherence, appropriateness, outcomes). We retrieved potentially relevant articles and reviewed their reference lists to identify studies that our search strategy may have missed (Figure 1). We also searched our personal archives to identify additional studies. We included studies if they 1) were original reports providing empirical results; 2) measured knowledge, guideline adherence, mortality, or some other quality-of-care process or outcome; and 3) included years since graduation from medical school, years since certification, or physician age as a potential explanatory variable. We excluded studies if they described practice variation that is not known to affect quality of care (for example, assessed test-ordering behavior in clinical situations where optimal practice is unknown) or evaluated the performance of fewer than 20 physicians. For studies that examined several different end points, we included only those outcomes that are linked to knowledge or quality of care.
Figure 1. Selection process for studies included in analysis.
We used a standardized data extraction form to obtain data on study design and relevant results. We categorized studies into 4 groups on the basis of whether they evaluated knowledge (for example, knowledge of indications for blood transfusion), adherence to standards of care for diagnosis, screening, or prevention (for example, adherence to preventive care guidelines), adherence to standards of care for therapy (for example, appropriate prescribing), or health outcomes (for example, mortality). We classified the results of each study into 6 groups on the basis of the nature of the association between length of time in practice or age and performance: consistently negative, partially negative, no effect, mixed effect, partially positive, and consistently positive. “Consistently negative” studies were those for which all reported outcomes demonstrated a statistically significant decrease in performance with increasing years in practice or age. “Partially negative” studies showed decreasing performance with increasing experience for some outcomes and no association for others. We used similar definitions for “consistently positive” and “partially positive” studies. “Concave” studies found performance to initially improve with years in practice or age, then peak, and subsequently decrease.
We did not use formal meta-analytic techniques because the included studies used many different effect measures and some did not report parameter estimates.
Since studies based on self-reported practice may suffer from social desirability bias (21), we explored the effect of study quality on results by subcategorizing studies according to whether they measured outcomes with self-reports (that is, using surveys and interviews) or observed practice (that is, using chart audits or administrative data review). We also compared studies according to whether they performed multivariable modeling to adjust for patient and physician covariates. We used the Fisher exact test to compare the observed frequencies. We conducted all analyses with SAS, version 8.2 (SAS Institute, Inc., Cary, North Carolina).
Role of the Funding Source
The Harvard Pilgrim Health Care Foundation supported this study. It had no role in the design, conduct, or reporting of the study or in the decision to submit the manuscript for publication.
Results
Fifty-nine articles that reported data on 62 groups of relevant outcomes formed the basis of our analysis. Overall, 32 of the 62 evaluations (52%) demonstrated a negative association between increasing experience and performance (that is, performance decreased as experience increased) for all outcomes assessed; 13 (21%) reported a negative association for some outcomes but no association for other outcomes; 2 (3%) reported a concave relationship (that is, performance initially increased as experience increased, then peaked, and subsequently decreased); 13 (21%) reported no association; 1 (2%) reported a positive association (that is, performance increased as experience increased) for some outcomes but no association for other outcomes; and 1 (2%) reported a positive association for all outcomes assessed (Figure 2).
Figure 2. Distribution of study results relating physician age to clinical performance in various domains.
Knowledge
Twelve studies assessed the knowledge of practicing physicians, and all studies reported a negative association between knowledge and experience (Table 1). Studies by Ayanian and colleagues (7) and Salem-Schatz and colleagues (22) had large sample sizes, high response rates, and good sampling methods; used rigorous criteria to evaluate knowledge; and performed multivariate analysis.
Ayanian and colleagues (7) surveyed physicians to assess their beliefs about the survival benefit of therapies for acute myocardial infarction; the appropriate use of these therapies has been well-established in randomized, controlled trials. Specialists were more knowledgeable than generalists; however, after adjustment for this and other variables, physicians younger than 40 years of age were more likely to correctly believe in the value of therapies that improve survival (for example, thrombolytic agents, aspirin, and β-blockers). They were also significantly less likely to believe in the value of therapies that have been disproved (for example, prophylactic lidocaine) (P < 0.05).
Salem-Schatz and colleagues (22) interviewed surgeons and anesthesiologists to assess their knowledge of the risks associated with and indications for the transfusion of blood products. They found a highly significant negative association between knowledge and the number of years the physicians had been in practice (P < 0.001).
Adherence to Standards of Practice for Diagnosis, Screening, and Prevention
Twenty-four studies have assessed the appropriateness of physician use of diagnostic and screening tests, as well as preventive health care (Table 2). Overall, 15 (63%) of these studies demonstrated that physicians in practice for more years were less likely to adhere to standards of practice in this domain.
In the largest of these studies, Czaja and colleagues (33) surveyed participants to assess their adherence to cancer screening guidelines endorsed by the American Cancer Society and the National Cancer Institute. Physicians who had graduated more than 20 years before the survey were consistently less likely to adhere to recommended practices (odds ratio, 0.62 to 0.72; P < 0.05).
Using more objective measures of guideline adherence, Aubin and colleagues (17) assessed the practice of 21 physicians and found that after adjustment for patient and physician covariates, younger physicians were more likely to appropriately screen for hypertension (odds ratio, 1.11 [95% CI, 1.06 to 1.15).
Several other studies provide contrary results. Streja and Rabkin (47) assessed the use of recommended preventive care measures and found that after adjustment for other physician covariates (such as specialty, practice style, and number of diabetic patients in their practice), older physicians were more likely than younger physicians to test for proteinuria (odds ratio, 2.62 [CI, 1.61 to 4.37]) and to refer their patients for screening ophthalmology assessments (odds ratio, 1.48 [CI, 1.01 to 2.18]). However, older physicians were no more likely to order a high-density lipoprotein cholesterol level test. Their analysis did not adjust for any patient variables, such as the presence of macrovascular and renal disease. Rhee (12) evaluated the performance of 454 physicians treating patients in 15 different medical and surgical diagnostic categories and found a concave relationship between years in practice and adherence to standards of practice. Physicians in practice for 6 to 15 years provided the most appropriate care, whereas physicians with more or fewer years of experience provided less appropriate care.
Adherence to Standards of Appropriate Therapy
Table 3 presents the 19 studies that have assessed the influence of physician age and years in practice on adherence to standards of therapy. Of these studies, 14 (74%) found a partially or consistently negative association between physician age and adherence to standards of appropriate use of therapy.
A large and well-designed study by Beaulieu and colleagues (64) examined the prescribing behavior of physicians caring for patients with stable angina. After multivariate adjustment in a hierarchical model, older physicians were significantly less likely to prescribe aspirin (odds ratio for physicians in practice for > 20 years compared with those in practice < 10 years, 0.58). Age did not affect use of β-blockers or lipid-lowering agents.
Outcomes
Seven studies present data on the relationship between number of years in practice and actual health outcomes (Table 4). The strongest of these was conducted by Norcini and colleagues (14), who analyzed mortality for 39 007 hospitalized patients with acute myocardial infarction managed by 4546 cardiologists, internists, and family practitioners. After controlling for a patient's probability of death, hospital location and practice environment, physician specialty, board certification, and the volume of patients seen, these researchers observed a 0.5% (SE, 0.27%) increase in mortality for every year since the treating physician had graduated from medical school.
Hartz and colleagues (11) specifically assessed the association between experience and mortality rates for surgeons performing cardiac artery bypass grafting. After adjustment for both patient and physician variables, they found that physicians who have been in practice longer had higher operative mortality rates (P < 0.001). In contrast, Burns and Wholey's (69) large study of patients hospitalized for various conditions found no difference in mortality rates for physicians of different ages, but physicians in practice for more years had longer lengths of stay even after adjustment for patients' comorbid conditions and other physician factors.
Study Quality
To determine the influence of methodologic quality on study results, we stratified the 43 reports pertaining to adherence to standards of practice on the basis of whether outcomes were assessed by using self-reported data or more objective measures (that is, use of chart audits or administrative databases). Overall, 30 (70%) of these studies demonstrated a consistently or partially negative association between length of time in practice or physician age and adherence to standards of care. While the proportion of studies that found a consistently or partially negative association was slightly larger for self-reported studies than for those studies that used objective measures (71% vs. 62%), these differences were not statistically significant (P > 0.2).
Stratifying studies on the basis of whether they performed a multivariable analysis yielded similar results: 71% of the studies that adjusted for patient covariates found a consistently or partially negative association compared with 74% of studies that did not adjust for these factors, and 68% of the studies that adjusted for physician covariates found a consistently or partially negative association compared with 67% of studies that did not.
Discussion
Although based on heterogeneous studies, our systematic review of empirical studies evaluating the relationship between clinical experience and performance suggests that physicians who have been in practice for more years and older physicians possess less factual knowledge, are less likely to adhere to appropriate standards of care, and may also have poorer patient outcomes. These effects seem to persist in those studies that adjusted for other known predictors of quality, such as patient comorbidity and physician volume or specialization. The results are somewhat paradoxical since it is generally assumed that clinical experience enhances knowledge and skill and, therefore, leads to better patient care.
Our findings have many possible explanations. Perhaps most plausible is that physicians' “toolkits” are created during training and may not be updated regularly (70). Older physicians seem less likely to adopt newly proven therapies (71, 72) and may be less receptive to new standards of care (73). In addition, practice innovations that involve theoretical shifts, such as the use of less aggressive surgical therapy for early-stage breast cancer or protocols for reducing length of stay, may be harder to incorporate into the practice of physicians who have trained long ago than innovations that add a procedure or technique consistent with a physician's preexisting knowledge (74).
Our findings may also reflect the substantial environmental changes that have occurred in medicine over the past several decades; evidence-based medicine has been widely adopted, and quality assurance techniques, such as disease management and performance evaluation, are frequently used. More experienced physicians may have less familiarity with these strategies and may be less accepting of them. Given this, our results may represent a cohort effect; that is, when the current generation of more recently trained physicians has been in practice for a longer time, there may be smaller differences between their practice and those of their younger colleagues than our data would suggest.
Our study has several limitations. First, although we attempted to systematically review the literature on the association between number of years in practice or physician age and performance, our search strategy may have missed reports. This reflects the limited attention to this issue and the lack of consistent search terms to identify clinical experience. In addition, studies that were specifically designed to assess the relationship between experience and performance but found no association may have been less likely to be submitted or accepted for publication, and published studies that included number of years in practice or age among other physician characteristics may not have presented non–statistically significant results for these particular variables. Therefore, while we have no reason to suspect that we were more likely to identify studies showing decreasing performance with age, our findings are still potentially subject to an under-reporting bias.
Second, few reports included in this review were designed to specifically evaluate length of time in practice as their primary characteristic of interest. Consequently, our results may have been due to chance arising from multiple testing. However, we believe this is unlikely given the relative consistency of the results in several different domains, their “dose–response” relationship, and their overall plausibility. Moreover, restricting our analysis to the 32 studies that considered a broader set of physician characteristics, including number of years in practice or age as the focus of their investigation (that is, excluding those studies that considered physician age or number of years in practice only as confounders), does not change our results: 21 of the 32 (66%) studies reported a consistently or partially negative association between physician age and performance, whereas only 1 study demonstrated a partially positive association.
Third, disagreements may exist between clinical practice guidelines (33), and, thus, establishing appropriate norms may be difficult. As a result, assessing performance on the basis of guideline adherence may not reliably assess health care quality. Despite this, some studies included in our review used norms that had been adopted by several professional associations and that consequently reflect widely accepted standards of practice. Even for these studies, we observed age effects.
Finally, length of time in practice may be associated with other dimensions of quality that are not captured by the outcome measures that we evaluated. While we identified studies that assessed various conditions and aspects of performance, the relationship between age and performance may be different for other diseases and outcomes. For example, older physicians may be more effective at delivering the humanistic, rather than the technical, aspects of medical care. If this were true, one would expect that the patients of older physicians would report higher satisfaction, which has been demonstrated in some studies (75, 76) but not others (77, 78). Alternatively, physicians who have been in practice for a longer time may have better clinical judgment and may thus provide better care in complex cases or may be better diagnosticians. These outcomes have not been rigorously assessed.
Despite these limitations, our results are troubling. Although it is difficult to draw firm conclusions about the performance of older physicians in managing specific conditions or clinical scenarios, our results do suggest that older physicians may need quality improvement interventions that are generally applicable to all physicians. In addition, the requirements that are imposed on physicians to keep up to date and to demonstrate continuing competence should be further considered. Widely adopted continuing medical education techniques, such as the distribution of printed materials and lectures, are largely ineffective even in experimental conditions (79). Our results reinforce this. Moreover, many experienced physicians are exempt from the recertification requirements to which their more recently trained colleagues must adhere. For example, the American Board of Internal Medicine only requires physicians who received initial Board certification in or after 1990 to appear for periodic recertification examinations.
Our results also have implications for further research. The link between experience and performance should be further evaluated with studies that are designed a priori to specifically measure this association. These studies should use objective and widely accepted measures of performance; should be disease- or process-specific; and should be replicated for physicians of different specialties, demographic characteristics (such as sex), and different environment practices. The effect of age for physicians who routinely collaborate with other physicians, who frequently engage in evidence-based discussions, or whose practices are influenced by disease management, performance feedback, and computerized reminder systems may be different from that for physicians who practice in relative isolation or in more traditional settings.
An optimal study would follow a particular cohort of physicians over time. However, this is not practical and may be confounded by other secular trends in health care provision. Alternative designs would be similar to those of the highest quality included in our review and would adequately control for patient comorbidity, other physician factors, and the clustering of patients within physicians. These studies should also model the nature of the relationship between experience and performance since performance may improve during the initial phases of independent practice, plateau for some period of time, and then decrease. Finally, the ability of behavior change strategies to reduce the disparities in quality created by physician age should be evaluated in well-controlled clinical trials.
In summary, our results suggest that physicians with more experience may paradoxically be at risk for providing lower-quality care. The extent, magnitude, and nature of these results must be clarified, and added attention should be given to this subgroup of physicians who may need quality improvement interventions.
References
1.
McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348:2635-45. [PMID: 12826639]
2.
Schuster MA, McGlynn EA, Brook RH. How good is the quality of health care in the United States? Milbank Q. 1998;76:517-63, 509. [PMID: 9879302]
3.
Jha AK, Perlin JB, Kizer KW, Dudley RA. Effect of the transformation of the Veterans Affairs Health Care System on the quality of care. N Engl J Med. 2003;348:2218-27. [PMID: 12773650]
4.
Jencks SF, Cuerdon T, Burwen DR, Fleming B, Houck PM, Kussmaul AE, et al. Quality of medical care delivered to Medicare beneficiaries: A profile at state and national levels. JAMA. 2000;284:1670-6. [PMID: 11015797]
5.
Thompson JW, Ryan KW, Pinidiya SD, Bost JE. Quality of care for children in commercial and Medicaid managed care. JAMA. 2003;290:1486-93. [PMID: 13129989]
6.
Chassin MR, Galvin RW. The urgent need to improve health care quality. Institute of Medicine National Roundtable on Health Care Quality. JAMA. 1998;280:1000-5. [PMID: 9749483]
7.
Ayanian JZ, Hauptman PJ, Guadagnoli E, Antman EM, Pashos CL, McNeil BJ. Knowledge and practices of generalist and specialist physicians regarding drug therapy for acute myocardial infarction. N Engl J Med. 1994;331:1136-42. [PMID: 7935639]
8.
Halm EA, Lee C, Chassin MR. Is volume related to outcome in health care? A systematic review and methodologic critique of the literature. Ann Intern Med. 2002;137:511-20. [PMID: 12230353]
9.
Tu JV, Austin PC, Chan BT. Relationship between annual volume of patients treated by admitting physician and mortality after acute myocardial infarction. JAMA. 2001;285:3116-22. [PMID: 11427140]
10.
Birkmeyer JD, Stukel TA, Siewers AE, Goodney PP, Wennberg DE, Lucas FL. Surgeon volume and operative mortality in the United States. N Engl J Med. 2003;349:2117-27. [PMID: 14645640]
11.
Hartz AJ, Kuhn EM, Pulido J. Prestige of training programs and experience of bypass surgeons as factors in adjusted patient mortality rates. Med Care. 1999;37:93-103. [PMID: 10413397]
12.
Rhee SO. Factors determining the quality of physician performance in patient care. Med Care. 1976;14:733-50. [PMID: 972566]
13.
Stolley PD, Becker MH, Lasagna L, McEvilla JD, Sloane LM. The relationship between physician characteristics and prescribing appropriateness. Med Care. 1972;10:17-28. [PMID: 5007748]
14.
Norcini JJ, Kimball HR, Lipner RS. Certification and specialization: do they matter in the outcome of acute myocardial infarction? Acad Med. 2000;75:1193-8. [PMID: 11112721]
15.
Epstein SA, Gonzales JJ, Weinfurt K, Boekeloo B, Yuan N, Chase G. Are psychiatrists' characteristics related to how they care for depression in the medically ill? Results from a national case-vignette survey. Psychosomatics. 2001;42:482-9. [PMID: 11815683]
16.
Roetzheim RG, Van Durme DJ, Brownlee HJ Jr, Herold AH, Pamies RJ, Woodard L. Compliance with screening mammography. Survey of primary care physicians. J Fla Med Assoc. 1991;78:426-9. [PMID: 1919495]
17.
Aubin M, Vézina L, Fortin JP, Bernard PM. Effectiveness of a program to improve hypertension screening in primary care. CMAJ. 1994;150:509-15. [PMID: 8313262]
18.
Sanazaro PJ, Worth RM. Measuring clinical performance of individual internists in office and hospital practice. Med Care. 1985;23:1097-114. [PMID: 4033236]
19.
Anderson GM, Beers MH, Kerluke K. Auditing prescription practice using explicit criteria and computerized drug benefit claims data. J Eval Clin Pract. 1997;3:283-94. [PMID: 9456428]
20.
Epstein SA, Gonzales JJ, Onge JS, Carter-Campbell J, Weinfurt K, Leibole M, et al. Practice patterns in the diagnosis and treatment of anxiety and depression in the medically ill. A survey of psychiatrists. Psychosomatics. 1996;37:356-67. [PMID: 8701014]
21.
Adams AS, Soumerai SB, Lomas J, Ross-Degnan D. Evidence of self-report bias in assessing adherence to guidelines. Int J Qual Health Care. 1999;11:187-92. [PMID: 10435838]
22.
Salem-Schatz SR, Avorn J, Soumerai SB. Influence of clinical knowledge, organizational context, and practice style on transfusion decision making. Implications for practice change strategies. JAMA. 1990;264:476-83. [PMID: 2366281]
23.
Golden NH, Seigel WM, Fisher M, Schneider M, Quijano E, Suss A, et al. Emergency contraception: pediatricians' knowledge, attitudes, and opinions. Pediatrics. 2001;107:287-92. [PMID: 11158460]
24.
Meskauskas JA, Webster GD. The American Board of Internal Medicine recertification examination: process and results. Ann Intern Med. 1975;82:577-81. [PMID: 1119774]
25.
Norcini JJ, Lipner RS, Benson JA Jr, Webster GD. An analysis of the knowledge base of practicing internists as measured by the 1980 recertification examination. Ann Intern Med. 1985;102:385-9. [PMID: 3970474]
26.
Ramsey PG, Carline JD, Inui TS, Larson EB, LoGerfo JP, Norcini JJ, et al. Changes over time in the knowledge base of practicing internists. JAMA. 1991;266:1103-7. [PMID: 1865543]
27.
Cruft GE, Humphreys JW Jr, Hermann RE, Meskauskas JA. Recertification in surgery, 1980. Arch Surg. 1981;116:1093-6. [PMID: 7020647]
28.
Gemson DH, Colombotos J, Elinson J, Fordyce EJ, Hynes M, Stoneburner R. Acquired immunodeficiency syndrome prevention. Knowledge, attitudes, and practices of primary care physicians. Arch Intern Med. 1991;151:1102-8. [PMID: 2043012]
29.
Lewis CE, Freeman HE, Corey CR. AIDS-related competence of California's primary care physicians. Am J Public Health. 1987;77:795-9. [PMID: 3592031]
30.
Shapiro JA. General practitioners' attitudes towards AIDS and their perceived information needs. BMJ. 1989;298:1563-6. [PMID: 2503118]
31.
Evans CE, Haynes RB, Gilbert JR, Taylor DW, Sackett DL, Johnston M. Educational package on hypertension for primary care physicians. Can Med Assoc J. 1984;130:719-22. [PMID: 6697280]
32.
Schroen AT, Detterbeck FC, Crawford R, Rivera MP, Socinski MA. Beliefs among pulmonologists and thoracic surgeons in the therapeutic approach to non-small cell lung cancer. Chest. 2000;118:129-37. [PMID: 10893370]
33.
Czaja R, McFall SL, Warnecke RB, Ford L, Kaluzny AD. Preferences of community physicians for cancer screening guidelines. Ann Intern Med. 1994;120:602-8. [PMID: 8117000]
34.
Cook RL, Wiesenfeld HC, Ashton MR, Krohn MA, Zamborsky T, Scholle SH. Barriers to screening sexually active adolescent women for chlamydia: a survey of primary care physicians. J Adolesc Health. 2001;28:204-10. [PMID: 11226843]
35.
Richards C, Klabunde C, O'Malley M. Physicians' recommendations for colon cancer screening in women. Too much of a good thing? Am J Prev Med. 1998;15:246-9. [PMID: 9791644]
36.
Jacques CH, Jones RL, Houts P, Bauer LC, Dwyer KM, Lynch JC, et al. Reported practice behaviors for medical care of patients with diabetes mellitus by primary-care physicians in Pennsylvania. Diabetes Care. 1991;14:712-7. [PMID: 1954806]
37.
Kenny SJ, Smith PJ, Goldschmid MG, Newman JM, Herman WH. Survey of physician practice behaviors related to diabetes mellitus in the U.S. Physician adherence to consensus recommendations. Diabetes Care. 1993;16:1507-10. [PMID: 8299440]
38.
Marrero DG, Moore PS, Fineberg NS, Langefeld CD, Clark CM Jr. The treatment of patients with insulin-requiring diabetes mellitus by primary care physicians. J Community Health. 1991;16:259-67. [PMID: 1955577]
39.
Schwartz JS, Lewis CE, Clancy C, Kinosian MS, Radany MH, Koplan JP. Internists' practices in health promotion and disease prevention. A survey. Ann Intern Med. 1991;114:46-53. [PMID: 1983932]
40.
Sherman SE, Hershman WY. Exercise counseling: how do general internists do? J Gen Intern Med. 1993;8:243-8. [PMID: 8505682]
41.
Zerr DM, Del Beccaro MA, Cummings P. Predictors of physician compliance with a published guideline on management of febrile infants. Pediatr Infect Dis J. 1999;18:232-8. [PMID: 10093943]
42.
Heath K, Hogg RS, Singer J, Schechter MT, O'Shaughnessy MV, Montaner JS. Physician concurrence with primary care guidelines for persons with HIV disease. Int J STD AIDS. 1997;8:609-13. [PMID: 9310219]
43.
Skotniski EM, Woodward C, Hutchison B, Abelson J, Brown J, Norman G. HIV testing practices of primary care physicians: an Ontario survey. Can J Public Health. 1996;87:172-5. [PMID: 8771919]
44.
Ely JW, Goerdt CJ, Bergus GR, West CP, Dawson JD, Doebbeling BN. The effect of physician characteristics on compliance with adult preventive care guidelines. Fam Med. 1998;30:34-9. [PMID: 9460614]
45.
Rattay KT, Fulton JE, Galuska DA. Weight counseling patterns of U. S. Pediatricians. Obes Res. 2004;12:161-9. [PMID: 14742855]
46.
Ford LG, Hunter CP, Diehr P, Frelick RW, Yates J. Effects of patient management guidelines on physician practice patterns: the Community Hospital Oncology Program experience. J Clin Oncol. 1987;5:504-11. [PMID: 3819812]
47.
Streja DA, Rabkin SW. Factors associated with implementation of preventive care measures in patients with diabetes mellitus. Arch Intern Med. 1999;159:294-302. [PMID: 9989542]
48.
Anis NA, Lee RE, Ellerbeck EF, Nazir N, Greiner KA, Ahluwalia JS. Direct observation of physician counseling on dietary habits and exercise: patient, physician, and office correlates. Prev Med. 2004;38:198-202. [PMID: 14715212]
49.
Hulka BS, Kupper LL, Cassel JC. Physician management in primary care. Am J Public Health. 1976;66:1173-9. [PMID: 1008113]
50.
Saraiya M, Cookson ST, Tribble P, Silk B, Cass R, Poonja S, et al. Tuberculosis screening among foreign-born persons applying for permanent US residence. Am J Public Health. 2002;92:826-9. [PMID: 11988454]
51.
McFall SL, Warnecke RB, Kaluzny AD, Aitken M, Ford L. Physician and practice characteristics associated with judgments about breast cancer treatment. Med Care. 1994;32:106-17. [PMID: 8302103]
52.
Montaner JS, Hogg RS, Heath KV, Phillips P, Craib KJ, Schechter MT, et al. Heterogeneity of physician agreement with recommended therapeutic guidelines for the management of HIV-associated disease. Antivir Ther. 1996;1:157-66. [PMID: 11322249]
53.
Roy-Byrne P, Russo J, Dugdale DC, Lessler D, Cowley D, Katon W. Undertreatment of panic disorder in primary care: role of patient and physician characteristics. J Am Board Fam Pract. 2002;15:443-50. [PMID: 12463289]
54.
Hynes DM. The quality of breast cancer care in local communities: implications for health care reform. Med Care. 1994;32:328-40. [PMID: 8139298]
55.
Becker MH, Stolley PD, Lasagna L, McEvilla JD, Sloane LM. Characteristics and attitudes of physicians associated with the prescribing of chloramphenicol. HSMHA Health Rep. 1971;86:993-1003. [PMID: 5138286]
56.
Ray WA, Federspiel CF, Schaffner W. Prescribing of chloramphenicol in ambulatory practice. An epidemiologic study among Tennessee Medicaid recipients. Ann Intern Med. 1976;84:266-70. [PMID: 769624]
57.
Moride Y, Du Fort GG, Monette J, Ducruet T, Boivin JF, Champoux N, et al. Suboptimal duration of antidepressant treatments in the older ambulatory population of Quebec: association with selected physician characteristics. J Am Geriatr Soc. 2002;50:1365-71. [PMID: 12164992]
58.
Beers MH, Fingold SF, Ouslander JG, Reuben DB, Morgenstern H, Beck JC. Characteristics and quality of prescribing by doctors practicing in nursing homes. J Am Geriatr Soc. 1993;41:802-7. [PMID: 8340556]
59.
Dhalla IA, Anderson GM, Mamdani MM, Bronskill SE, Sykora K, Rochon PA. Inappropriate prescribing before and after nursing home admission. J Am Geriatr Soc. 2002;50:995-1000. [PMID: 12110057]
60.
Geller SE, Burns LR, Brailer DJ. The impact of nonclinical factors on practice variations: the case of hysterectomies. Health Serv Res. 1996;30:729-50. [PMID: 8591927]
61.
Payne BC, Lyons TF, Neuhaus E. Relationships of physician characteristics to performance quality and improvement. Health Serv Res. 1984;19:307-32. [PMID: 6746295]
62.
Fehrenbach SN, Budnitz DS, Gazmararian JA, Krumholz HM. Physician characteristics and the initiation of beta-adrenergic blocking agent therapy after acute myocardial infarction in a managed care population. Am J Manag Care. 2001;7:717-23. [PMID: 11464429]
63.
Willison DJ, Soumerai SB, Palmer RH. Association of physician and hospital volume with use of aspirin and reperfusion therapy in acute myocardial infarction. Med Care. 2000;38:1092-102. [PMID: 11078050]
64.
Beaulieu MD, Blais R, Jacques A, Battista RN, Lebeau R, Brophy J. Are patients suffering from stable angina receiving optimal medical treatment? QJM. 2001;94:301-8. [PMID: 11391028]
65.
Blanc PD, Trupin L, Earnest G, San Pedro M, Katz PP, Yelin EH, et al. Effects of physician-related factors on adult asthma care, health status, and quality of life. Am J Med. 2003;114:581-7. [PMID: 12753882]
66.
O'Neill L, Lanska DJ, Hartz A. Surgeon characteristics associated with mortality and morbidity following carotid endarterectomy. Neurology. 2000;55:773-81. [PMID: 10993995]
67.
Katon W, Rutter CM, Lin E, Simon G, Von Korff M, Bush T, et al. Are there detectable differences in quality of care or outcome of depression across primary care providers? Med Care. 2000;38:552-61. [PMID: 10843308]
68.
Davidson W, Molloy DW, Bédard M. Physician characteristics and prescribing for elderly people in New Brunswick: relation to patient outcomes. CMAJ. 1995;152:1227-34. [PMID: 7736373]
69.
Burns LR, Wholey DR. The effects of patient, hospital, and physician characteristics on length of stay and mortality. Med Care. 1991;29:251-71. [PMID: 1997754]
70.
Carthy P, Harvey I, Brawn R, Watkins C. A study of factors associated with cost and variation in prescribing among GPs. Fam Pract. 2000;17:36-41. [PMID: 10673486]
71.
Freiman MP. The rate of adoption of new procedures among physicians. The impact of specialty and practice characteristics. Med Care. 1985;23:939-45. [PMID: 4021579]
72.
Hlatky MA, Cotugno H, O'Connor C, Mark DB, Pryor DB, Califf RM. Adoption of thrombolytic therapy in the management of acute myocardial infarction. Am J Cardiol. 1988;61:510-4. [PMID: 3125736]
73.
Young MJ, Fried LS, Eisenberg J, Hershey J, Williams S. Do cardiologists have higher thresholds for recommending coronary arteriography than family physicians? Health Serv Res. 1987;22:623-35. [PMID: 3692862]
74.
Greer AL. The state of the art versus the state of the science. The diffusion of new medical technologies into practice. Int J Technol Assess Health Care. 1988;4:5-26. [PMID: 10287114]
75.
Hall JA, Irish JT, Roter DL, Ehrlich CM, Miller LH. Satisfaction, gender, and communication in medical visits. Med Care. 1994;32:1216-31. [PMID: 7967860]
76.
Blanchard CG, Labrecque MS, Ruckdeschel JC, Blanchard EB. Physician behaviors, patient perceptions, and patient characteristics as predictors of satisfaction of hospitalized adult cancer patients. Cancer. 1990;65:186-92. [PMID: 2293865]
77.
Jackson JL, Chamberlin J, Kroenke K. Predictors of patient satisfaction. Soc Sci Med. 2001;52:609-20. [PMID: 11206657]
78.
Schmittdiel J, Selby JV, Grumbach K, Quesenberry CP Jr. Choice of a personal physician and patient satisfaction in a health maintenance organization. JAMA. 1997;278:1596-9. [PMID: 9370506]
79.
Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, et al. Changing provider behavior: an overview of systematic reviews of interventions. Med Care. 2001;39:II2-45. [PMID: 11583120]
Comments
Sign In to Submit A CommentInformation & Authors
Information
Published In
Annals of Internal Medicine
Volume 142 • Number 4 • 15 February 2005
Pages: 260 - 273
History
Published online: 15 February 2005
Published in issue: 15 February 2005
Keywords
Copyright
© 2005 American College of Physicians.
Authors
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. For an editable text file, please select Medlars format which will download as a .txt file. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
Download article citation data for:
Systematic Review: The Relationship between Clinical Experience and Quality of Health Care. Ann Intern Med.2005;142:260-273. [Epub 15 February 2005]. doi:10.7326/0003-4819-142-4-200502150-00008
Systematic Review: The Relationship between Clinical Experience and Quality of Health Care. Ann Intern Med.2005;142:260-273. [Epub 15 February 2005]. doi:10.7326/0003-4819-142-4-200502150-00008
View More
Get Access
Login Options:
Purchase
You will be redirected to acponline.org to sign-in to Annals to complete your purchase.
Access to EPUBs and PDFs for FREE Annals content requires users to be registered and logged in. A subscription is not required. You can create a free account below or from the following link. You will be redirected to acponline.org to create an account that will provide access to Annals. If you are accessing the Free Annals content via your institution's access, registration is not required.
Create your Free Account
You will be redirected to acponline.org to create an account that will provide access to Annals.
No Title
Drs. Samuels and Ropper raise concerns that our search strategy was "not in the least systematic" and was "circuitous." As they describe in their letter, we identified studies from multiple sources and applied explicit inclusion and exclusion criteria. This process and its reporting are precisely what make our review systematic. We used methods recommended by the Cochrane Collaboration [1], which are similar to those frequently presented in the clinical literature.
Drs. Samuels and Ropper take issue with the outcomes we evaluated, asserting that, "(p)atients do not seek consultation for a more fastidious application of standards." We agree that many aspects of health care do not fit into clinical practice guidelines and that expert consultation by appropriately trained physicians likely leads to better care in these situations. However, an important aspect of quality is, in our opinions, application of the best available research evidence to the care of patients. This evidence is tailored to individual patients, to be sure, but applies to most patients most of the time.
Drs. Poses and Diaz indicate that our results could have been biased by including data from poorly designed original studies. However, we did exclude the smallest studies and those that simply reported practice variation; we also identified which studies adjusted for patient and physician covariates (see Tables 2-4). We reported that restricting our analysis to higher quality studies did not change our results. Unfortunately, accepted methods for incorporating quality scores into systematic reviews of heterogeneous observational studies are not yet available. Even in reviews of clinical trials different scales generate widely discrepant results[2].
Dr. Szabo notes that the studies in our review were published over a large time span and that restricting the analysis to those studies published within the past 5 years demonstrates that "fifty percent (8/15) actually showed a neutral or positive effect"¦." Unfortunately, this statement deserves clarification. Of the 18 studies that were published in 2000 or later, 6 showed a consistently negative association between length of time in practice and performance, 4 showed partially negative results and 8 showed no association. Dr. Szabo's suggestion that older physicians are busier and have less time to read clinical practice guidelines may be a partial explanation for our findings and merits study.
Drs. Norman and Ewa provide clarification of Norcini et al's results about the impact of years in practice on acute myocardial infarction mortality by pointing out that the results are relative, not absolute, risks. We agree that in absolute terms the adjusted effect sizes from this study are modest, although the magnitude of this effect is similar to the impact of volume on outcomes.
Finally, Dr. Loder suggests that one of us failed to disclose his role as an editor of UpToDate. However, as Dr. Loder points out, we did not mention this particular textbook or any other in our article. In any case, editors and authors of textbooks, unlike authors with ties to companies selling drugs and devices, have so far not been asked to declare this as a conflict. Our comments on the limitations of traditional continuing medical education are based on available research evidence, summarized in reference 79.
Delivering high quality care is the primary goal of all health care providers. We believe that care should be guided by the best available evidence. The thought-provoking letters illustrate the importance of re- energizing our efforts to establish effective quality-improvement mechanisms for physicians of all ages and specialties.
Conflict of Interest:
None declared
No Title
TO THE EDITOR:
The meta-analysis by Choudry, et al., (Feb 15) appears to have limitations beyond those listed. For example, what is the clinical experience of the authors of the 62 articles? Are the studies vicarious? If younger doctors do better following preventive protocols, one must ask, Which protocols? (The five major studies conflict). And in assessing "adherence to guidelines," one becomes confused looking at the more than 1000 sets now listed by the National Clearinghouse (www.guideline.gov). Don't the more experienced doctors order fewer tests because they don't need them (they still do histories and physicals)? Haven't they learned to jettison impractical or unproven studies? And older doctors should have poorer outcomes; they are treating more complex, older patients with many problems, while their younger confreres attract the young families whose main challenges are URIs.
Anyway, shouldn't I have been better tutored than my father, and my son better than I "“ since we trained in the half-century which has seen 90 percent of medical innovations? Rather than accept the authors' solemn forecast of partly cloudy for the progress of medicine, I would judge it at least partly sunny. Isn't this progress?
And, by the way, the bottom line was already enunciated by Dr. Osler: Get a quinquennial brain-dusting!
William C. Waters III, MD, MACP
Conflict of Interest:
None declared
No Title
COMMENTS:
I find the conclusions of that article to be personally offensive & journalistically lacking. If 52% of the studies showed one conclusion - one assumes that the other 48% did not. How can the authors (and editors) base a conclusion if the results are "essentially 50-50"? I have been a medical graduate since 1963 & use my own accumulated clinical data base in my practice on a daily basis. In addition I have been teaching at UCSF for 36 years & can attest that the current crop of physicians will never learn the diagnostic skills (e.g. cardiac physical diagnosis) since they run to the echocardiogram immediately. Why listen to the heart? Are they better clinicians or quicker to use technology. Do they measure physician competence by per cent of patients that receive beta blockers, ACE inhibitors, statins according to algorithms? If that is the case, why do we need M.D. graduates at all; surely, we can teach nurse practitioners to follow algorithms more slavishly than M.D.'s who want to think for themselves.
Arnold Goldschlager, M.D., FACP, FACC, FAHA
Conflict of Interest:
None declared
Value of Experience
This is a crucial issue and the responses you have thus far published seem unworthy of the importance. Medical practice depended traditionally on shared knowledge and experience yet experience is never examined for its quality. My experience is no more than the recollections ( highly selective) of the clinical encounters thus far in my practice. How accurately are they recalled. We do not know. Record keeping till now has been thoroughly inadequate. We tend to remember (and seek to replicate) our successes and I imagine suppress our failures. Only now with the entry of computers into clinical care can we begin to record and display what any individual's experience actually is and sharing this adds to a communal knowledge of how best to help our patient patients. The review while fragile highlights a key issue and needs better responses than have appeared thus far.
Conflict of Interest:
A Commitment to offering patients the best available information, advice and interventions in a humane manner.
Experience for diagnosis, not guidelines
I read with great interest the article by Choudry et al (1), and was impressed by the (expected) number of critiques that followed.
I would invite those who criticized the conclusions to make a self- relfection: indeed, experienced doctors are definitely more prone to recognize special subsets of diseases, and bring into light alternative diagnosis, even without the aid of laboratory examinations. However is medicine that hard? An old, very wise physician back in school always told me that medicine is 95-99% easy, and a very good doctor is only needed in the rest of the cases. Therefore, adherence to guidelines by younger doctors may lead their results into better general care, because most patients don't need someone to make a difficult diagnosis, but only to treat them following evidence-based medicine! On the other hand, those difficult to diagnose patients will probably benefit from an older, wiser and more experienced physician.
But, as we see everyday in practice, and Choudry demonstrated, more experienced physicians tend to deviate from guidelines and evidence-based medicine and will probably not treat the majority of patients following these.
Anyway, there will always be exceptions, as the older physician (specially those in academic positions) who keep their minds open to the advancements in medicine (and this will be the best doctor!!!), and the younger one who is careless about studying (who will be the worst!). So please, let's not make this a battlefield, as probably most experienced physicians who responded to this article belong to the very first class (those who keep up with articles and education), being the ones that we should seek for. But if you're not experienced AND still studying, I would go for the younger doctor.
1. Choudhry NK, RH Fletcher and SB Soumerai, Systematic Review: The Relationship between Clinical Experience and Quality of Health Care. Ann Intern Med 15 Feb 2005.142:260-73.
Conflict of Interest:
None declared
Disappointment with the College
To the Editor:
Shame on you! I am distressed that the College, certainly well aware of the American penchant for generalization, would permit an article relating to age to quality of medical practice to be published without challenge (1). Will you next publish a meta analysis of the quality of care provided by a blue vs brown eyed physicians, or between those east of the Mississippi vs those west, or women vs men, or Caucasians compared with African Americans? Clearly there are confounding issues relevant to any such Article. The cited publication not only fails to detail confounding variables, but also overlooks obvious exceptions. These could include the dramatic announcement of K.F. Meyer at age (gasp) 72 that he had completed the development and testing of a vaccine against Japanese Encephalitis, or the creations, with reference to any text of medicine, by former President of the College Jack D. Myers of the computer program for Internist I after he had retired as chairman of medicine at the University of Pittsburgh at age 65. Age may be a measure of distance from formal education, but it is not a priori a measure of skill in delivery of health care. Yet your publication may be used by every young aspiring ladder climber to attempt to dislodge older persons on higher rungs of medical achievement. It certainly provides solace for those who discriminate against those of us over 65 based on nothing then our biologic age.
It is told, in a dramatic story of civilizations that fail to honor their seniors, of one such society that took elderly persons, placed them in boxes and pushed them off a cliff. One such senior person simply suggested his younger executioner keep the box and push his intended victim off this cliff without it. When asked why, the elder is said to have replied, "So it will be available for you in a future year when you are old." Perhaps you should save the boxes.
I am dismayed that the College that I have so respected all these years should be so shallow as to have published such transparent data. The possible effects of the publication on the professional contributions of senior physicians and on their lives are to me incalculable.
Charles P. Craig, MD
1. Choudhry NK, RH Fletcher and SB Soumerai, Systematic Review: The Relationship between Clinical Experience and Quality of Health Care. Ann Intern Med 15 Feb 2005.142:260-73.
Conflict of Interest:
None declared
Quantification Sometimes Undermines Quality
Choudry and colleague's paper is technically flawed, and potentially destructive (1). It has something to say and no doubt the authors were not seeking to be inflammatory but their conclusions, taken in isolation, have obtained consideraable lay attention and therefore require a broader discussion and context. We are as disappointed by the opportunity lost in the accompanying editorial to clarify the value of experience in medicine.
Behind the mask of "evidence-based medicine," the authors review 59 articles, selected by their own peculia process from hundreds of potential papers relevant to the subject. They then analyze this sample according to four parameters: knowledge, adherence to standards of practice for diagnosis, screening and prevention; adherence to standards of appropriate therapy and outcomes. They conclude that "physicians who have been in practice longer may be at risk for providing lower-quality care. Therefore, this subgroup of physicians may need quality improvement interventions."
The systematic review is defective in several respects. First, it was not in the least systematic. Of 245 articles retrieved by an internet search engine, 167 were excluded because the authors believed that "reported practice variation was not related to the quality of care or for which the quality of care was not clearly established" (83 papers); or "the study did not report physician practice variation" (49 studies); or because "the study did not assess the relationship of physician practice variation (49 studies); or because "the study did not assess the relationship of physician characteristics tothe outcomes" (18 papers); or because "the study assessed the performance of trainees" (9 papers); or because "the paper was an editorial or did not collect original data" (5 papers). Three papers were excluded for "other reasons," which were not characterized. To the 78 papers that remained, the authors added nine articles from their "personal archives" and 35 more from a "reference list search." Of the 122 papers discovered by this circuitous route, 63 more wre excluded by the authors because "the reported practice variation or outcomes were not clearly related to quality of care" (32 papers), or "the articles did not report the relationship between length of time in practice or physician age and outcomes" (27 papers). Four additional papers were excluded for "other reasons." This extraordinary process yielded the 59 articles that were the subject of the "systematic review."
Note that the authors did not exclude articles that did not correct for the patients' age and the acuity or severity of the medical condition. Had they done so, they would have had virtually nothing to review. Older doctors by and large have older and sicker patients. Without taking this into account the interpretation of all such studies is falwed from the beginning. In addition, all four of the parameters whereby the 59 selected papers were analyzed speak to the core weakness of measuring quality through adherence to evidence based criteria. The firt of these is called "knowledge" by the authors, but what was studied was not knowledge, and certainly not its application, but information. If patients desired and sougth onoy information, they could consult the internet. In fact, this is precisely what many people now do, yet with allof this information, patients are, if anything less satisfied with their medical care than they were before all of this medical information was generally available.
The next two parameters are also superficial gauges of quality medicine: adherence to standards of practice for diagnosis, screening, prevention and therapy. Anyone can apply standards of practice for diagnosis, screening, prevention and therpay. Anyone can apply standards or algorithms written by committees of medical societies. Patients do not seek consultation for a more fastidious application of standards. They look to us for a wise and compassionate analysis of their problem; typically one in which the illness is not presented in written form as it was on the tests and questionnaires that were surveyed in the paper. To cite a specific example, many patients consult us asking whether they are candidates for the newest cutting edge treatment for multiple sclerosis. A substantial number of these do not even have multiple sclerosis. They have been told that they did, sually by physicians who have come to depend excessively ont eh results of tests (read MRI). Certainly, a patient would be in better hands with a doctor who did not know the newest therapy for MS, but did know how to recognize the disease. Knowledge of practice standards is not wisdom. If our strength were the ability to recite the latest practice standards and drug names, sick patients would be right to avoid us.
Finally, the authors conclude, that older doctors produce worse outcomes than younger ones. The authors need to be reminded that even in the era of evidence-based medicine, the mortality rate remains one per person. It just happens that younger doctors have on the average younger patients and are therefore not there as often when a patient dies. As younger doctors age, so will their panel of patients. Do the authors really believe that if we all had younger doctors, the mortality rate would fall?
The crux of the issue is the value of experience and the methods by which one obtains it. Certainy, no one argues for the repetition of mistakes masquerading as experience. If evidence-based medicine is the antidote for hubris, it may yet serve us well. At the moment it is largely a method by which to define best practices for large populations of patients in circumstances where outcome can be measured based on single and sometimes simplisitc items. Age brings two things: graciousness and the time to see fads in both treatment and medical reform come and go. It is the weak physician, young or old, who fails to observe carefully his and her own patients and assess the course of disease. The ability to hear and see large amounts of clincial data and choose the singular and salient features is what marks clincial maturity and creastes value. From this can come a degree of clarity that cuts through the true complexity of many medical problems. The lack of confidence in their own judgment and dependence on tests to solve problems has led to a dismal dependence on authoritative solutions.
This "systematic review" is potentially harmful to medicine. It glorifies information, algorithms and consensus criteria rather than experience and judgment. The analysis is simplistic, but its publication is perhaps not unexpected as part of an unfortunate course of events in medicine where, masquerading as evidence, quantification inadvertantly undermines quality.
1. Choudhry NK, Fletcher R, Soumerai S. Systematic Review: The Relationship between Clinical Experience and Quality of Health Care. Ann Intern Med. 2005;142:260-73.
Conflict of Interest:
None declared
Don't Denigrate Experience
I sat at my kitchen table eating a cold dinner after a twelve-hour day of seeing thirty patients, doing five preventive medicine physicals, two urgent preops, answering over twenty phone calls, sending two patient's to the ER from the office and read with grat interest Dr. Choudhry and colleagues article about Clinical Experience and Quality in the February 15, 2005 Annals (1). My twenty-year-old son sat next to me, who is as old as I have been in Internal Medicine Practice. I finished the article and said, "there will be an editorial calling for more frequent testing of physicians". Nonetheless, there stood the Editorial by Dr. Weinberger of the ACP and Drs. Duffy and Cassel representing the ABIM calling for "ongoing measurement of performance" (2). Excuse my cynicism, but how many Millions of dollars will be made by mandating additonal testing and recertification, despite the fact that there is, as of yet, no proof that it will improve the quality of care.
Dr. Choudhry and colleagues reviewed fifty-nine articles addressing various disease states. It is interesting that over two-thirds of the studies are over five years old. In fact, many are almost thirty years old. However, the explosion of evidence-based data has really been most evident in the last several years. Supporting this is the fact that although seventy percent of all of the studies suggested a negative effect of "experience" on performance, of those studies cited that were published within the past five years, over fifty percent (8/15) actually showed a neutral or positive effect of "experience" on performance. Things appear to be getting better.
Of greatest importance is the variable that is probably most correlated with "experience" and that is TIME. Unfortunately, TIME is inversely related to years in practice. It is likely that the older, "busier" physician has less time to deal with practice guidelines because there is just more to do. I do not believe that any of the studies looked at the physcian's time available as a variable. I would imagine that given the same amount of limited time, an older physician is likely to accomplish more and with more humanism than his younger counterpart. The older physician may not do a depression screen because he even knows why the patient is depressed.
Overall, I am disappointed that the Annals has chosen to disparage older physicians with flimsy data at a time when malpractice and payment pressures are driving older, highly experienced physicians out of medicine early. I was privileged to train under Dr. J. Willis Hurst and Dr. Bruce Logue at the end of their careers. They could do more feeling a patient's pulse than we do with a trans-esophageal echocardiogram. They were venerated. Why are we now denigrating "experience"? Please be careful.
1. Choudhry NK, Fletcher R, Soumerai S. Systematic Review: The Relationship between Clinical Experience and Quality of Health Care. Ann Intern Med. 2005;142:260-73.
2. Weinberger SE, Duffy FD, Cassel CK. "Practice Makes Perfect"...Or Does It? Ann Intern Med. 2005;142:302-303.
Conflict of Interest:
None declared
Young Doctor and Old Lawyer
Many years ago my older lawyer brother told me as I was finishing training that one should have a young doctor and old lawyer. It seems that Choudhry et al. have proved at least half of this contention (1).
Lee W. Roof, MD
1. Choudhry NK, Fletcher R, Soumerai S. Systematic Review: The Relationship between Clinical Experience and Quality of Health Care. Ann Intern Med. 2005;142:260-73.
A Double Standard
I was surprised that the authors of this article reported no potential financial conflicts of interest. (1) One of the authors and his wife serve as Co-Editors in Chief of the adult primary care section of UpToDate, described on its web site as "a comprehensive evidence-based clinical information resource available to physicians on the internet, CD- ROM, and Pocket PC... designed to get physicians the concise, practical answers they need when they need them the most - at the point of care." (2, 3)
In interviews following the article's publication this author identifies computerized databases as a principal solution to the quality problems identified by the review, although the article itself contains little evidence to support such a contention. While he does not specifically refer to the product UpToDate, phrases such as "up to date," "at that time of care" and others identical to or highly reminiscent of those used in official descriptions of UpToDate do appear in the interviews, and immediately brought that product to my mind. (4, 5) If UpToDate were a drug rather than an information database, and the author were employed by or associated with its manufacturer, he might be criticized for his failure to report that connection. The author's employer has a clear financial interest in the subject matter of a manuscript that identifies traditional, competing continuing medical education techniques as "largely ineffective" and prompts interviews in which the answer to the "problem" identified in the review conveniently includes products that sound a lot like UpToDate.
I am sure the author's intentions are laudable, and that the failure to report a connection with UpToDate was an oversight, but there seems to be a double standard, unrecognized at the highest levels of academia, about what poses a potential financial conflict of interest. Readers of this article and reporters who conducted interviews with the author based on it were denied the chance to consider whether this potential financial conflict influenced the focus of the review or the framing of recommended solutions.
References 1.Choudhry NK, Fletcher RH, Soumerai SB. Systematic Review: The Relationship between Clinical Experience and Quality of Health Care. Ann Intern Med 2005;142:260-273. 2.http://www.uptodate.com/physicians/adult_primary_care_meet_editors.asp Accessed March 8, 2005 3.http://www.uptodate.com/service/index.asp Accessed March 8, 2005 4.http://www.abc.net.au/am/content/2005/s1303813.htm Accessed March 8, 2005 5.http://www.ama-assn.org/amednews/2005/03/07/prsb0307.htm Accessed March 8, 2005
Conflict of Interest:
Dr. Loder is 46 and re-certified in Internal Medicine in 2000. She has been a paid and unpaid consultant and content developer for commercial and nonprofit continuing medical education projects.
Are Older Doctors Really Dumber?
The review article by Choudry et al about physicians' clinical experience versus the quality of their care(1) created headlines. Most media coverage suggested that older doctors are dumber than younger ones, e.g., see (2). However, such conclusions appear to go beyond the data provided in the article.
This review article did not use standards set a priori to exclude original articles of poor methodologic quality for review, or to rate the methodologic quality of any article it did review. Such standards are available.(3,4)
Thus, the review article included original articles whose criteria for physician performance were not evidence-based, such as tests of knowledge not related to the physicians' practices. Furthermore, the review included articles regardless of their study architecture, age, sample size, patient selection criteria, whether and how they controlled for patients' characteristics, and their effect size and its precision. Thus, the review's results could have been biased by including data from poorly designed or performed original studies, and original studies which are unlikely to generalize to modern physicians.
For example, consider the 13 original articles that used chart audit to assess adherence to standards of treatment. Only 6 showed a consistently negative effect of increasing age. Of these, one (Becker) was published 34 years ago, and included only 37 physicians. One (Moride) did not account for the severity of the patients' symptoms, and had a very small effect size (OR=1.12, CI 1.01, 1.24). One (Anderson) used a standard of care for inappropriate drug selection that might be debated. One (Dhalla) used that same standard, did not adjust for patients' clinical characteristics, and also had a very small effect size (OR=1.14). One (Payne) was published 21 years ago, and used practice standards defined by consensus, not evidence. One (Sanazaro) was published 20 years ago, included only 66 physicians, and again used practice standards defined by a panel, not evidence.
Thus, the results of the review may have been biased by the methodologic weaknesses of the original articles it included. Its key conclusion, that older doctors' performance is worse than that of younger ones, was not well supported by the evidence it presented.
We agree that physicians' professional values mandate serious, ongoing examination of our own performance. However, the principles of clinical epidemiology apply to such studies just as they apply to studies of patients. We should not rush to negative conclusions about physician performance without examining the strength of the relevant evidence.
References
1. Choudry NK, Fletcher RH, Soumerai SB. Systematic review: the relationship between clinical experience and quality of health care. Ann Intern Med 2005; 142: 260-273
2. Allen S. Greater risk seen with older doctors. Boston Globe, February 15, 2005.
3. C. David Naylor, Gordon H. Guyatt, for the Evidence-Based Medicine Working Group. Users' guides to the medical literature: XI - How to use an article about a clinical utilization review. JAMA 1996; 275:1435-1439.
4. C. David Naylor and Gordon H. Guyatt for the Evidence-Based Medicine Working Group. Users' guides to the medical literature: X - How to use an article reporting variations in the outcomes of health services research. JAMA. 1996; 275:554-8.
Conflict of Interest:
None declared
PHYSICIAN COMPETANCY IN ACADEMIC PRACTICE
We recently published a study examining factors that influence a clinician's decision regarding when to deliver a woman who has premature preterm rupture of the membranes.(1) Approximately 40% (n = 717) of U.S. maternal fetal medicine physicians (high risk pregnancy subspecialists) responded. Two expert review articles were published at the time of the questionnaire, which both agree with delivery at 34 weeks gestation, unless fetal pulmonary maturity could be documented.(2,3) In that case, delivery at 32 weeks was recommended. Respondent demographics including years of practice and type of practice were solicited.
We found that those in practice for the fewest number of years were most likely to recommend delivery in line with the expert reviews. As the years of practice increased, the portion of responses consistent with the review articles continuously decreased. Furthermore, for each cohort of years of practice, a higher portion of full time faculty respondents managed their patients in line with the recent review articles compared with those in other practice settings (full time faculty range: 36% to 23%; others: 29% to 10%; full time faculty compared with all other practice types combined: p < 0.01). The differences in clinical behavior were greatest for the cohort of physicians with the most years of practice.
If one considers that expert written review articles may be a surrogate for an emerging standard of care for some clinical issues, our results are in agreement with the findings of the recent manuscript by Choudhry et al regarding declining physician performance on some variables with increasing years of practice.(4) We make the additional observation that full-time faculty across the country performed more in keeping with published expert opinion across all experience levels.
Our observations are limited by factors such as the lack of a clearly established standard of care for this clinical issue and a limited response rate to the questionnaire. However, our findings do raise the question as to whether there are educational advantages to being a full- time faculty member worth exploring as the dialogue over improving ways of maintaining physician competency continues. These may include faculty requirements to teach and do research. Faculty may also be more likely to work in a larger and more diverse institutional medical community, resulting in exposure to expanded and frequent formal educational opportunities as well as increased opportunities for physician collaboration.
1. Healy A, Veille JC, Sciscione A, McNutt LA, Dexter S. The Timing of Elective Delivery in Preterm Premature Rupture of the Membranes: A Survey of Members of the Society of Maternal-Fetal Medicine. Am J Obstet Gynecol 2004;190: 1479-81.
2. Duff P. Evaluation and Management of Preterm Premature Rupture of Membranes. OBG Management 2003;15:57-61.
3. Mercer BM. Preterm Premature Rupture of the Membranes. Obstet Gynecol 2003;101:178-193.
4. Choudhry NK, Fletcher RH, Soumerai SB. Systematic Review: The Relationship between Clinical Experience and Quality of Health Care. Ann Intern Med. 2005; 142:260-273.
Conflict of Interest:
None declared
The relationship between clinical experience and some easily-measured aspects of care quality
I read with concern the article by Choudhry and colleagues regarding the relationship between clinical experience and quality of health care (1). Their conclusion that physicians, who have been in practice longer, provide lower-quality care seems to have substantially discounted the value of senior physicians.
First, this finding is probably not generalizable to senior physicians in the university settings. It is this group of physicians who largely form panels of expert and issue many of the practice guidelines used in the studies which their analysis based upon. It is also senior physicians who hand down medical education to us"”and to the authors. In clinical practice, when difficult cases are encountered, senior physicians often give advice to younger physicians. In fact, the performance of younger physicians may have been influenced by senior physicians.
Second, the analysis relies too heavily on the measurable indicators of quality of care. Other important aspects of care quality that cannot be easily measured are thus under-represented, simply because it is more difficult to conduct a study on them. For example, while many studies explore physicians' medical knowledge, none explores the physician knowledge of the patient; indeed very few studies explore patient satisfaction with care. Moreover, no study investigates the patient confidence in the physician, which can influence compliance with treatment. No study explores personal attributes, friendship, professionalism, and ease of communication with the physicians. In fact, the highest rate of malpractice lawsuit is observed among physicians age around 40s', suggesting that the authors' finding may have, indeed, undervalued the quality of care from the patients' perspective (2). As a patient, I continue seeing my older physicians who have followed me for a long period because of the continuity of care"”and because I trust them; there is no need for any study to show that this strategy can provide me with a better outcome.
Finally, not only the analysis provides information narrowly on the published, measurable aspect of care quality; it also fails to address the quantity of care. It is important not to disregard the amount of contribution that senior physicians have provided and continue to provide to our patients. Senior physicians serve as a backbone for most medical cares in rural areas of our country (3). Besides, disregarding some insignificant details"”often excessively captured in guideline auditing studies"”may allow more patients to be cared for, and more serious matters to be taken care of.
In my view, because the practice of medicine is not only a science but also an art, the measurement of its artistic quality would depend largely on the beholders"”the patients. I believe one cannot ignore the fact that a patient life is not a mathematic equation, or a computer program would have provided the best medical care.
References 1: Choudhry NK, Fletcher RH, Soumerai SB. Systematic review: the relationship between clinical experience and quality of health care. Ann Intern Med. 2005 Feb 15;142(4):260-73. 2: Taragin MI, Wilczek AP, Karns ME, Trout R, Carson JL. Physician demographics and the risk of medical malpractice. Am J Med. 1992 Nov;93(5):537-42. 3: Thompson MJ, Lynge DC, Larson EH, Tachawachira P, Hart LG. Characterizing the general surgery workforce in rural America. Arch Surg. 2005 Jan;140(1):74-9.
Conflict of Interest:
None declared
Too Bad To Be True?
To the Editor:
The article by Choudhry, Fletcher and Soumerai (1) showing a consistent negative relationship between experience and performance is controversial. While few will be surprised that recent graduates do better on knowledge tests, or are more likely to adhere to practice guidelines, the relationship to patient outcomes, particularly mortality, is much more difficult to dismiss. In particular, the one unequivocal study, by Norcini (3), observed "a 0.5% increase in mortality for every year [since graduation]". Taken at face value this is a terrifying statistic. If a recent graduate has a 10% in-hospital mortality, then someone 30 years in practice has a mortality of (10 + 30 x 0.5) or 25%, two and a half times as much.
In fact, the statistic is too bad to be true. The data from Norcini are actually RELATIVE risks, not absolute risks. In Table 2, (page 1197) the weight for years since graduation is 0.005%. By contrast, certification has a coefficient of - 0.15, and specialty training has a coefficient of - 0.25. On page 1195, Norcini states that certification was associated with a 15% reduction in mortality and specialization with a 25- 26% reduction. Thus, 20 years in practice equates to a relative increase in risk of about (20 x .005) = 0.10 and a absolute increase in mortality of about 1%, In a subsequent study by Norcini (3)using similar data, the effect of experience was smaller still (0.002%) and not statistically significant.
These results are consistent with the one other positive study of mortality cited by Choudhry (5) where relative risk of years since training was small and non-linear, resulting in a decrease to about 0.9 after 5-15 years of practice and climbing to 1.1 after 20 years of practice and 1.3 after 30 years.
The relation between years in practice and "hard" patient outcomes is weak. Thirty to 50 years of practice has no more impact on outcome than passing the certification exam or two years of subsepecialty training.
References
1) Choudhry NK, Fletcher RH, Soumerai SB. Systematic review: the relationship between clinical experience and quality of health care. Ann Intern Med 2005; 142: 260-273.
2) Norcini JJ, Kimball HR, Lipner RS.Certification and specialization: do they matter in the outcome of acute myocardial infarction? Acad Med. 2000; 75:1193-8.
3) Norcini JJ, Lipner RS, Kimball HR. Certifying examination performance and patient outcomes following acute myocardial infarction. Med Educ 2002; 36: 853-9.
4) Hartz AJ, Kuhn EM, Pulido J. Prestige of training programs and experience of bypass surgeons as factors in adjusted patient mortality rates. Med Care. 1999; 37: 93-103.
Conflict of Interest:
None declared
When I was younger, I knew more, but couldn't tell when to use it.
The article on "The Relationship between Clinical Experience and Quality of Health Care" missed some key points. How can younger physicians be trained by dumber older physicians? If the standards of care were chosen by older physicians, how would younger physicians measure up? How many first or second year physicians choose their classmates as their surgeons or physicians over some of their "older" teachers and more experienced colleagues.
I certainly thought I knew more when I finished my residency than I think I know now. However, like the teenager who knows everything, I could not always decide when best to use the information. Use of that knowledge comes with experience. Now that I am older I know that I don't know everything and limit my actions to what I have learned works best in my hands. Most of my older colleagues limit their practice to what they do well. They shouldn't be tested on what they are smart enough not to do. If I were to be tested on joystick games, I would fail, however ask my patients how I do and you would find I excel.
Logically our performance as physicians should improve at least for awhile before declining. Any report to the contrary doesn't pass the smell test and shouldn't be reported except in the National Enquirer type tabloids.
Conflict of Interest:
None declared
Do I Want a Fresh Graduate to take care of me ?
In reference to the article by Choudhry et al (1) I would like to know whether they would prefer a recent graduate, say in cardiac surgery or anesthesiology, take care of someone close to them. Brushing off senior surgeons and anesthesiologists because of a lack of consensus on parameters for transfusions - does not, in my mind, make a senior physician "behind the times". I find it interesting that residents and fellows come to us to be trained in the spciality of cardiac surgery and anesthesiology ,but, according to Choudhry, these same residents on graduation, are suddenly better than the physicians who trained them! How does one make such assumptions ? Is there for example, a difference in the knowledge base between physicians from academic vs non academic programs. Does it make a difference if a physician is specialized in a particular field and is involved with the teaching of medical students, residents and fellows ? Does experience count when it comes to a surgical speciality such as cardiac ? The reference to the article by Burns(2) about senior cardiac surgeons having a longer length of stay in their patients following cardiac surgery, was compiled in 1988 and it actually states( abstract) that surgery and graduation from medical school, not age, made a difference.
I do agree with the concept of continuing medical education, but to view all physicians with over 20 years of experience to be "over the hill " is a disservice to all physicians.
Choudhry may not think so - but please get me a surgeon and an anesthesiologist with 20 plus years of experience over a recent graduate when it comes time to fix my heart !
References 1) N.K. Choudhry et al : The relationship between clinical experience and quality of health care. Annals of Internal Medicine Feb. 05 vol.142Issue 4 . (260-273)
2) Burns LR The effects of Patient, Hospital, and Physician Characteristics on length of stay and Mortality: Medical Care 1991; 29 ( 251 71 )
Conflict of Interest:
None declared