The Quality of Written Feedback by Attendings of Internal Medicine Residents

Overview

Journal J Gen Intern Med

Publisher Springer

Specialty General Medicine

Date 2015 Feb 19

PMID 25691242

Citations 25

Authors

Jeffrey L Jackson

Cynthia Kay

Wilkins C Jackson

Michael Frank

Affiliations

Soon will be listed here.

Abstract

Background: Attending evaluations are commonly used to evaluate residents.

Objectives: Evaluate the quality of written feedback of internal medicine residents.

Design: Retrospective.

Participants: Internal medicine residents and faculty at the Medical College of Wisconsin from 2004 to 2012.

Main Measures: From monthly evaluations of residents by attendings, a randomly selected sample of 500 written comments by attendings were qualitatively coded and rated as high-, moderate-, or low-quality feedback by two independent coders with good inter-rater reliability (kappa: 0.94). Small group exercises with residents and attendings also coded the utterances as high, moderate, or low quality and developed criteria for this categorization. In-service examination scores were correlated with written feedback.

Key Results: There were 228 internal medicine residents who had 6,603 evaluations by 334 attendings. Among 500 randomly selected written comments, there were 2,056 unique utterances: 29% were coded as nonspecific statements, 20% were comments about resident personality, 16% about patient care, 14% interpersonal communication, 7% medical knowledge, 6% professionalism, and 4% each on practice-based learning and systems-based practice. Based on criteria developed by group exercises, the majority of written comments were rated as moderate quality (65%); 22% were rated as high quality and 13% as low quality. Attendings who provided high-quality feedback rated residents significantly lower in all six of the Accreditation Council for Graduate Medical Education (ACGME) competencies (p <0.0005 for all), and had a greater range of scores. Negative comments on medical knowledge were associated with lower in-service examination scores.

Conclusions: Most attending written evaluation was of moderate or low quality. Attendings who provided high-quality feedback appeared to be more discriminating, providing significantly lower ratings of residents in all six ACGME core competencies, and across a greater range. Attendings' negative written comments on medical knowledge correlated with lower in-service training scores.

Citing Articles

Narrative comments in internal medicine clerkship evaluations: room to grow.

Crumbley C, Szauter K, Karnath B, Sonstein L, Belalcazar L, Qureshi S Med Educ Online. 2025; 30(1):2471434.

PMID: 39998485 PMC: 11864032. DOI: 10.1080/10872981.2025.2471434.

Best Practices in Formative Feedback in Resident Evaluations: A Narrative Review.

VanderStoep A, Constant K, Kavic S J Surg Educ. 2025; 82(3):103417.

PMID: 39799709 PMC: 11786976. DOI: 10.1016/j.jsurg.2024.103417.

Medical Student Evaluations of Medical Faculty: Characteristics of Highly and Lower-Rated Teachers.

Jackson J, Gavinski K, Thompson M, Storch D, Murphy M, Nickoloff S J Gen Intern Med. 2025; .

PMID: 39777713 DOI: 10.1007/s11606-024-09085-y.

Translation and psychometric evaluation of composite feedback-seeking behavior questionnaire among Iranian medical residents.

Shavoun A, Mirzazadeh A, Kashani H, Raeeskarami S, Gandomkar R BMC Med Educ. 2024; 24(1):594.

PMID: 38811982 PMC: 11137997. DOI: 10.1186/s12909-024-05586-w.

Gender influences resident physicians' perception of an employee-to-employee recognition program: a mixed methods study.

Tischendorf J, Krecko L, Filipiak R, Osman F, Zelenski A BMC Med Educ. 2024; 24(1):109.

PMID: 38302913 PMC: 10835820. DOI: 10.1186/s12909-024-05083-0.

References

Kay C, Jackson J, Frank M . The relationship between internal medicine residency graduate performance on the ABIM certifying examination, yearly in-service training examinations, and the USMLE Step 1 examination. Acad Med. 2014; 90(1):100-4. DOI: 10.1097/ACM.0000000000000500. View

Salerno S, OMalley P, Pangaro L, Wheeler G, Moores L, Jackson J . Faculty development seminars based on the one-minute preceptor improve feedback in the ambulatory setting. J Gen Intern Med. 2002; 17(10):779-87. PMC: 1495113. DOI: 10.1046/j.1525-1497.2002.11233.x. View

GRAY J . Global rating scales in residency education. Acad Med. 1996; 71(1 Suppl):S55-63. DOI: 10.1097/00001888-199601000-00043. View

Durning S, Pangaro L, Lawrence L, Waechter D, McManigle J, Jackson J . The feasibility, reliability, and validity of a program director's (supervisor's) evaluation form for medical school graduates. Acad Med. 2005; 80(10):964-8. DOI: 10.1097/00001888-200510000-00018. View

Jensen A, Wright A, Kim S, Horvath K, Calhoun K . Educational feedback in the operating room: a gap between resident and faculty perceptions. Am J Surg. 2012; 204(2):248-55. DOI: 10.1016/j.amjsurg.2011.08.019. View

Haber R, Avins A . Do ratings on the American Board of Internal Medicine Resident Evaluation Form detect differences in clinical competence?. J Gen Intern Med. 1994; 9(3):140-5. DOI: 10.1007/BF02600028. View

Anderson P . Giving feedback on clinical skills: are we starving our young?. J Grad Med Educ. 2013; 4(2):154-8. PMC: 3399605. DOI: 10.4300/JGME-D-11-000295.1. View

Durning S, Cation L, Jackson J . The reliability and validity of the American Board of Internal Medicine Monthly Evaluation Form. Acad Med. 2003; 78(11):1175-82. DOI: 10.1097/00001888-200311000-00021. View

van de Ridder J, Stokking K, McGaghie W, Cate O . What is feedback in clinical education?. Med Educ. 2008; 42(2):189-97. DOI: 10.1111/j.1365-2923.2007.02973.x. View

10.

Davis D, Mazmanian P, Fordis M, Harrison R, Thorpe K, Perrier L . Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA. 2006; 296(9):1094-102. DOI: 10.1001/jama.296.9.1094. View

11.

Cantillon P, Sargeant J . Giving feedback in clinical settings. BMJ. 2008; 337:a1961. DOI: 10.1136/bmj.a1961. View

12.

Turnbull J, Gray J, MacFadyen J . Improving in-training evaluation programs. J Gen Intern Med. 1998; 13(5):317-23. PMC: 1496958. DOI: 10.1046/j.1525-1497.1998.00097.x. View

13.

Moss H, Derman P, Clement R . Medical student perspective: working toward specific and actionable clinical clerkship feedback. Med Teach. 2012; 34(8):665-7. DOI: 10.3109/0142159X.2012.687849. View

14.

Berbano E, Browning R, Pangaro L, Jackson J . The impact of the Stanford Faculty Development Program on ambulatory teaching behavior. J Gen Intern Med. 2006; 21(5):430-4. PMC: 1484783. DOI: 10.1111/j.1525-1497.2006.00422.x. View

15.

Delva D, Sargeant J, MacLeod T . Feedback: a perennial problem. Med Teach. 2011; 33(10):861-2. View

16.

Thompson W, Lipkin Jr M, Gilbert D, Guzzo R, Roberson L . Evaluating evaluation: assessment of the American Board of Internal Medicine Resident Evaluation Form. J Gen Intern Med. 1990; 5(3):214-7. DOI: 10.1007/BF02600537. View

17.

Archer J . State of the science in health professional education: effective feedback. Med Educ. 2010; 44(1):101-8. DOI: 10.1111/j.1365-2923.2009.03546.x. View

18.

Holmboe E, Fiebach N, Galaty L, Huot S . Effectiveness of a focused educational intervention on resident evaluations from faculty a randomized controlled trial. J Gen Intern Med. 2001; 16(7):427-34. PMC: 1495246. DOI: 10.1046/j.1525-1497.2001.016007427.x. View

19.

Gil D, Heins M, Jones P . Perceptions of medical school faculty members and students on clinical clerkship feedback. J Med Educ. 1984; 59(11 Pt 1):856-64. DOI: 10.1097/00001888-198411000-00003. View

20.

Frye A, Hollingsworth M, Wymer A, Hinds M . Dimensions of feedback in clinical teaching: a descriptive study. Acad Med. 1996; 71(1 Suppl):S79-81. DOI: 10.1097/00001888-199601000-00049. View