S3: Evaluation and Feedback
3.1 Principles of Evaluating Learners and Giving Feedback
- Formative: Provide observations to help the learner assess strengths and learning needs to improve future performance.
- Summative: Assessing a learner’s performance using specific criteria to determine competency.
Goals of Evaluation
- Clear and articulate description of performance.
- More objective evaluation of core competencies.
- Consistent evaluation of core competencies.
RIME Approach to Evaluation
- Accurately gathers information.
- Basic skill to perform a history and physical examination.
- Distinguishes normal from abnormal.
- Reports information without attempting integration or synthesize.
- Demonstrates the integration and synthesis of information reported.
- Prioritizes clinical problems at a basic level.
- Develops a more patient-oriented assessment including a differential diagnosis.
- Actively participates in patient care.
- Reports, interprets, and applies information to the patient.
- Makes diagnostic or management suggestions.
- Uses textbooks and the primary literature to better understand the patient.
- Goes beyond the basics of self-directed learning.
- Critical appraisal skills.
- Participates in the education of others.
- Reporter: Passing but requires additional work to address deficiencies. “C” Student.
- Interpreter: Meeting Expectations. “B” Student.
- Manager/Educator: Exceeding Expectations. “A” Student.
- Feedback can be either:
- Positive: reinforcing what was done right
- Constructive: correcting mistakes or misconceptions
- Feedback goals:
- Ensuring the learner improves
- While keeping his/her self-respect intact
Elements of Feedback
- Clear expectations
- Appropriate location
- Timely and regular
- Descriptive and nonjudgmental
- Based on direct observations
- Focus on areas that can be changed
- Limit to what the learner can use
- The learner understands the feedback and knows the next step
Giving Positive Feedback
- Not general praise . . . . .
- “You did a good job with that patient.”
- “You are doing fine.”
- Identify a specific behavior or action that was done well
Giving Constructive Feedback
- Identify a specific behavior or action that was done wrong
- Tell the student how to avoid or correct the error in the future
Tips for Giving Feedback
- Solicit feedback from the learner
- Label the feedback
- Give the feedback
- Make suggestions for improvement
- Make sure the learner understands the feedback
Feedback: Helpful Hints
- Ask for the learner’s thoughts about his/her performance first
- Avoid vague, harsh or judgmental statements
- Feedback sandwich
- Tell what was done well
- Tell what was done wrong
- Discuss what to do next time Student “A”
- His case presentations are disorganized and incomplete.
- Student “A” has consistent trouble distinguishing pertinent information from extraneous data.
- He attempts to synthesize information in establishing a differential diagnosis, but his fund of knowledge may limit his/ her ability in synthesizing more complex issues.
- Student “B” takes initiative for his own education and made a sincere effort to improve his clinical skills.
- He accurately obtains a basic history and identifies the major findings.
- His physical exam is complete with some technical inadequacies.
- His case presentations are disorganized and incomplete.
Evaluations should be completed by the designated lead clinical faculty from each practice twice for each student; a mid-rotation evaluation and an end of rotation evaluation. Please review a draft copy of the evaluation rubric and evaluation form at the end of this section.
T. Andrew Albritton, MD, Associate Dean for Curriculum Medical College of Georgia
3.2 Feedback: A key Feature of Medical Training
Feedback: A Key Feature of Medical Training
Beverly P. Wood, MD, MSc
As we instruct trainees in the health sciences, we serve as teachers, tutors, and preceptors with knowledge and experience to be transmitted to and used by learners. Many diverse methods of instruction are used throughout a training experience, but the most available and influential method of learning probably is feedback. Feedback is immediate information. It is inexpensive, requires little preparation, and is highly desired by the learner; yet, in medical instruction, it is a scarce occurrence, and our trainees miss an experience that is highly formative in their development.
In the context of feedback, consider two issues: To practice medicine well, one must necessarily be an effective life-long learner, and learning is under the learner's strategic control of memory, experience, and attention. The sparseness and sterility of feedback occur because of two possible situations: the infrequency with which trainees are closely observed performing a skill or participate in deep discussion with a mentor, and the unease felt by faculty preceptors in delivering feedback. Inadequacy of a feedback mechanism produces a deficiency in training, which should be reversed to ensure high-quality graduates of training programs. There are preparatory steps to ensure that meaningful feedback occurs, and there are general guidelines to follow in giving feedback. Both will be discussed in this article....
3.3 Feedback in Clinical Medical Education
Feedback in Clinical Medical Education
Jack Ende, MD
From the Evans Memorial Departments of Clinical Research and Medicine, Section of General Internal Medicine, Boston University Medical Center
In the setting of clinical medical education, feedback refers to information describing students' or house officers' performance in a given activity. It is a key step in the acquisition of clinical skills, yet feedback is often omitted or handled improperly in a clinical training. This can result in important untoward consequences, some of which may extend beyond the training period. Once the nature of the feedback process is appreciated, however, especially the distinction between feedback and evaluation and the importance of focusing on the trainees themselves, the educational benefit of feedback can be realized. This article presents guidelines for offering feedback that have been set forth in the literature of business administration, psychology, and education, adapted here for use by teachers and students of clinical medicine....
3.4 How to Use the M3/M4 Clerkship Student Assessment Form
Using the M3/M4 Clerkship Student Assessment Form
- REVIEW: Before the student starts working with you, review both the clerkship objectives and the final M3/M4 evaluation form. By knowing ahead of time how you need to evaluate the student, you can take better notes about the student’s performance along the way.
- TAKE NOTES: At the end of the day (or week) make a few written notes of your impressions about a student. Write down some specific behaviors (e.g. wrote an excellent detailed note, or interacted well with an anxious toddler, or looked up an article and presented it to the team). If the student gives a short talk on a topic, write it down! It may difficult to remember the specific details (or even the topic) by the end of the clerkship.
- MID-ROTATION FEEDBACK: Midway through the rotation, take time to provide the student with feedback by utilizing the 6 competencies listed in the final evaluation form. Let the student know where they fall in their performance at that time and how they can make improvements.
- COMPLETE THE EVAL: Complete the evaluation on or near the last day of the rotation. Some preceptors like to print the form and review it with the student on the last day. The sooner the form is completed, the more accurate it will be.
- COMMENTS: Please include specific comments. These comments are used to construct the Dean’s Letter that goes with each student in their application for residency. A comment such as “frequently searched the literature and shared their findings with the team” may help show a student’s drive and motivation. A comment like “needs multiple reminders to pick up new patients,” could help justify the lower rating you chose. Your comments really do matter and help me support the grade a student receives. Generic comments like “Good student” are not as helpful.
How is the final grade determined?
The final grade is determined by the following components:
- Community Preceptor Evaluations – 50-70% (depending on the rotation). Some students at community sites have only one evaluator while others have several that are averaged together for the entire rotation.
- Shelf examination = 20-30% (depending on the rotation).
- There are cutoff scores the student must make to qualify for certain grades
- Must score > 5th percentile nationally for C; > 30th for B; > 70th to make an A
- Departmental Exam – 5-15% (depending on the rotation).
- Other Portfolio Items – 5-15% (depending on the rotation).
How is the clinical score determined from the M3/M4 Clerkship Student Assessment Form?
- The score is automatically calculated based on which radio button you select. Not all competencies are weighted equally.
- The descriptors on the Left hand column are for behaviors of Meets Expectations, the radio buttons in general follow the letter grades below, however we strongly recommend that you focus on behaviors NOT grades, as your “A” student may be another’s “B” student, please focus on behaviors:
- A – Exemplary and High Exceeding Expectations. It is quite unusual for a student to receive most marks in the Exemplary category. Remember, it is supposed to be the top 5% of all students for that category.
- B – Low Exceeds Expectations and High Meets Expectations
- C – Low Meet Expectations
- D – Marginal
- F – Does Not Meet Expectaions
Frequently Asked Questions
How do I get the M3/M4 Clerkship Student Assessment Form?
- The form will be sent to you via One45, our web-based evaluation system. You will receive an email from “Rachel McDaniel (via One45)” with “new evaluation” in the subject line. It is very important that we have your correct email address.
What if I did not receive an email via One45?
- Check your junk mail to make sure it was not filtered out
- Make sure we have your correct email address
What if I cannot find the email from One45 that was sent to me?
- Contact Rachel McDaniel (email@example.com) in the Clerkship Office, she will resend the link.
What if I receive an evaluation on a student I did not work with?
- Rachel McDaniel (firstname.lastname@example.org) so she can redirect the evaluation to the correct person.
- Do NOT complete the evaluation. We do not know until we are calculating final grades that the form was sent to the wrong evaluator.
What should I do if I submit an evaluation, but then remember something else I’d like to include?
- Contact Rachel McDaniel (email@example.com) so she can re-open the evaluation for you.
What if a student contacts me with concerns about their grade?
- Contact Site Clerkship Director if you are directly approached by a student with a concern about their grade. You should not be contacted directly by students concerning grades or evaluations.
3.5 How Do I Complete an Evaluation of a Phase III Medical Student on One45?
Evaluations must be completed using the One45 system. One45 is AU’s web based medical education evaluation software. It is used to manage evaluations and assessments, patient/procedure logs, ACGME/LCME curriculum mapping, and etc.
To complete an evaluation on One45 please follow the step-by-step instructions below.
When you have an item to complete in the one45 system, you will receive email notification.
- Follow the auto login link in the email message or enter your username and password through the log in page. First time users will have their computer generated password displayed in the email message.
First time users will be asked to change their password when they first log on. It is important to write this password down. If this password is forgotten, it must be reset by clicking on forgot my password on the login page.
After logging on, you will be redirected to a To Do's page. To complete an item, click on the link and follow the instructions.
Once an evaluation has been completed, you can choose to submit or save the form in your To Dos.
When you have completed your "To Do" items, log out of the system.