Blogs My bet is if we place fifty educators in a room with a rubric and piece of student writing, we’re likely to come back with numerous different scores on that piece of writing, and we’re likely to notice a variety of interpretations of the rubric’s criteria as well. However, if we have a shared understanding and we calibrate with one another on the use and interpretation of the rubric, then the rubric becomes truly effective in informing our instruction and improving student learning.
Improved student learning is exactly what drives those of us who serve on the EQuIP Review Panel. (Watch our video series on how teachers evaluate lessons with Achieve’s EQuIP rubrics.) When we first began working together, we calibrated on multiple lessons as a large group, before beginning to review Common Core aligned lessons and units individually and in small groups. Since you, too, strive for improved student learning in your classrooms, we thought it might be useful to offer a few tips and suggestions for you to consider when you use the EQuIP tools in your own professional learning communities.
1. Assign a facilitator if you don’t have one already. As Review Panel members, we take turns serving as facilitator or “lead reviewer.” The point here is that one person takes the responsibility each time you meet to:
- Collect all the feedback
- Ensure all group members feel comfortable contributing
- Keep the group moving and on track during tough conversations
ON YOUR OWN (an important part of the process is individual examination and evaluation of the lessons and units before talking with colleagues):
2. Closely read and examine the same exact lesson materials and associated texts, graphics, tasks, rubrics, etc. This might be a lesson you have created or an open source lesson you wish to evaluate for use in your own classroom.
3. Closely read and examine the EQuIP rubric dimensions and criteria. The best way to familiarize yourself with the rubrics is to use them, but this close read before you get started with a review lets you see what you will look for when you evaluate the lesson.
4. Re-read the lesson materials and make initial evidence based evaluations for all the criteria in the four dimensions. As you will notice in the EQuIP Review Panel videos, you might check all the boxes in a particular dimension even if you feel there’s still room for improvement on a specific criterion.
TOWARD THE COLLECTIVE
5. Before beginning the coming to consensus part of a review, begin moving in that direction by having each member of the team state aloud, one at a time, a score for each of the rubric dimensions — without explanation, elaboration, or justification (the facilitator makes note of all the scores). It’s important not to elaborate or cite evidence at this point. Hearing how your team members rate each dimension allows you to see how aligned you are with one another before you begin justifying a rating and influencing each others’ thinking.
AS A GROUP
6. Discuss any differences in the scores and why people scored differently in each rubric dimension (pay special attention to areas where there’s a greater gap between team members’ scores). If you have areas with larger gaps, the best thing to do is go back to the Common Core Standards. In the EQuIP mathematics video, you see teachers doing this as they discuss using versus finding standard deviation. Thinking and talking about the standards in a meaningful way allows us to teach the standards better and impact student learning.
7. Have group members explain and justify scores by pointing to specific evidence in the lesson. This is where different perspectives and thinking about your own students matters. If you are adamant that a particular criterion is/is not met, you can defend your rating with specific criterion-based evidence.
8. Discuss any issues centered on differing interpretations of the rubric and how the criteria are applied. The conversations often help you see another person’s perspective and reasoning for a rating, and this improves your understanding of the rubric, the standards, and ways to teach the standards effectively.
9. Come to consensus on overall ratings for each dimension, as well as on an overall rating for the entire lesson or unit. It’s important to note that you are not averaging scores together for a final score. Rather, through discussion and professional agreeing/disagreeing, you determine together a final rating for each dimension based on evidence within the lesson.
The EQuIP Review Panel spent time calibrating all together on at least two lessons, before we went into our smaller groups to rate lessons and provide official feedback. Our group facilitators decided we were ready to move on to the real thing when our scores in step five above were closer together than they were apart. As you work with your own teams, you may also decide to calibrate all together on a couple of lessons before working in smaller groups.
By giving everyone time to examine the same materials individually and then collectively, it encourages team members to dig deeply into each criterion in the rubric. It ensures a shared understanding of the intent of the criteria, as well as how we might apply it as a reviewer. This entire process promotes a precise use and application of the rubrics, and it promotes the professional expertise of educators as well. When scoring is calibrated, each lesson or unit receives the same score — regardless of whom on the team scores it — because all team members are interpreting and applying the rubric in the same way.
WATCH OTHERS USE RUBRICS AND EVALUATE LESSONS
Video Playlist: Strengthening Lessons for the Common Core
