Thoughts from the Calibration Trenches

Yesterday was calibration marathon day. Three different calibration sessions with three different teams with a staff meeting scrunched in between. It's not exactly what most people would consider an enjoyable day at the office. Granted, compared to countless calibration sessions I've endured with many different client's, our calibration sessions are a cake walk.

Nevertheless, as I was driving home I got a call from one of my teammates struggling with discouragement after the session and we had a great conversation about the calibration process. It got me thinking about some basic lessons I've learned through the years in calibration:

  • Calibration, by its' very nature, is a conflictive process. When you try to get a group of people to analyze the same call the same way, there are bound to be disagreements. The calibration session is not focused on the 90-95 percent of the call a team agrees on, but on the handful of things on which they disagree. You have to accept this going in and keep it in perspective. It's always wise to try and bring some levity and laughter to the session. Remind people of all the things that you agreed on which weren't conflictive. Keep the big picture in front of the team.
  • Calibration is often not about who is "right" and who is "wrong" but how we are going to consistently and objectively approach and analyze a given behavior or situation. People will see things differently. Often, I recognize that our team is grappling with multiple, legitimate ways to analyze a given situation. Because a manager or a team decides to do it a particular way does not mean that another person's way of doing it was "wrong," it just means that someone had to choose the method that works best in that moment. A good manager will regularly encourage his or he team with this fact.
  • A constructive calibration process will not get mired in a singular circumstance, but look for patterns and principles to apply across all calls. Many calibraiton sessions turn into a war over a small piece of one call. I am always asking myself, "what's the principle we can glean from this discussion that will help us be more consistent in scoring all of our calls?" Our team will keep a "Calibration Monitor" document that tries to summarize the general principles we discussd in the session which will aid all analysts with future calls.
  • You have to choose your battles. I will sometimes feel very strongly about a given situation when I was the lone person in the room who seemed to view it that way. Making my argument and stating my case is only met with blank stares. Despite the tremendous personal effort it takes to let it go, I have learned that it makes no sense to keep arguing. If it is a worthwhile and relevant issue, then I will have another opportunity in future calibration sessions to make my point when more people might see it. If that opportunity never emerges, then I was making a mountain of a mole-hill anyway.

One thought on “Thoughts from the Calibration Trenches

  1. This post helped me, I was faced with this situation in my last calibration session. I would like to know if any other QA analyst have issues with agents supervisors taking the scores personal? Also, is it posible to get everyone on the same page as it relates to monitoring agents? Its seem that supervisors and agents always think QA is out to get them no matter what. My stand point has always been the same I am hear to brin awareness to area’s of improvement to help the center improve our performance. Please share!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s