A Simple Way to Start Calibrating

Scores I sat in the office of a call center manager as I shared the results of an audit our group had performed of their quality program. In this case, we had collected a sample of calls that had been scored by the supervisors and scored the same calls using their internal form. Not only were we able to provide a number of procedural recommendations that would improve their quality process, but we were able to unearth which supervisors tended to be unduly harsh or lenient in the analysis of their teams.

As we covered the results for one particular supervisor, the call center manager laughed and looked at the quality manager. In unison, they rolled their eyes. Our audit revealed that the supervisor commonly missed marking down for obvious quality errors. As a result, the quality scores for the team were grossly inflated. Both the call center manager and the quality manager suspected this had been the case.

While calibration can be a very detailed process, there is a very simple way to get started. Begin by comparing the average overall service scores of everyone who analyzes calls on your team. Use a minimum of thirty calls per analyst. Who is high? Who is low? What’s the average? How big is the variance? While this doesn’t give you much detailed information, it can be a road map for your first steps.

  • If you suspect that the analyst with the highest average overall service score may be inflating scores, simply pull a few calls and score them yourself. Compare your evaluation with that of the analyst. You’ll notice pretty quickly if things are being missed.
  • Likewise, the person with the lowest average may be marking things more strictly than the rest of the team. Scoring a few of the same calls and comparing your scores to theirs can be very revealing.
  • If the average overall scores are within a few data points, it’s a good sign that your quality form is driving a consistent result.
  • If you have a wide range in average overall service scores, it’s quite possible that you’ve got a lot more work to do.

The call center manager with whom I shared our audit results knew that they had issues with their quality process simply by looking at the average overall scores of his supervisors and knowing that the scores couldn’t be an accurate reflection of the service being delivered. By doing a more thorough audit of their quality process, they were able to drive down to parts of the process which were creating the problems and make positive changes. As a result, the entire call center team feels better about the quality process, service quality has improved and customer satisfaction is up!

Creative Commons photo courtesy of Flickr and brtsergio

  1 comment for “A Simple Way to Start Calibrating

  1. August 26, 2008 at 9:02 pm

    I’m very blessed. The managers are more than happy to tell me if my scores are off. There’s always room for improvement in everything you do.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: