I’m often asked to sit in on client’s calibration sessions. Whenever I walk into the room and find 20 people sitting there I silently scream inside and start looking for the nearest exit. It’s going to be a long, frustrating meeting. Each person you add to a calibration session exponentially increases the amount of time you’ll spend in unproductive wrangling and debate.
QA scales are a lot like the law. No matter how well you draft it, no matter how detailed your definition document is, you’re going to have to interpret it in light of many different customer service situations. There’s a reason why our legal system allows for one voice to argue each side and a small number of people to make a decision. Can you imagine the chaos if every court case was open for large-scale, public debate and a popular vote?
One of the principles I’ve learned is that calibration is most efficient and productive with a small group of people (four or five max). If you have multiple call centers or a much larger QA staff, then I recommend that calibration have some sort of hierarchy. Have a small group of decision makers begin the process by calibrating, interpreting and making decisions. If necessary, that small group can then hold subsequent sessions with a broader group of coaches (in equally smaller groups) to listen and discuss the interpretation.
Like it or not, business is not a democracy. Putting every QA decision up for a popular vote among the staff often leads to poor decisions that will only have to be hashed, rehashed and altered in the future. Most successful QA programs have strong, yet fair, leaders who are willing to make decisions and drive both efficiency and productivity into the process.