A quality score of 82.5% looks precise, but it is the output of many design choices: which behaviours were measured, how much each one weighs, and which mistakes are so serious that they cancel everything else. Understanding those choices is the difference between reading a quality report and being misled by one.
Start with a single call
Imagine a customer service process for a broadband provider. Its quality form has seven scored parameters with weights that add up to 100:
| Parameter | Weight |
|---|---|
| Opening and greeting | 5 |
| Probing and understanding the issue | 15 |
| Accuracy of resolution | 30 |
| Hold and transfer etiquette | 10 |
| Empathy and tone | 15 |
| Documentation in the CRM | 15 |
| Closing and further assistance | 10 |
An agent resolves the issue correctly but puts the customer on hold twice without asking permission, losing all 10 points for hold etiquette, and sounds rushed, earning half credit for empathy (7.5 of 15). The call scores 100 − 10 − 7.5 = 82.5%.
The weights carry the process’s priorities. Here, resolution accuracy is worth six times the opening, which reflects a belief that getting the answer right matters far more than a scripted greeting. A form where the greeting and closing together outweigh accuracy is telling agents to polish the edges and neglect the middle.
What makes an error “fatal”
Some mistakes cannot be offset by good behaviour elsewhere. If the agent discloses account details to someone who failed verification, a warm tone does not compensate. Quality forms handle this with fatal errors (sometimes called critical errors or auto-fails): if one occurs, the call scores zero regardless of the other parameters.
Common examples, which vary by process and client, include:
- Skipping mandatory identity verification or sharing data with an unverified caller.
- Giving incorrect information that leads to financial loss, such as a wrong refund amount or wrong fee.
- Missing a mandatory regulatory disclosure, for example on a collections or insurance call.
- Rude or abusive behaviour, or disconnecting a customer deliberately.
- Processing a transaction the customer did not authorise.
- Wrong or falsified call disposition that misstates what happened.
A useful test for whether something belongs in the fatal list: would the client, the customer or a regulator consider this call a failure even if everything else was perfect? If yes, it is fatal. If the answer is “it was a bit sloppy”, it belongs in the weighted parameters.
Three kinds of critical error
Many mature quality frameworks split critical errors by who is harmed. The naming varies, but the logic is consistent:
- Customer-critical: the customer is hurt or left worse off. Wrong information, issue not resolved when it could have been, a promise of a callback that was never logged.
- Business-critical: the client company loses money or reputation. An unauthorised waiver, a refund issued outside policy, a cancellation processed without the mandated retention attempt.
- Compliance-critical: a law, regulation or contractual control is breached. Failed verification, missed disclosure, card details captured in a way the payment security rules forbid, personal data shared on an unofficial channel.
The value of this split is in the reporting. “Fatal error rate 6%” tells a manager little. “Customer-critical 4%, business-critical 1.5%, compliance-critical 0.5%” points at different owners and different fixes: knowledge gaps for the first, policy clarity for the second, controls and possibly disciplinary action for the third. A single call can carry more than one category, and the framework should say whether it is counted once or in each.
From calls to a monthly number
Suppose 50 calls were audited for an agent this month. Three had fatal errors. The other 47 averaged 88% on the weighted parameters. There are at least three honest ways to report this, and they give different numbers:
- Quality score with fatals applied: (47 × 88 + 3 × 0) ÷ 50 = 82.72%.
- Non-fatal score: 88%, which shows how well the agent handles the softer parts of the call.
- Fatal accuracy: 47 of 50 calls free of fatal errors, or 94%.
Some processes, especially back-office ones, use an opportunity-based accuracy instead. If each transaction has 8 checkpoints and 50 transactions were audited, there were 400 opportunities for error. With 22 errors found, accuracy is 1 − 22 ÷ 400 = 94.5%, or 55,000 defects per million opportunities if you are speaking Six Sigma. This is fairer to complex transactions but can make a single serious error look small, so it is usually paired with a separate fatal count.
The important habit is to say which method a number uses. “Quality 88%” and “quality 82.72%” can describe the same agent on the same calls. When a vendor and a client calculate differently, they will argue about performance while both being arithmetically correct.
Design mistakes that distort the score
Too many fatal parameters. If a missed customer name counts as fatal, fatal rates balloon, agents stop taking the category seriously, and genuinely dangerous errors get lost in the noise.
Partial credit everywhere. A five-point scale on every parameter adds subjectivity and makes calibration harder. Binary yes/no marking on clearly defined behaviours is often more reliable, with partial credit reserved for genuinely graded behaviours like tone.
Weights that never change. If documentation has scored near 100% for a year, its 15 points no longer discriminate between agents. Reviewing weights periodically keeps the form pointed at what currently goes wrong.
Not applicable treated as a pass. If a call had no hold, does the hold parameter score 10 out of 10, or is it removed from the denominator? Either is defensible, but mixing both across auditors quietly inflates some scores.
Written by the RP Strativa Team. The form, weights and figures are illustrative.
Leave a Reply