In many companies AI-powered quality assurance is a new tool that people are not used to. This can bring some challenges that you want to be aware before using AI so you can get your colleagues aligned.
Finding Issues is Easier than Solving Them
Naturally, issues are easier to find than it is to solve them. There might be many technical, process, and regulatory reasons why an issue is not easy to solve. Any issue found in the contact center also competes with other priorities.
Recommendations
There is no silver bullet for every possible issue. Solving every issue you find is rarely possible. Solving every issue is also rarely necessary. Prioritization is the key. You can use the traditional prioritization method to compare the effort versus value.
Priority of each item is based on:
- Criticality (vertical axis) — How bad it is when this problem appears in a conversation. Low criticality can be punctuation issues, minor clarity issues, etc. High criticality are typically legal, compliance and privacy issues. You can roughly categorize issues based on knowledge of your business.
- Effort (horizontal axis) — How difficult is to solve the issue. Low effort could be actions such as talking to an agent, adding extra info to training, etc. High effort could be technical issues that need to be fixed in you platform, process issues, issues requiring cross-team coordination, etc.
- Quantity (bubble size) — How often the issue happens in your conversations. Auto Reviews in Salted CX help you to quantify this. Use data from Salted CX to either justify effort spent on a solution or use the data for leaving the solution for later.
Each of the above metrics is approximate and it is often difficult to calculate them precisely. Human judgement is necessary when using these indicators to prioritize issues to resolve.
How to approach discovered issues that do not have an immediate low effort solution:
- Start watching the issue quantity in reporting. This makes sure that the issue does not happen more frequently in the future which could influence its priority. Watching the trend also enables you to confirm any taken action has an actual impact.
- Let other people know about issues you find. This tells people that you are aware of the issue and the reason the issue is not resolved has a reason — low priority, high effort, it is work in progress, blocker on a vendor side etc. The good place to track these issues is your issue tracking such as Asana, JIRA, Trello and similar tools. You can also create a form in Salted CX that contains list of these issues.
- Consider partial non-perfect solutions. It is tempting to resolve any issue completely and perfectly. This is not always necessary. Perfect solution may be an order of magnitude more difficult to implement than a good solution or an acceptable solution. Involving coworkers when thinking about these steps can help with different perspectives and ideas how to tackle issues from different angles.
- Build any higher effort solution iteratively. Try to deliver solution in smaller chunks. You can check in reporting what impact each iteration has. If the issue becomes rare enough after an iteration you can decide not to build the next iterations.
Working with Approximate Numbers
The accuracy of AI depends on the specific scenario you would use it for. In every case, the accuracy will not be 100% percent. This would be also the case for people if they were performing the same job. It is practically impossible to perfectly calibrate people. However, when people review conversations this is not that visible as they review only a small subset of conversations.
Most people do not have much experience when working with an approximate numbers.
Recommendations
Salted CX also provides features that help to improve accuracy and mitigate the impact of inaccuracies:
- Every Auto Reviewer should go through calibration to ensure that the findings really represent moments in conversations you intended to detect.
- Provide feedback to auto reviews. Quality assurance people can mark any finding by AI as Correct, Unclear, or Incorrect. This feedback is used for training new versions of the model.
- Let agents acknowledge and dispute auto reviews. While AI searches for potential issues in agent behavior there is a natural incentive for an agent to dispute the findings. Agents can use agent profile to provide feedback to auto reviews.
Transparency
With easy access to issues found in conversations it is easy for anybody in the company to check what is currently happening. Communication and transparency is critical to ensure everybody is on the same page.
To maximize transparency:
- List you current issues in priority order in a single list or board. A shared list of the current issues helps everybody to see issues in the perspective.
- Have a clear process to report new findings. Ensure people have a way to report a new issues they discover either in dashboards in the customer journey. They also have to have a confidence that once they report anything it is acted on even if that means just giving it a low priority.
- Information at users’ fingertips. When you have a dashboard use the description sections to give users more information about how to use the dashboard and what conclusions to get out of the dashboard. If you create custom metrics also provide similar details to their description.