You can use AI to review 100% of conversations with the customers automatically. The AI works with different accuracy for different cases and it is not usable in every scenario. You can use AI to provide automatic reviews of individual turns in conversations.
How AI Works
In Salted CX, automated quality assurance receives conversation content, including transcripts and individual messages from a customer journey. The workflow applies configured prompts, relevant knowledge, and review criteria to produce automatic reviews. Manual reviews may provide examples for refining and validating those configurations. Customer data is not used to train or fine-tune the underlying AI models.
Prompts, knowledge, and review examples
General-purpose AI does not automatically understand your business rules or quality criteria. Automated reviews therefore use configured prompts, relevant knowledge, and representative human-review examples. The number and type of examples needed depend on the criterion being evaluated.
Human review examples
People can review conversation content and record examples of expected findings through manual reviews. These examples can help refine prompts, knowledge, and evaluation criteria for the configured automated-review workflow. Customer conversations, reviews, and feedback are not used to train or fine-tune the underlying AI models.
Manual Reviews for Improving Automated QA
Representative human reviews help define and validate what an automated review should identify. The number of examples depends on the specific criterion, and some observable behaviors are easier to identify consistently than others.
Turn-Level Reviews
Turn-level reviews provide focused examples tied to the relevant wording in a conversation. This makes it clearer which observable content is significant for the configured review criterion.
Engagement-wide reviews that are common for legacy quality assurance do not point to the exact moment in a conversation which makes it significantly more difficult for AI to understand what content in the conversation is the main contributor to the searched phenomenon. Legacy quality assurance would lead to significantly lower accuracy and the need to provide orders of magnitude more engagements.
In the example above you see a comparison of turn-level tags with engagement level quality assurance questions. You can notice that the snippet from a conversation contains examples of both good objection handling and poor objection handling. Turn-level tags provide focused evidence of what reviewers consider a positive or negative example.
With engagement-level questions, the totally opposite behaviors are not distinguishable from each other. Additionally, an engagement-level example can contain unrelated conversation topics that make the intended criterion less clear.
To sum it up engagement-wide questions used in legacy quality assurance processes have these issues:
- Noise from unrelated parts of a conversation. If the behavior appears only in part of the conversation, the remaining content can obscure the intended criterion. This can require more manual-review examples and may still reduce accuracy.
- An engagement can contain both good and bad behavior. In this case, the engagement is not suitable as one undifferentiated example. Using it that way can lead to non-actionable findings and reduced accuracy.
Tags and Questions
Provide relevant manual-review examples for each tag or question used by the automated-review workflow. You can create custom tags and questions for AI-powered auto reviews.
When you create tags and questions for AI-powered quality assurance these tags and questions should be individually actionable:
- Unwanted behavior. The behavior of either an agent or a customer that represents a situation that is not welcome in your business. You might want to address these either by talking to the agent or changing processes and policies.
- Exceptionally good behavior. The behavior that might be used as a good example is when you exceed customer expectations thanks to your products, services, processes, or agents. This should not include a baseline expected performance from the agent. Such behavior should be considered a default and should not require attention.
- Unexpected behavior. The behavior that is new or unexpected and you may want to check how common it is and whether it is necessary to adapt your processes and train agents for it.
Behavior that is not worth noticing and represents an expected customer experience is not worth tagging in most cases. We recommend spending the time as efficiently as possible and focusing on behavior you can act on later on — you can tell that you need to talk to an agent, change something in your business process, or fix something in a product.
Check built-in tags and questions.
How to Review Conversations
To review a conversation in an AI-friendly manner we strongly encourage you to follow our recommendations in the Quality Assurance article. Especially focusing on these:
- Read a conversation in chronological order. This gives you an understanding of the context and helps you to relive the customer experience.
- When you encounter an unwanted behavior, exceptionally good behavior or an unexpected behavior tag it. You can use one of the built-in tags, create your own, or if the behavior is something new use the generic Bookmark tag to return to it later.
- Wrap up your review session by considering whether you want dedicated tags for a new behavior you have encountered. You might also consider adding more granularity to existing tags to distinguish between behaviors you want to report on separately.
Process Improvements
Companies evolve and externalities change. This leads to discoveries in conversations on an ongoing basis.
- Keep your tags and questions up to date. When you encounter a new behavior that is worth watching create a tag for it. Also, remove tags for behaviors that no longer appear in the conversations.
- Add tags and questions to your forms after review sessions. If you could not tag a behavior from the form you had at hand during the review session consider adding the tag to your form to avoid switching between different forms. This saves you some valuable time. Always try to balance the length of the form with its usefulness for your reviews. Longer forms may be harder to navigate and slow down some reviews.