The question we hear most often is not "can AI answer our tickets". It is "what happens when AI answers them wrong". That is the right question, and the answer decides whether an AI first line of support helps your team or damages your reputation. Here is where we drew the line and why we drew it there.
The dangerous failure is not a missing answer. It is a confident, wrong one. The customer acts on it and two days later finds out the feature does not exist. You pay twice: your team fixes the mess, and the customer stops trusting what you write.
That is why AI search in the knowledge base answers strictly from the content of your articles, and when it finds no basis for an answer, it returns a fixed sentence saying it is not certain and points the customer to your team. This is not a limitation to work around. It is the only version of such a search worth putting in front of a customer.
Money. Invoices, refunds, plan changes, trial extensions. Every one of these conversations ends in a decision that costs something, and cost decisions are not made on text similarity.
Personal data and access. Changing the account email, adding a user, recovering access. You do not know for certain who is writing, so automation is not the right party to settle identity.
Complaints and outages. A customer writing that email sending has been broken since morning does not need a link to a help article. They need a person who will check whether it is their configuration or our problem.
A conversation that already went wrong. If the customer is writing a second time about the same thing, or the message is visibly angry, automation makes it worse with every additional sentence.
Customer: How do I add a second user to the account?
Answer from the knowledge base: You add users in Settings, in the Users section. Instructions linked below.
Customer: Fine, but I need the invoice issued to a different company than the one on the account.
This is where the first line ends. The ticket goes to the team and the customer is told who will reply and when.
In practice, almost everything depends on how the handover looks. If the customer has to repeat the whole story to a human, the saving disappears and the experience is worse than if they had written to your team in the first place.
So a ticket passed on keeps the full thread, and the person taking it over can see what the customer already read and what was missing from it. The customer gets one sentence confirming a human is on it. Without that sentence they are left wondering whether anyone is reading at all.
An answer from the knowledge base is labelled as an answer from the knowledge base. We do not sign it with an employee name and we do not add artificial delays to make it look typed live. A customer who realises they were talking to a bot presented as a human stops discussing your product and starts discussing being misled.
Three numbers are enough to start. First: how many conversations closed without a human. Second: how many of those came back within a week about the same thing, because those are the answers that only looked effective. Third: how many tickets the automation handed over, which tells you how often it recognises its own limit.
If the first number grows and the second grows with it, your first line is not saving time, it is postponing it by a week. If the third is zero, the automation is answering everything, including what it does not know.
Not with picking a model, but with the knowledge base. Automation is exactly as good as the content it answers from. Start with the twenty questions your team gets most often and write them up properly. Only then connect AI search and set the boundary: money, access, outages and repeat tickets go to a human. The rest can safely go to the first line.
Money matters (invoices, refunds, plan changes), access and personal data requests, outages, and repeat tickets where the customer is writing a second time about the same issue. Each of these needs a human decision or identity verification.
It should say it is not certain and point the customer to your team. A confident wrong answer costs more than no answer, because the customer acts on it and someone has to repair the consequences.
The ticket must keep the full thread, and the person taking over should see what the customer already read and what was missing. The customer gets one sentence confirming a human is handling it, with who will reply and when.
No. An answer from the knowledge base should be labelled as such, with no artificial delays that make it look typed live. A customer who discovers they spoke to a bot presented as a human stops discussing the product and starts discussing being misled.
Three numbers: how many conversations closed without a human, how many of those returned within a week with the same issue, and how many tickets the automation handed over. If the first two grow together, time is only postponed. If the third is zero, the automation answers things it does not know.
With the knowledge base, not the model. Automation is only as good as the content it answers from. Write up the twenty most common questions your team gets, then connect AI search and define which cases always go to a human.
