Guide · Customer support AI
An AI support chatbot that doesn’t embarrass you.
Everyone has seen the screenshots: a support bot inventing a refund policy, agreeing to sell a car for one dollar, or insulting its own company. Those bots weren’t built badly by accident — they were built without scope, grounding, limits and a way out to a human. Here’s the blueprint for one that helps customers and protects your brand.
By Nythrex EngineeringUpdated 3 min read
- 01
Customer message
Web, app, messenger
- 02
Understand
Intent · language · account
- 03
Answer or act
Approved content · scoped tools
- 04
Check
Policy · confidence · tone
- 05
Reply or hand off
Human gets full context
How support bots embarrass companies
01Invented policies
The model fills a gap with a plausible refund or warranty rule that doesn’t exist — and customers screenshot it.
02Jailbroken persona
Users talk the bot into rude, off-brand or absurd statements, then post them.
03Trapped customers
No way to reach a person, or a hand-off that forces the customer to repeat everything.
04Confident about stale content
An old help article outranks the current one; the bot quotes last year’s prices.
05Actions without limits
A bot that can issue refunds without caps or verification is a fraud channel.
The blueprint
- 1
Scope it on paper first
List the top intents by volume. Mark each: bot answers, bot acts (with which tool and limits), or hand off. Everything unlisted is a hand-off.
- 2
Ground every answer
Use RAG over approved help content and policies. Answers cite sources; no source, no answer.
- 3
Actions through narrow tools
“Check order status”, “cancel subscription”, “issue refund up to X after verification” — each with validation and audit, never generic access.
- 4
Guardrails on input and output
Detect prompt injection and abuse, keep the persona on-brand, block topics you never want discussed, and check replies against policy before sending.
- 5
Hand-off that respects the customer
One click to a human, with the conversation summary, detected intent and account context attached. Hand off automatically on low confidence, frustration or sensitive topics.
- 6
Measure and review weekly
Unanswered questions, hand-off reasons and thumbs-down replies drive content fixes and new test cases.
Metrics that tell the truth
| Metric | Why | Trap to avoid |
|---|---|---|
| Resolution without hand-off | Real work taken off the team | Counting conversations where customers simply gave up |
| Customer satisfaction on bot conversations | Quality as customers see it | Only surveying resolved conversations |
| Hand-off quality | Agents shouldn’t have to re-ask | Ignoring the agent side of the experience |
| Answer accuracy on the test set | Catches regressions before customers do | A test set that never grows |
| Cost per resolved conversation | Keeps the business case honest | Ignoring model usage growth |
Frequently asked questions
Keep reading
Want a second opinion on your project?
Tell us what you’re building and where you’re stuck. We’ll reply within one business day with the most practical next step — even if that step isn’t us.
