kakinang INSIGHTS · 2026-08-28
Choosing an AI customer service platform: validate the real workflow
Start selection with customer Help me evaluate: Choosing an AI customer service platform: validate the real workflow. questions, available people, and flows that must not fail—not a feature list. A visible capability is available only after its authorisation and activation are complete.
What data grounds an answer?
Confirm what comes from approved knowledge, what queries live systems, and whether the service plainly states a missing source.
Implementation example: Use ten real FAQs, two unknown questions, and one changed-price question to see whether the system states its evidence and limitation Assign an accountable owner and align the public page, conversation state, and back-office task; verify both a normal enquiry and missing-data or duplicate-submit cases. The model must not cross confirmation, permission, or staffing boundaries. Review completion, repeat questions, and human corrections weekly, then update the operating rule.
How are data isolated and retained?
Conversation data, contact details, and attachments need clear business data boundaries, retention handling, and understandable use notices.
Implementation example: Ask a vendor to demonstrate data isolation between businesses, retention, export, and deletion instead of accepting marketing promises Assign an accountable owner and align the public page, conversation state, and back-office task; verify both a normal enquiry and missing-data or duplicate-submit cases. The model must not cross confirmation, permission, or staffing boundaries. Review completion, repeat questions, and human corrections weekly, then update the operating rule.
Are channels authorised or only advertised?
- Check actual authorisation, platform policy, and activation status for LINE, Meta, and every other channel.
- Implementation example: Put every planned LINE, Meta, or web entry on an acceptance sheet marked authorised, under verification, or activation incomplete Assign an accountable owner and align the public page, conversation state, and back-office task; verify both a normal enquiry and missing-data or duplicate-submit cases. The model must not cross confirmation, permission, or staffing boundaries. Review completion, repeat questions, and human corrections weekly, then update the operating rule.
How does human handoff work?
Ask about trigger conditions, notification recipients, deduplication, work queues, and the customer message when nobody is on duty.
Implementation example: Simulate a refund request, repeated request for a person, and a failed notification recipient to see who takes ownership Assign an accountable owner and align the public page, conversation state, and back-office task; verify both a normal enquiry and missing-data or duplicate-submit cases. The model must not cross confirmation, permission, or staffing boundaries. Review completion, repeat questions, and human corrections weekly, then update the operating rule.
Do data-changing actions need confirmation?
Orders, bookings, external writes, and payment-related actions should show a summary, revalidate, and be confirmed by the customer.
Implementation example: Test booking or external-write summaries, one-time confirmation, repeat results, and a changed price or time slot Assign an accountable owner and align the public page, conversation state, and back-office task; verify both a normal enquiry and missing-data or duplicate-submit cases. The model must not cross confirmation, permission, or staffing boundaries. Review completion, repeat questions, and human corrections weekly, then update the operating rule.
How is usage counted fairly?
- Count a valid request through terminal response, not streaming pieces, retries, or failed redelivery as multiple events.
- Implementation example: Evaluate valid request-to-terminal-response, zero-results, human corrections, and operator time rather than feature-box totals Assign an accountable owner and align the public page, conversation state, and back-office task; verify both a normal enquiry and missing-data or duplicate-submit cases. The model must not cross confirmation, permission, or staffing boundaries. Review completion, repeat questions, and human corrections weekly, then update the operating rule.
Questions teams ask
Is the biggest feature list best?
No. Prefer a scope that reliably solves the core problem and that the team can maintain.
Does knowledge retrieval eliminate wrong answers?
No. Sources, unknown answers, and tool permissions still require validation.
Can governance wait until after launch?
It should not. Data boundaries, confirmation, and human responsibility belong before public service.
What should be confirmed before testing ends?
That data is manageable, permissions are clear, core flows are repeatably verifiable, and inactive capability is not promised.
Sources and further reading
- AI Risk Management Framework — NIST
- OWASP Top 10 for Large Language Model Applications — OWASP Foundation