Work · Support tooling
Signal
Model-assisted triage that routes confident cases automatically and gets everything else to the right person faster.
Client
Confidential · B2B software
Sector
Support tooling
Duration
12 weeks
Our role
Applied ML, backend, product design
Triage · queue 42
routed → Billing
routed → Legal
needs a human
54%
tickets auto-routed
−41%
first-response time
96%
routing precision on the eval set
The challenge
Nine support agents triaged every inbound ticket by hand. Around half were routine and repetitive; the rest needed a specialist who often saw them a day late.
Our approach
We built a retrieval-backed classifier with an explicit confidence threshold and an evaluation harness run on every change. Above threshold, tickets route and draft themselves. Below it, they go to a person with the retrieved context attached — the model never closes a ticket alone.
Deliverables
What we shipped
- 01Retrieval-backed triage and routing service
- 02Confidence thresholds with human-in-the-loop fallback
- 03Evaluation harness with a labelled regression set
- 04Agent-facing draft replies with cited sources
- 05Quality dashboard tracking drift and override rate
Technology
- Python
- Postgres
- pgvector
- Evals