The problem
People kept asking support questions whose answers were already sitting in the platform's own data. The information existed. It just wasn't reachable at the moment someone needed it.
Answering meant a human reading the portal on the user's behalf. That does not scale with event volume, and it is slowest exactly when an event is live and the pressure is highest.
What I built
- A retrieval pipeline over portal data: chunking, embeddings, and semantic search tuned so what comes back is actually relevant, not just textually similar.
- An answer layer that stays inside the retrieved context, so the assistant says it doesn't know rather than inventing something plausible.
- Token by token streaming to the UI over the Socket.IO channel already in the product, so answers appear as they generate instead of after a long silence.
- Built into the existing NestJS backend rather than bolted on beside it. One auth model, one deployment, no second system to operate.
The hard part
Getting it to refuse. A RAG system that always produces something looks great in a demo and quietly destroys trust in production. The real work sits in retrieval quality and in the confidence thresholds that decide when the honest answer is “this isn't in your data”.
Outcome
Fewer repeat tickets
Removed a recurring class of support request
Answers in the product
No switch to a separate support channel
Grounded, not guessed
Declines when the data has no answer
Stack
Have a similar problem?
Thirty minutes, no pitch. If this isn't the right approach for you, I'll say so.