When we set out to build SristiBot, we thought we were building a chatbot. Eighteen months later, we had built a conversational AI platform, a retrieval-augmented generation engine, a multi-tenant orchestration layer, and a real-time analytics pipeline. Here's what we learned along the way.
Start with the Data Problem, Not the Model
Our biggest early mistake was focusing on the AI model before solving the data problem. Enterprise customers have knowledge scattered across SharePoint, Confluence, Salesforce, PDFs, email archives, and legacy databases. Before SristiBot could answer a question, we needed a robust ingestion, chunking, and indexing pipeline. We spent three months on this infrastructure and it was the best investment we made.
RAG Architecture: The Devil Is in the Chunking
Retrieval-Augmented Generation (RAG) is the right architecture for enterprise knowledge bases — but naive chunking destroys context. We tried fixed-size chunks (too much context loss), sentence-level chunks (too granular), and recursive character splitting (better, but still inconsistent). Our current approach uses semantic chunking with overlap, plus metadata tagging at ingest time. Response accuracy improved by 38% when we switched.
Latency Is a Feature, Not an Implementation Detail
Enterprise users have zero tolerance for slow AI responses. Our initial architecture had P95 latency of 4.2 seconds — way too slow. We solved this through streaming responses (perceived latency drops dramatically), aggressive caching of common queries, parallel retrieval from multiple vector stores, and model quantisation for less complex queries. We now consistently hit P95 under 1.8 seconds.
Multi-Tenancy Requires Paranoid Data Isolation
When one customer's data leaks into another's responses, trust is destroyed. We built strict tenant isolation at every layer: separate vector store namespaces, row-level security in PostgreSQL, per-tenant encryption keys, and automated isolation audits that run daily. This was more engineering work than we expected, but it's non-negotiable for enterprise.
What We'd Do Differently
In hindsight, we would have built the analytics and observability layer earlier. Understanding which questions SristiBot fails on — and why — is critical for improvement. We also would have invested sooner in automated evaluation: LLM-as-judge pipelines that continuously score response quality against a golden dataset. Both of these paid enormous dividends once we implemented them.