Platform — Knowledge Base
RAG Knowledge Base for Voice AI: how mid-call retrieval actually works
Your AI receptionist doesn't memorise your price list. It searches it — during the call, in under 400 milliseconds — and answers from the exact document you uploaded. This is Retrieval-Augmented Generation (RAG), and it's the difference between an AI that invents answers and one that cites yours.
What RAG actually is, in plain language
Large language models (LLMs) are trained on billions of words from the internet. They're good at language — grammar, tone, conversation flow. But they don't know your business. They don't know your prices, your hours, your insurance list, or your cancellation policy. If you ask them a specific question about your practice, they'll either refuse to answer or — worse — invent something plausible-sounding but wrong.
RAG solves this by adding a retrieval step before generation. When a caller asks a question, the system doesn't just send the question to the language model. It first searches your uploaded documents for relevant passages, finds the best match, and then sends both the question and the retrieved passage to the model. The model's job becomes: "answer this question using this specific text." It generates a natural-sounding response grounded in your actual data.
Think of it like giving a smart assistant a filing cabinet. Without the cabinet, they'd have to guess. With it, they look up the answer and read it back to you — in their own words, but from your documents.
How mid-call retrieval works — step by step
Caller asks a question
"Do you take Delta Dental PPO?" — the AI recognises this as a factual question that needs a verified answer, not a conversational response.
Vector search runs against your documents
The question is converted to a vector embedding and compared against your pre-indexed documents. The system finds the most semantically similar passages — in this case, your insurance list PDF, page 2, "Accepted Plans" section.
Retrieved passage is sent to the model
The LLM receives: "Answer this caller's question using ONLY the following text: 'We accept Delta Dental PPO, Premier, and HMO plans. Patients with Delta Dental PPO have no waiting period for preventive services.'" The model cannot invent — it can only use what it's given.
AI responds naturally
"Yes, we do accept Delta Dental PPO. There's no waiting period for preventive services like cleanings and exams. Would you like to schedule an appointment?" — accurate, natural, and sourced from your actual insurance list.
Audit log records the source
The call transcript shows: source document "insurance-list-2026.pdf", page 2, confidence 0.94. Reviewable by your team for compliance or quality assurance.
Total time from question to answer: under 400 milliseconds. The caller hears no pause.
Why RAG stops the AI inventing answers
A language model without RAG is like an employee who memorised a textbook but never read your specific policies. They'll sound confident, but they might tell a caller you accept an insurance plan you don't, or quote a price from two years ago, or invent a cancellation policy that doesn't exist.
With RAG, the model is constrained. It can only answer from the passages retrieved from your documents. If no relevant passage is found (confidence below threshold), the AI doesn't guess — it says: "I don't have that specific information. Let me transfer you to someone who can help." This is the critical safety mechanism: the AI knows what it doesn't know.
This is fundamentally different from "fine-tuning" a model on your data. Fine-tuning bakes information into the model's weights — but the model can still hallucinate, you can't update it instantly, and there's no audit trail showing where an answer came from. RAG keeps your data external, searchable, updatable, and auditable.
What you upload to the knowledge base
Common uploads
- • Price lists and service menus
- • Opening hours and location details
- • Insurance/provider lists (dental, medical)
- • Cancellation and refund policies
- • FAQ documents
- • Product specifications
- • Staff bios and specialties
Supported formats
- • PDF (including scanned with OCR)
- • Word documents (.docx)
- • Excel spreadsheets (.xlsx)
- • Plain text and Markdown
- • HTML and web page URLs
- • Google Docs (auto-sync)
- • Notion pages (auto-sync)
Most businesses are live with 3–5 documents. You don't need to upload everything — start with the questions callers ask most often (pricing, hours, insurance, services offered) and add more over time.
The audit trail: know exactly where every answer came from
Every time the AI uses your knowledge base to answer a question, the call transcript records:
Source document
Which file the answer came from
Page and section
Exact location within the document
Confidence score
How closely the passage matched the question
Retrieval latency
Time from question to retrieved passage
This matters for regulated industries (dental, medical, legal) where you need to prove the AI gave correct information. It also helps you spot gaps — if the AI frequently says "I don't have that information," you know which documents to add.
Worked example: "Do you take Delta Dental?"
⚡ RAG retrieval (380ms) → insurance-list-2026.pdf, page 2
"Accepted dental plans: Delta Dental PPO, Delta Dental Premier, Delta Dental HMO, Cigna DPPO, MetLife PDP, Aetna DMO. Delta Dental PPO: no waiting period for preventive; 6-month wait for major."
⚡ Calendar check → Thursday 2:30 PM available
The AI answered from the practice's actual insurance document, not from training data. If the practice drops Delta Dental next month, they update the PDF and the AI immediately stops saying "yes."
RAG knowledge base FAQ
What is RAG in plain language?
RAG stands for Retrieval-Augmented Generation. Instead of relying on what a language model 'memorised' during training, RAG searches your actual documents during the conversation and feeds the relevant passages to the model before it answers. The model generates a response grounded in your real data — not from memory, not invented.
How fast is mid-call RAG retrieval?
Under 400 milliseconds from question to retrieved passage. The caller experiences no noticeable pause — the AI's response sounds as natural as if it already knew the answer. This is possible because documents are pre-indexed into vector embeddings when uploaded, so retrieval is a similarity search, not a full-text scan.
What file formats can I upload to the knowledge base?
PDF, Word (.docx), Excel (.xlsx), plain text, Markdown, HTML, and web URLs. You can also connect Google Docs and Notion pages for automatic sync. Most businesses upload their price list, service menu, FAQ document, insurance/provider list, policies, and opening hours.
Does RAG completely prevent hallucinations?
RAG reduces hallucinations to near zero for questions covered by your documents. If the answer exists in your knowledge base, the AI will find and cite it. If the answer does NOT exist in your documents, the AI says 'I don't have that information — let me transfer you to someone who can help' rather than inventing an answer.
How does the audit trail work?
Every AI response that uses your knowledge base is logged with: (1) the exact document and page/section the answer came from, (2) the passage that was retrieved, (3) the confidence score of the retrieval match, and (4) the final response generated. This is visible in the call transcript and exportable for compliance review.
How often should I update the knowledge base?
Whenever your information changes. Updates are instant — edit a document and the next call uses the new version. There is no retraining or redeployment step. Most businesses update their knowledge base when prices change, new services are added, hours shift, or policies update. The AI always answers from the latest version.
Upload your first document. See RAG in action on a live call.
Most businesses are live with 3–5 documents in under an hour.