Knowledge Base (RAG)
The collection of documents an AI agent retrieves from at decision time. The mechanism is called retrieval-augmented generation (RAG).
Also known as: KB · RAG · retrieval-augmented generation · knowledge corpus
What is Knowledge Base (RAG)?
A knowledge base is the collection of documents (policies, procedures, FAQs, product specs, prior client communications) that an AI agent retrieves from at decision time. The retrieval mechanism is called retrieval-augmented generation (RAG): the agent searches the knowledge base for relevant chunks, includes them in its working context, and produces output grounded in those sources. This is how agents stay accurate on business-specific information without being retrained. The quality of the knowledge base directly determines the quality of agent output.
Christos Papadimitriou, theagency47 · Updated May 15, 2026How RAG works in practice
When a question comes in:
- The agent (or a retrieval layer) embeds the question into a vector.
- The vector is matched against pre-embedded knowledge base chunks.
- The top 3–10 chunks are pulled into the agent’s context window.
- The agent generates a response grounded in those chunks, with citations.
If a relevant chunk is missing from the knowledge base, the agent typically should not invent an answer, it should escalate. This discipline is what prevents hallucination at production scale.
What goes into a knowledge base
For a typical client engagement, the knowledge base includes:
- Company policies and procedures
- Product documentation and FAQs
- Past correspondence in the firm’s voice (for voice training)
- Industry-specific regulatory text (where relevant)
- Client lists with notes (for personalization)
- Eval cases (for retrieval verification)
theagency47 typically ingests 50–500 documents during a Spark build and up to 2,000+ for Workforce Pro engagements. When the knowledge base is the bottleneck, we build it as a standalone foundation first, a Second Brain with Obsidian: a plain-markdown vault your agents read from and write back to.
FAQ
What happens when the knowledge base is wrong or outdated?
The agent’s output is wrong. KB hygiene is the most important ongoing maintenance task, typically handled under a retainer. We rerun the eval suite after every KB update.
Is RAG the same as fine-tuning?
No. Fine-tuning modifies the model itself (expensive, slow, hard to update). RAG keeps the model fixed and changes what data the model sees at inference time. RAG is the standard pattern in 2026 for business-specific knowledge.
Related terms
System Prompt
The instructional text that defines an AI agent's role, voice, boundaries, and decision rules. The single most important customization layer of an agent.
Hallucination
An AI agent producing output that sounds confident but is factually wrong, typically because the model invented information not present in its inputs.
AI Agent
A software system that takes a goal, plans steps, calls tools, and acts autonomously to produce an output, without step-by-step human prompting.