Retrieval augmented generation means the AI looks something up before it answers
Retrieval augmented generation, RAG, is a technique where an AI system searches a set of documents for relevant information and feeds what it finds into the model alongside the question, before generating an answer. It exists because a model's training data is fixed and can be outdated or missing anything private to a business, and retrieval gives it something current and specific to work from instead of relying only on what it memorised.
Definition
Retrieval augmented generation is an architecture that pairs a search step with a generation step: given a question, the system first retrieves the most relevant passages from a defined set of documents, then passes those passages to a language model as context so its answer is grounded in that retrieved material rather than in memorised training data alone.
The three steps, in plain terms
Retrieve: the question is converted into a search against a document store, an internal wiki, a set of product docs, a database, and the system pulls back the passages most relevant to it.
Augment: those passages are inserted into the prompt sent to the model, alongside the original question, as extra context the model would not otherwise have.
Generate: the model writes its answer using both the question and the retrieved material, which grounds claims that would otherwise depend entirely on what it happened to memorise during training.
Why it exists: what a model alone gets wrong
A language model's training data has a cutoff and is fixed once training ends. RAG does not retrain the model to fix that; it updates the document store instead, so an answer can reflect this week's information without the model itself ever changing.
A model also has no visibility into private data by default. RAG connects it to a business's own documents, so an answer can be grounded in that specific business's material rather than only in general knowledge from the open web.
The model does not need to know everything by heart if it can look the right thing up before it answers.
Where a business actually runs into RAG
A customer support assistant grounded in a company's own help documentation, so it answers from the actual policy rather than inventing a plausible one.
An AI workflow system that checks a specific record, an order status, an inventory count, a CRM entry, before answering, rather than guessing at a number it was never given.
AI answer engines fit the same pattern at a larger scale. When ChatGPT or Perplexity browse the web to answer a question, they are retrieving pages in real time and using them as grounding material before generating a response, the same underlying idea applied to the open web instead of one business's documents.
| Approach | What it changes | Best fit |
|---|---|---|
| Prompting alone | Nothing about the model; relies only on what it already knows | Simple questions the model's existing knowledge already covers |
| Fine-tuning | Retrains the model's weights on a custom dataset | Teaching a stable skill or style, not injecting new or changing facts |
| RAG | Adds an external, updatable knowledge source the model reads at answer time | Answers that depend on current, private, or frequently changing information |
What RAG does not fix
Retrieval only helps if the document store actually holds the right material and the search step finds it. Thin, outdated, or poorly organised documents produce thin, outdated, or poorly grounded answers, the retrieval step cannot invent quality that is not there.
It also does not eliminate errors outright. A model can still misread a retrieved passage or apply it to the wrong part of a question, so RAG reduces one specific failure mode rather than guaranteeing correctness.
Common questions
What does RAG stand for?
Retrieval augmented generation. Retrieval finds relevant information, augmented means that information is added to what the model sees, and generation is the model producing the final answer from both.
Is RAG a way of training a model?
No. Fine-tuning retrains a model's weights; RAG leaves the model itself untouched and instead gives it extra material to read at the moment it answers. Updating a RAG system usually means updating its document store, not retraining anything.
Does RAG stop an AI system from making mistakes?
No, it reduces one specific cause of errors, the model inventing facts it never had, by giving it real material to draw from. The system can still misread or misapply what it retrieved, so accuracy still depends on the quality of the document store and how relevant the retrieval step judges its results to be.
Is RAG something only large companies use?
No. Any business with a defined set of documents, help articles, product specs, internal policies, can run a small RAG setup, and most AI workflow systems and modern chatbot builders now offer it as a built-in option rather than something built from scratch.
How is RAG related to AI answer engines like ChatGPT and Perplexity?
When those systems browse the web to answer a question, they are effectively doing retrieval at web scale, fetching pages and using them as grounding material before generating a response. It is the same underlying idea applied to the open web instead of one business's own documents.
AI workflow systems
Where retrieval-backed answers fit inside a business system.
What is an AI workflow system
The broader category RAG-backed tools belong to.
What is an AI agent
How an agent differs from a single retrieval-and-answer step.
AI workflows vs manual processes
What changes when a lookup step gets automated.
See whether a workflow system with retrieval built in could replace a manual lookup step in your business.