FIELD GUIDE 05
RAG brings
evidence to generation.
A language model can search a maintained source collection before it writes. The retrieved passages become context for the answer.
01 Start here
What is retrieval-augmented generation?
Retrieval-augmented generation connects a search step to a text-generating model.
A short definition
Retrieval-augmented generation, usually shortened to RAG, is a way to let a language model use relevant material from an external collection while answering a prompt. The application finds candidate passages, adds selected text to the model’s context, and asks the model to compose a response using that context.
The original RAG research described a pre-trained generator paired with a retriever and an external, non-parametric memory. In plain language, some information is stored in documents the system can search, rather than relying only on patterns encoded in model parameters during training. Modern systems use different components and retrieval methods, so RAG describes a family of designs rather than one required software stack. See the original Lewis et al. RAG paper.
What can be searched?
Approved documents, help pages, product notes, records or another collection available to the application.
What looks relevant?
A search and ranking step selects passages that may help answer the question.
What writes the answer?
The language model receives the question and selected context, then generates text.
02 The workflow
From a question
to useful context.
The application has work to do before the model sees the final prompt.
A simple RAG pipeline
- 01 / prepareMake the collection searchable.
Documents may be extracted, divided into smaller passages and indexed. Chunk size, overlap and metadata are implementation choices; they affect what a search can return.
- 02 / retrieveSearch with the user’s question.
The system can compare wording, semantic similarity, filters or combinations of signals. A vector store is one option, not a requirement for every RAG design. OpenAI’s Retrieval guide documents searchable vector stores; its separate File Search guide shows a hosted search tool used with generation.
- 03 / selectChoose passages that fit the question.
Search results may be filtered or reranked and then limited to the context the application can send. More text is not automatically better: irrelevant or conflicting passages can distract the model.
- 04 / generateWrite with the selected text in view.
The model receives a prompt, the question and retrieved passages. Application instructions can ask it to stay within the supplied evidence or say when the sources do not answer.
- 05 / returnShow the answer and useful provenance.
An application can expose links or source labels when it has reliable metadata. Retrieval alone does not guarantee that an answer is cited, complete or supported.
03 Make it concrete
A policy question
with a paper trail.
RAG is easiest to picture when the model needs a changing source.
Example: answering from an employee handbook
Suppose an employee asks whether a particular expense is reimbursable. A RAG application can search the current handbook, retrieve the passages about that expense and provide them alongside the question. The model can then summarize the policy and point to the handbook section—if the application has preserved source links and instructs the model to use them.
This can be easier to update than retraining the model whenever the handbook changes. But the content pipeline still needs to ingest the current version, remove or mark superseded copies and respect document access rules. If search returns an old policy or misses a relevant exception, a fluent answer may still be wrong.
04 What it changes
Fresh context.
Not guaranteed truth.
Retrieval expands what the model can see for a request; it does not certify the result.
Where RAG helps—and what it cannot promise
Bring in current material
Teams can update a searchable collection without fine-tuning the base model for each document revision.
Keep answers near a corpus
Relevant excerpts can give the model task-specific context it may not have in its training data.
Trace the evidence path
With document identifiers and well-designed citations, people can review what the system retrieved.
Measure both stages
Test whether retrieval finds the right material and whether the generated answer uses it faithfully.
RAG does not automatically eliminate hallucinations, repair contradictory documents or enforce permission boundaries. Search quality, source quality, prompt design, access control and evaluation all matter. A robust system also decides what to do when nothing relevant is found, rather than forcing an answer from weak evidence.
RAG and fine-tuning solve different problems. Retrieval supplies context at request time; fine-tuning changes model behavior through training. A project may use either approach, both, or neither after testing the actual need.
05 Build carefully
Test the search
and the answer.
A good demo is not evidence that the full workflow is reliable.
A practical evaluation checklist
- Use representative questions, including ambiguous wording, missing information and edge cases.
- Check whether the expected passage appears among retrieved results before judging the answer.
- Verify that answers reflect the relevant source, preserve important qualifications and avoid unsupported details.
- Check that any displayed citation points to the passage that actually supports the claim.
- Test document freshness, metadata filters and authorization with realistic users and source versions.
- Define a safe fallback for low-confidence retrieval, conflicting sources or questions outside the corpus.
For a visual walkthrough of how a transformer processes tokens, continue to the transformer field guide. If you are learning the broader foundations first, see the machine-learning beginner guide.
FAQ RAG / quick answers
Questions,
answered.
Short explanations with the limits left visible.
01 / answerWhat is retrieval-augmented generation in simple terms?
It is a method in which an application searches an external knowledge source for relevant passages and gives those passages to a language model as context for its answer.
02 / answerDoes RAG train or fine-tune the language model?
Not by itself. A typical RAG system retrieves information at answer time and adds it to the model’s context; changing or indexing documents does not automatically update the model’s learned weights.
03 / answerDoes retrieval-augmented generation prevent hallucinations?
No. Retrieval can provide useful evidence, but the search can miss relevant material and the model can misread, ignore or go beyond the passages it receives.
04 / answerDoes every RAG system need a vector database?
No. Retrieval can use keyword search, vector search or a hybrid approach. The suitable method depends on the corpus, queries, ranking needs and application.
Sources Primary research / documentation
Follow the
mechanism.
- Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (original research paper).
- OpenAI API, Retrieval (vector-store search and indexed file content).
- OpenAI API, File Search (retrieval tool used with model responses).
These sources describe specific research and product implementations. RAG designs differ; the paper and API documentation are not a guarantee that another system uses the same pipeline.