FDE Academy

Module 2 · AI engineering foundations · Chapter 9 · Free

RAG end to end

18 min1 pipeline diagram1 retrieval lab1 debugging exercise3 checks
The problem The assistant confidently tells an enterprise customer they have 14 days to request a refund. The real policy, updated last year, says 30. Worse, in the same demo it quoted a line from a Legal-only contract. The model was never "wrong" in its writing. It was fed the wrong pages, including pages it should never have seen.

What RAG is

Retrieval-augmented generation means: before the model answers, look up relevant passages from your own data and put them in the prompt. The model then writes an answer grounded in those passages. It lets a general model answer from private, current information without retraining it, and it lets you show sources.

The RAG query path A question is embedded and searched against the vector index with a permission filter, optionally re-ranked, then the top passages and the question go to a language model that writes a cited answer. Question Retrievekeywords + meaning Permissionfilter Re-rank Prompt assembledrules, passages, question Language modelcited answer
  1. The same pieces from Chapter 8 are already in place: your documents were chunked, embedded and stored with metadata.
  2. Search by meaning (embeddings) and by exact words (keyword search such as BM25). Combining them is called hybrid search. Meaning finds paraphrases; keywords find exact names, codes and numbers.
  3. Filter the candidates against the asker's permissions before anything goes near the model. A model cannot leak what it never received.
  4. A second, more careful model reorders a short list of candidates by how well each answers this exact question. Slower than the first pass, so it runs on only a handful of chunks.
  5. Rules, the top few passages with their sources, and the question share the context budget from Chapter 7.
  6. The model answers using the passages and cites them. Instruct it to say "I don't know" when the passages do not contain the answer.
UserRetrievalModelPrompt

Retrieval lab: break it, then fix it

The question is "How long do Enterprise customers have to request a refund?" The corpus has the current policy, a retired policy, a vague marketing FAQ, and a Legal-only contract excerpt. Start with everything off, then switch stages on one at a time. Try asking as a Support agent and as Legal.

What the lab shows With everything off, the vague FAQ ranks first because it is "close in meaning" to the question, and the Legal-only excerpt slips into the prompt for a support agent. Hybrid search pulls up the current policy because it contains the exact words "Enterprise" and "refund". The permission filter is the only control that stops the confidential text from ever reaching the model. Re-ranking helps with order but would not have fixed the leak. Every stage fixes something different.

Diagnose the symptom

When a RAG answer is bad, find the stage that failed before you change anything. Match each symptom to the stage to fix.

Check your understanding

Recap

In the field The most expensive RAG bug in enterprise deployments is not a wrong answer. It is a correct answer built from a document the asker was never meant to see. Teams that treat permissions as a "phase two" item end up redesigning their index. Store the access list with every chunk from day one, and test with a low-privilege user.

Chapter quiz

Four questions. Score 70% or more to mark this chapter complete.

Review