FDE Academy

Module 2 · AI engineering foundations · Chapter 8 · Free

Embeddings and vector databases

15 min1 interactive map1 decision tree3 checks
The problem A support agent types "how do I get my money back?" into the company search. The policy is titled "Refund and cancellation terms" and never uses the words "money back". Keyword search finds nothing. The search needs to understand meaning, not just match letters.

An embedding is a position for meaning

An embedding model turns a piece of text into a list of numbers called a vector. The training makes texts with similar meaning land close together and unrelated texts land far apart. Searching by meaning then becomes a geometry problem: embed the question, find the nearest stored vectors.

Below is a 2D picture of that idea. Real embeddings have hundreds or thousands of dimensions, but the intuition carries over. Pick a question and watch which snippets are closest. Try the second one, "Who can see employee salaries?": it sits between two topics, which is exactly where real systems get confused.

What to notice The question "How do I get my money back?" never uses the word "refund", yet it lands next to the refund snippets. That is the value of embeddings. The salary question lands between access control and HR, which is why permission filters (Chapter 9) matter: closeness in meaning says nothing about who is allowed to read a document.

Where the vectors live

You need somewhere to store millions of vectors and quickly find the nearest ones to a query. That is a vector database, or a vector index inside a database you already run. Step through the flow.

Indexing and querying a vector store Two lanes. Offline: documents are split into chunks, embedded and stored in a vector index with metadata. Online: a question is embedded, the index returns the nearest chunks, and a filter applies permissions. OFFLINE: indexing Documents Chunks Embedding model Vector indexvectors + text + metadata(source, owner, access list) ONLINE: each question Question Embedding model Nearest search+ filter by access Top-k chunks
  1. PDFs, wiki pages, tickets, emails. Their quality limits everything downstream.
  2. Documents are cut into passages of a few hundred tokens, often overlapping slightly. Too big and the match is vague; too small and the meaning is lost.
  3. The embedding model turns every chunk into a vector. Use the same model later for questions.
  4. Vectors are stored with the original text and metadata such as source, date, owner and who may read it. The metadata is what makes filtering and citations possible.
  5. The user asks in their own words.
  6. Same model, same space, so the question lands near relevant chunks.
  7. The index returns the closest vectors. At scale it uses an approximate method (such as HNSW graphs) that trades a little accuracy for large speed gains. Filters on metadata, like permissions, apply here.
  8. The best k chunks go to the next step: usually a language model that writes the answer (Chapter 9).
UserModelVector storeYour data

Which store should you pick?

There is no universal answer. Answer the questions for a typical customer situation.

Features, limits and pricing of specific products change often. Treat this as a framework and check current documentation before choosing.

Mistakes that bite later

Check your understanding

Recap

In the field A search demo worked beautifully on a handful of documents. On the full corpus, near-duplicate versions of the same policy flooded the top results and the outdated one ranked first. The fix was not a better model: it was metadata (document version and date) and a rule to prefer the current version.

Chapter quiz

Four questions. Score 70% or more to mark this chapter complete.

Review