Use embeddings for similarity search without confusing similarity with truth
Compare semantic retrieval with keyword search, preserve document permissions and evaluate whether the relevant evidence is returned.
In brief
An embedding represents content as numerical features used for comparisons. Similarity search can find related wording even without an exact keyword match, but the score is not a probability that an answer is correct.
It is useful when users ask the same question with different words, while exact identifiers may still need keyword matching.
Choose a task where meaning helps
A user may ask “recover a deleted document” while the source says “restore a file.” Semantic retrieval can connect these expressions. A product code or exact error number is a different kind of query; keep exact matching available rather than assuming vectors solve every search.
Keep context with the vector
Store the original passage, source, version and permission metadata alongside the representation. Retrieve only authorized material. A vector without the readable evidence cannot support an answer a reader can verify.
Measure retrieval quality
Test whether the needed passage appears among the results and whether unrelated passages dominate. Use the same dataset for comparisons and keep costs in view. Do not publish a “better search” percentage unless you actually ran and documented that evaluation.
Before you act
Embeddings can carry sensitive information and depend on the model that created them. A different embedding model can require rebuilding the index; mixing incompatible vectors is not a valid shortcut.
What to do next
- Define representative search questions and the passages that should answer them.
- Compare a keyword baseline with semantic or hybrid retrieval on the same material.
- Inspect misses and false matches, preserving version and access filters in the index.
