A maintenance technician asks which gasket fits a particular pump revision. The assistant finds the manual for a similar pump, returns a plausible part number, and cites the wrong manual. The answer reads well because the model has enough context to sound certain. The mistake began earlier, when the retrieval pipeline treated a nearly identical model identifier as interchangeable.
Retrieval-augmented generation, or RAG, gives a language model information retrieved for the current request. RAG tooling can cover document parsing, indexing, retrieval, ranking, answer generation, or evaluation. A team that buys an end-to-end service still has to decide which documents count as authoritative, who is allowed to see them, and how to tell whether an answer follows from the passages it cites.
Follow a document from source to answer
Start with the original asset. A maintenance manual might include text, parts tables, diagrams, and revision history. A PDF parser that extracts prose cleanly may flatten a table into an unreadable sequence. The index can then contain all the words from the manual while losing the association between a pump revision and its gasket. Inspect extracted content before tuning retrieval. For table-heavy material, preserve headings, row relationships, and enough context to interpret individual values.
The index typically stores chunks, which are pieces of a document used as retrieval units. Short chunks can improve precision for a specific fact, but they may separate a rule from its exception. Large chunks preserve context at the expense of sending more unrelated text to the model. One practical design retrieves a smaller passage and adds a surrounding section when context is necessary. Preserve a stable reference to the original location, along with the source version and any metadata needed for filtering.
At query time, the system selects candidates and may rerank them. The generator receives a subset and writes an answer. Log enough at each step to tell whether the relevant source was missing from the index, excluded by permissions, omitted from the candidates, pushed down by ranking, or ignored by the model. That distinction determines whether the next fix belongs to document ingestion, search configuration, or answer generation.
Give exact terms and meaning their own jobs
Lexical retrieval is a useful baseline for identifiers, policy numbers, error codes, and names. Vector retrieval can help when users ask questions in different words from the source documents. A technician might ask about a “rubber seal” even when the manual uses “gasket.” Hybrid retrieval combines results from both approaches. Azure AI Search is one example that documents running text and vector queries in parallel and using reciprocal rank fusion to combine ranked results. Azure AI Search retrieval guidance
Treat a model revision, jurisdiction, or effective status as structured data when it is a hard requirement. Boosting an identifier can improve rank, but a boost still permits another revision to appear. A validated filter can constrain results to the correct revision, provided the system has reliable metadata and the user’s identifier is clear. Where the identifier is ambiguous, the application can ask for the missing detail or show relevant alternatives without pretending that one applies.
Evaluate a reranker only after looking at the candidate pool. A reranker can improve the order of items it receives. It cannot rank a document the first stage never retrieved. Capture candidate recall before tuning the final order. Measure the extra latency and cost on the traffic likely to benefit. A multi-stage pipeline may serve ambiguous policy questions well while a simpler lookup handles exact part numbers faster.
Enforce access before content reaches the model
The application needs a trusted basis for deciding what the user can retrieve. Some systems carry access metadata into the index and apply filters at query time. Other architectures retrieve through an upstream source that enforces permissions. Examine how your chosen method handles inherited permissions, group membership changes, and documents shared with a limited audience. Microsoft documents several document-level access patterns for Azure AI Search, with applicability depending on the configuration and source. Azure AI Search document-level access
A useful permission test has two users ask the same question before and after one loses access. Inspect retrieval, answer text, citations, and shared caches. The revoked source should disappear according to the system’s defined access-change requirements. Test cached answer reuse across users as a separate condition. A well-filtered query can still be undermined if a cached answer was generated under a broader identity and later reused for someone with less access.
Retrieved content may contain instructions that resemble commands for the model. Keep tool authorization and data access in the application and service controls. Include documents with suspicious embedded text in tests and inspect whether the application treats them as evidence to interpret or instructions to follow. This matters most when the same application can retrieve documents and perform actions.
Evaluate retrieval and generation independently
A test set should capture common questions and the ways your corpus is difficult. Include an exact lookup, a paraphrase, a question that requires two sources, a retired policy, a question with no supporting source, and a query from a user with limited access. For answerable cases, identify supporting passages and the facts an answer needs. For unanswerable cases, define what a useful response looks like, such as asking for a revision number or pointing the user to the right owner.
Measure whether the system retrieves the evidence and whether the answer uses it correctly. An answer can mention a valid source while misreading a threshold or failing to include an exception. Sample citations at the claim level. Check that a user can follow each link and find the relevant support in the source. Where a source is restricted, check behavior with the affected user role instead of reviewing under an administrator account.
Keep the index revision with the result. Retrieval quality can shift when documents change even if the application code and model stay the same. For large collections, define how stale a document or its permissions may be before it poses a problem. A temporary delay may be acceptable for general reference material. A withdrawn instruction that could affect a business decision needs a more direct removal and verification process.
Decide how much infrastructure to own
A managed retrieval service can reduce operational work around search infrastructure. Review the ingestion connectors, export options, access model, and per-query components you would pay for. A modular stack can let a team replace a parser, embedding model, or search engine independently. It also leaves the team responsible for handling failures between services and preserving consistent identifiers across them. A framework that orchestrates requests can be useful in either design, especially when it makes each stage observable.
Estimate ongoing cost with a representative document update and a representative query. Include parsing, embedding, indexing, storage, reranking, generation, and reindexing after a model or schema change. Then run the pilot with documents that have conflicting revisions and different permissions. The ability to show exactly why the system selected a passage, and to remove it when it stops being valid, tells you more about production fit than a polished answer on a clean sample corpus.