AI search across company records is useful only when an employee can trust both the answer and the boundary around it. A fluent response is not evidence. The application must retrieve only material the requester is currently entitled to see, preserve the identity and version of each source, show where important claims came from and decline when the corpus cannot support an answer. That requires more than placing embeddings in a vector database. It requires an ingestion contract, identity propagation, document-level authorization, retrieval evaluation, prompt-injection defenses, freshness controls and a route back to the authoritative record. This guide lays out that complete design for enterprise teams.
Key takeaways
- Begin with one owned corpus and a narrow employee task, not every repository.
- Carry source permissions and classification into retrieval; never rely on the model to hide restricted text.
- Show citations, excerpts, owner and version so readers can verify consequential claims.
- Evaluate retrieval and answer quality separately using representative questions and denied-access cases.
- Operate freshness, revocation, feedback, incidents and cost as one service with named owners.
Choose a task with an authoritative source
A good first use case has recurring questions, identifiable source owners and a safe abstention path: approved HR policies, controlled product documentation or service procedures. Avoid an initial promise to search mailboxes, personal drives, chat history and every wiki. Define who asks, which decision follows, what sources are authoritative, how current they must be and what the system must never infer. For policy search, the answer may summarize current guidance but should direct exceptions to HR. For support, it may retrieve troubleshooting steps but should not expose another customer’s case. The Edilec guide to RAG knowledge bases for support teams provides a focused operational example.
Build an ingestion contract, not a document dump
For each source, record owner, system of record, supported formats, stable identifier, version or modified time, classification, permissions, retention, deletion behavior and indexing latency. Preserve page, section or record coordinates so a citation can open the exact evidence. Normalize text carefully but retain the original. Chunk around semantic boundaries such as headings, procedures and table rows; fixed-size fragments can separate a rule from its qualification. Remove obsolete duplicates or explicitly rank current versions. Microsoft’s current Azure AI Search RAG guidance identifies content preparation, terminology mismatch, multi-source access and token limits as practical challenges, not incidental tuning details.
| Corpus decision | Weak implementation | Production requirement |
|---|---|---|
| Identity | Filename becomes the key | Stable source and version identifiers |
| Permissions | One privileged crawler with no ACL metadata | Normalized principals and policy attributes |
| Freshness | Nightly refresh for every source | Source-specific update and revocation target |
| Chunking | Arbitrary token windows only | Structure-aware fragments with source coordinates |
| Deletion | Index cleaned manually | Verified deletion propagated from system of record |
Enforce authorization inside retrieval
Authenticate the requester and derive current group, role, tenant and classification attributes from trusted identity services. Apply the authorization filter before candidate passages enter the model context. A broad index created with a privileged crawler is acceptable only if every searchable unit carries enforceable access metadata and policy changes propagate quickly. For an answer assembled from several sources, the requester must be entitled to every supporting passage. Do not use prompt instructions such as “do not reveal confidential content” as access control. Microsoft’s document security filtering example demonstrates the principle that each user’s retrieval set should follow their permissions.
Test denial as deliberately as relevance. Include a user removed from a group moments ago, inherited folder permissions, a document reclassified after indexing, cross-tenant identifiers, administrator support access and a question whose wording hints at restricted material. The response must not reveal the existence, title, count or semantic outline of inaccessible content. Logs should record policy outcome and source identifiers for authorized troubleshooting while protecting sensitive query text. Emergency access needs an explicit elevated path with time limit, reason and audit—not a hidden service account bypass.
Design a permission-aware answer path
After authorization, query processing may combine keyword, vector and metadata retrieval, rerank candidates and select a compact evidence set. Preserve the original user question and make transformations inspectable; query rewriting can silently change intent. Treat retrieved text as untrusted data because a document can contain instructions intended to manipulate the model. Keep application policy structurally separate, restrict tools, sanitize unsupported markup and validate output links. Microsoft’s Foundry RAG security guidance explicitly recommends access control at retrieval time and treating retrieved content as untrusted input.
Make provenance visible and useful
A citation should help the reader check the answer. Show source title, owning team, effective date or version, relevant excerpt and a direct route to the authoritative record. Associate claims with specific sources rather than adding a generic bibliography after a long synthesis. When sources conflict, say so and present the relevant versions; do not let the model resolve policy. When evidence is insufficient, the system should state the gap and offer an owner or search refinement. For sensitive tasks, consider extractive answers or quoted procedures before free synthesis. Provenance also supports incident reconstruction and targeted reindexing.
| Answer condition | Expected behavior | What to record |
|---|---|---|
| Strong current evidence | Answer concisely with claim-level citations | Query, retrieved IDs, versions and response |
| Conflicting sources | Expose conflict and identify owners | Conflicting versions and ranking rationale |
| Insufficient evidence | Abstain and suggest a safe next step | Missing evidence class |
| Restricted evidence only | Decline without confirming hidden content | Authorization result without leaked detail |
| Unsafe instruction in source | Ignore instruction and flag document | Document ID, detection and review outcome |
Evaluate retrieval before judging prose
Create a question set from real work, with source owners marking relevant passages and acceptable answer characteristics. Measure whether retrieval finds the right evidence, not just whether a model writes a plausible sentence. Useful retrieval measures include recall at a chosen candidate count, ranking quality and no-access leakage. Then evaluate groundedness, citation correctness, completeness, useful abstention and task outcome. Segment by repository, document type, language, recency and permission pattern. A single average can hide a broken source. The related Edilec LLM evaluation framework explains how to maintain representative cases and release thresholds.
Include adversarial tests: instructions embedded in documents, malicious links, deceptive metadata, oversized fragments and attempts to reconstruct restricted facts through repeated questions. The NIST Generative AI Profile highlights governance, content provenance, pre-deployment testing and incident disclosure. Convert those themes into service evidence: known corpus, tested threats, documented limits, monitoring, response owner and a way to disable a source or answer mode quickly.
Operate freshness, quality and cost together
Monitor ingestion lag, failed parses, orphaned ACLs, deletion propagation, retrieval latency, answer latency, grounded-answer rate, abstention, citation opens, correction feedback, denied queries, incident signals and cost per useful answer. Sample apparently successful answers because users may not report subtle errors. Route source corrections to the owner rather than patching the index. Version embedding models, ranking logic, prompts and chunking rules; changes can alter access-safe behavior and relevance even when the user interface is unchanged. Establish a rollback to the last evaluated configuration and a read-only search fallback.
Roll out by corpus and consequence
Start with employees who know the source material and can flag omissions. Run shadow comparisons against current search, then expose answers with conspicuous citations and feedback. Expand one corpus or user group at a time only after access tests, quality thresholds and support load are acceptable. Publish limitations and teach users to verify consequential answers. The safe employee assistant guide adds guidance on acceptable use and connected actions.
Practical review checklist
- Prove that every indexed object has a stable source identifier, owner, current version, classification and normalized access attributes. Reject ingestion when required ownership or authorization metadata is missing.
- Measure revocation from source permission change to search denial. Test individual removal, group removal, document reclassification and deletion, and set a target appropriate to the sensitivity of each corpus.
- Build evaluation questions with source owners and include exact terminology, paraphrases, ambiguous requests, conflicting versions and questions with no approved answer. Preserve expected passages, not only expected prose.
- Review the answer interface with real employees. Check whether citations are understandable, source versions are visible, abstention is actionable and the route to report an incorrect source reaches its owner.
- Run a tabletop exercise for restricted-content exposure. Confirm the team can disable a corpus, preserve evidence, identify affected queries, notify accountable owners and verify that corrected permissions have propagated.
Frequently asked questions
Can all company records share one vector index?
They can share infrastructure if tenant, classification and document permissions are enforceable at query time and tested under revocation. Separate indexes may still be appropriate for regulatory, regional, operational or blast-radius reasons. Architecture should follow policy and lifecycle, not a preference for fewer components.
Do citations prevent hallucinations?
No. A model can cite an irrelevant passage or overstate what a source says. Citation correctness, source coverage and claim support need evaluation. The interface should make evidence easy to inspect and the system should abstain when sources cannot support the requested conclusion.
Conclusion
AI search across company records is an authorization and evidence system before it is a chat interface. Bound the corpus, preserve source identity, enforce permissions during retrieval, expose provenance, evaluate retrieval and answers separately, and operate revocation and correction. Those controls let employees gain speed without creating a new route around the company’s records and responsibilities.