Prepare company documents for AI search by choosing authoritative sources, assigning owners, preserving access rules and defining how changes reach the search system. Collecting every available file is a poor substitute for those decisions.
This is a preparation guide for a knowledge owner working with IT and security. Its output is a collection the team can evaluate: known sources, known users and questions whose answers can be checked. The checklists are proposed working methods, not results from a client deployment.
Start with the employee questions the collection should answer
Ask the intended users for questions they repeatedly bring to a colleague. Keep the original wording and the context that changes the answer. “What is our approval policy?” might mean a different procedure for a purchase, a discount or a recruitment request.
For each question, ask the document owner to identify the passage that should support an answer. Record cases where no approved answer exists. Those are useful evaluation cases because the expected behavior may be to acknowledge missing evidence and direct the user to an owner.
Choose the first collection around that work. An approved operating manual for one team is easier to judge than a drive containing policies, meeting notes, drafts and personal copies with no shared authority rule. Expand once the team can explain what a useful answer looks like.
Create a source register your team can maintain
A source register is a record of what the search system may use and who is responsible for it. It can start as a simple table. The important property is that someone can resolve an ambiguous document without guessing from its filename.
| Source detail | Decision it supports |
|---|---|
| Document owner | Who can confirm that the content is accurate and approved? |
| Source location and identifier | Where should a citation open, and how is the same document recognized after an update? |
| Status and effective date | Is this a draft, a current instruction or a superseded version? |
| Permitted audience | Which users or groups may retrieve the content? |
| Update and removal process | How will the search copy learn about changes and deletions? |
| Escalation contact | Who handles a question the approved material cannot answer? |
Remove duplicates when their status can be established. When two documents conflict, obtain an authority decision or preserve the disagreement explicitly. Asking a model to choose the “most reasonable” policy introduces a decision the business has not made.
Keep historical material when the workflow needs it, but label the role clearly. A user researching an old decision may need a superseded policy; an employee asking what to do today usually needs the current one. The expected use determines the retrieval rule.
Inspect the source format as well as the wording. For scanned pages and tables, check a sample of the text actually passed into retrieval. If row labels or footnotes are lost during preparation, the reviewer needs to detect that before judging the generated answer against an incomplete passage.
Carry document permissions through the answer workflow
Access to the search application must not silently mean access to its entire source library. Map the signed-in user to the documents that user is allowed to retrieve, then test the implementation with representative roles.
Azure AI Search documents approaches to enforcing document permissions across ingestion and query execution. The approaches have different capabilities and availability, so source and connector selection remain part of the design. 1
Build a permission test with the identity team. An allowed user should find the intended material, while a restricted user should not recover it through a search result, an answer or a follow-up question. Then revoke access and repeat the test after the agreed propagation interval.
Include titles and citations in that review. A hidden answer with a revealing document title can still disclose information. The guide to RAG access controls covers the deeper engineering questions; the preparation task is to supply the correct audience rules and people who can validate them.
Agree what happens when a document changes
A retrieval system can have a separate copy or index of the source. Establish what triggers a refresh, how failures are reported and what happens to material that is deleted or restricted. “The document was updated” is not a sufficient acceptance check for the answer a user sees.
Amazon Bedrock, for example, requires synchronization when files in a connected data source are added, changed or removed. That product behavior illustrates why update handling belongs in the scope instead of being assumed. 2
Choose a freshness requirement based on the work. A product instruction that changes during an incident may need a different operating response from a historical research archive. Define whether the system should withhold an answer, show a warning or send the user to the source when it cannot establish freshness.
Assign responsibility for monitoring imports and resolving failures. A scheduled job without an owner leaves the knowledge team unable to explain why a current document is absent. See the two pipelines in enterprise retrieval for the distinction between preparing sources and answering a query.
Run an acceptance exercise before adding more sources
Use a test environment and safe examples to check the collection as a workflow:
- Ask a routine question and inspect whether the answer follows from its cited passage.
- Ask a question that the approved collection cannot answer. Check the response and escalation route.
- Introduce an explicitly conflicting draft and confirm the authority rule still selects appropriate evidence or exposes the conflict.
- Change an approved instruction and confirm the new answer and citation after the agreed update interval.
- Remove a document or revoke a user's access and verify that restricted material stops appearing.
Record answer usefulness, source correctness, freshness and permission behavior separately. A persuasive answer does not compensate for an inaccessible citation or a permission failure. Give each failure to an owner who can distinguish a document problem from an integration problem.
Keep the questions that failed. They form a useful regression collection when the sources, retrieval configuration or model changes. Add questions because they represent a real task or failure, rather than because a target test count looks impressive.
What document preparation will not solve
Preparation cannot create a business rule that no one has agreed, recover information the organization does not possess or establish permission to use a restricted source. Escalate those gaps before treating them as model problems.
An answer interface may also be unnecessary. If employees mainly need to locate a known document, improving titles, navigation and existing search may solve the task. Evaluate that option against the same employee questions before introducing more software.
Questions before starting internal AI search
Must we clean every company document first?
Start with a bounded collection whose owners can verify the content and audience. Fix problems that prevent that workflow from answering useful questions. A whole-company cleanup does not need to precede a focused assessment.
Should we upload confidential files for the first discussion?
Begin with a description of the collection, access roles and example questions. Agree the sharing and deployment boundary before transferring confidential material. Redacted or synthetic examples can help explain the format without exposing the underlying records.
Who should attend a scoping conversation?
Bring a document owner, a representative user and someone who understands identity and source-system access. The internal knowledge solution lists the implementation decisions that their input should resolve.