An enterprise knowledge base cannot be assessed solely through a convincing demonstration. It must keep working when procedures change, an employee leaves a team or a document becomes confidential. That maintenance work belongs in the initial scope of an AI project.
Permissions must follow the information
Signing into an application does not grant access to all its content. Retrieval must account for the permissions of the person asking the question. The passages sent to the model should already respect that scope. An instruction telling the AI to “keep information confidential” is not a substitute for access controls.
We therefore recommend documenting groups, exceptions and the time required for changes to take effect. Someone removed from a project should not continue retrieving its documents because the index holds outdated permissions. This scenario deserves an explicit test.
Deleting a file may leave dependent content behind
A document may have produced several items: passages, search index entries, vector representations, summaries and cached answers. Our proposed approach is to retain a link between each derived item and its source, then define how to update, remove or rebuild it.
A summary combining several documents requires another decision: who can read it? Our default recommendation is to avoid widening access to the information used. If a summary for a broader audience is needed, its approval should follow a procedure defined by the company.
A hypothetical example: a maintenance procedure changes
Imagine a manufacturer replacing a machine adjustment instruction. The new version is approved, but an older summary remains available in the AI knowledge base. A technician asks a question and receives an answer based on that outdated summary.
The issue extends beyond model quality. The system needs to identify the applicable version, preserve useful references and withdraw invalidated derivatives. Archived documents may remain stored for their specific purpose while being excluded from routine operational answers. This example illustrates a risk to test; it is not a Mintera customer case study.
Decisions to make before production
- A business owner: who approves knowledge and its review date?
- A lifecycle rule: what happens to superseded versions, deletions and dependent summaries?
- A response to uncertainty: when should the system request approval or decline to answer?
- A maintenance budget: who handles connectors, reindexing, checks and human reviews?
A consulting engagement with practical acceptance criteria
A Mintera Knowledge project can start with a limited document scope while preparing for expansion. The proposed scoping work produces a map of sources and permissions, publication and withdrawal rules, a set of business tests and an estimate of operating costs.
The useful measures are practical: answers grounded in the current approved version, unauthorized access observed during testing, the time needed to propagate a permission revocation, correction effort and monthly maintenance costs. Together, they inform the decision to expand the system.
The purpose of an initial deployment is also to expose responsibilities that a demonstration can hide. Who resolves conflicting procedures? Who approves a revised answer? Who monitors a failed synchronization? Assigning these tasks is part of the work. A knowledge base becomes an asset when the company knows how to maintain it.