Retention schedules used to be about compliance. With AI, they are also about whether your answers are right.
An AI assistant answers from whatever it can find. If your files include several drafts of the same policy, a superseded version, and a trivial working copy, it has no reliable way to know which one is current. It picks what looks most relevant, and you get an answer that sounds confident and may be wrong.
How old content drowns the good content
- Superseded policies. The rule changed, but the old document still sits next to the new one.
- Drafts and duplicates. Many near-identical files compete for the same question.
- Trivial and expired items. Files that should have been disposed of long ago take space in what the AI searches.
- Missing metadata. Without a category, date, or owner, nothing marks a file as current or expired.
This is also a growth problem. Information keeps accumulating, so a mess this year is bigger next year unless something retires it.
What a useful retention schedule looks like for AI
- Simple enough to apply. Dozens of overlapping categories are hard to automate. Consolidate where the law allows.
- Attached to the content. Each file carries its category and dates, so a system can tell what is current.
- Backed by a disposal process. Expired content is actually removed, in batches, on a schedule.
- Clear on event-based rules. Triggers such as contract end or employee departure are the hardest to implement. Name an owner and a fallback period.
Disposal has to reach the AI's index
Many AI systems keep a searchable copy of your content. If you dispose of a document but the copy stays in the index, the AI can still answer from it. Ask three questions:
- Does the metadata from the original file, including retention category, travel with each indexed piece?
- When a file is disposed of, is its indexed copy removed too?
- How would we prove it?
How to test whether the AI forgot
- Run evals. Build a small set of test questions whose answers should have changed after a disposal. Run them after every cleanup and every system update.
- Query the index directly. If you use a vector database, search it for the content you removed and confirm nothing matches.
- Keep the results. The test log is evidence that your disposal reached the AI.
Automation can help, with limits
Some practitioners report that automatic classification can match or beat manual review when the categories are well defined and the content is clean. But accuracy matters. One speaker cautioned against tools that are right only about 72% of the time for sensitive records. Use confidence scores, send low-confidence items to a person, and sample the high-confidence ones to check them.
A practical starting plan
- Pick one repository and name an owner.
- Measure how much of it has a category and a date.
- Dispose of what has expired and consolidate what is duplicated.
- Connect AI to that clean repository first.
- Run your eval questions, then widen the scope.
We build document pipelines that classify, extract, and validate content, with a human reviewing what the model is unsure about. See intelligent document processing and Is your data ready for AI?. Related reading: how AI exposes data nobody labeled.
Illustrations in this guide are drawn from talks at an industry information-governance conference and from general practice. They are anonymized and are not Horus client results.
Part of our guide: Information governance for AI