9
Generative AI is only useful to a professional firm when it can be trusted. And it can only be trusted when every answer it produces can be traced to a source you control. That is the entire point of retrieval-augmented generation, or RAG, done correctly: the model does not answer from the open internet. It answers from your files, your precedents, your policies, your client records — and it tells you where each answer came from.
Done badly, RAG is a leak dressed up as a feature. The model retrieves from a shared or public corpus, or from a vendor’s cloud, and the firm loses the one thing it cannot afford to lose: control over what its AI is reading and repeating.
Grounding means a closed corpus
The first requirement is that the retrieval source is your own corpus and nothing else. The documents the model can draw on should be the documents you have indexed — your matter files, your engagement letters, your internal policies, your approved precedents. If the system can reach outside that set, it is not grounded. It is guessing with a wider net.
This is not a technical nicety. When a partner asks the system a question, the answer is only as defensible as the source it cites. A grounded system returns an answer with a citation to a specific document in your corpus, so the answer can be checked, challenged, and relied upon. An ungrounded system returns a confident paragraph with no provenance, and no one can say where it came from.
Run it where the data lives
The second requirement is infrastructure. If your corpus is sensitive — and for a law firm, a CPA practice, a government contractor, or an insurer it always is — the retrieval and the generation should run on hardware you control, inside your own network. The moment the corpus is sent to a vendor’s cloud to be indexed and queried, you have recreated the leak you were trying to close. The data leaves your walls, and your audit trail ends at the network boundary.
On-premises RAG keeps the entire loop inside your perimeter: the documents never leave, the index never leaves, the model never leaves, and the answers are produced and logged where your controls apply.
The audit trail is the deliverable
The third requirement is the one that makes the others defensible. Every query, every retrieved source, and every generated answer should be logged. When a regulator, an insurer, or a client asks what your AI did with their information, you should be able to produce a record: this question was asked, these documents were retrieved, this answer was given, and nothing else was touched.
That audit trail is what turns AI from a risk into an asset. It is the difference between a system you can explain and a system you can only hope was behaving.
RAG done right is not about making the model smarter. It is about making the model accountable — grounded in your files, running on your infrastructure, and leaving a record you can defend. That is the only version of AI a professional firm should run.
Start with the Confidential AI Exposure Checklist at 9tailsadvisory.com/checklist/. Nine lives. Zero leaks.