Read-Only by Design
Why the AI agent in a law firm's case system can't change anything — and what it took to make that useful anyway.
The question isn’t whether the model is accurate enough to write to your case system. It’s whether anyone would notice when it isn’t.
That’s the argument. Everything below is how I got there, and what the architecture looks like once you accept it.
The first thing anyone asks for is an agent that updates the case.
Have it move the matter to the next phase when the records come in. Have it fill in the incident date from the police report. Have it clear the checklist item when the document lands. It is the obvious product. Every demo of an AI agent on a case management system is some version of it, and the demos are good, because in a demo the model is right.
I decided not to build it. What I built instead is a system with no write path to the firm’s system of record at all — not a careful write path, not a guarded one. There is no function in the codebase that writes to it. This is a note about why that turned out to be the right call, what the architecture looks like when you make it, and what would have to exist before I’d change my mind.
The write isn’t the risk. The undetected write is.
Start with the failure mode, because it’s the whole argument.
A model mis-parses a date on a scanned report and writes it into the field the firm’s deadline reporting keys off. That is not a bug you find in a log. It’s a bug you find when the case is gone.
Three properties make this worse than the equivalent human error.
It’s silent. A paralegal who enters the wrong date is a person who was in the file. There is a memory of it, a Slack message, a “wait, I thought we said.” A model write leaves a row and nothing else. If the write succeeded, the system’s own account of itself says everything is fine.
It’s plausible. An LLM’s errors aren’t random noise; they’re confident and well-formed. Confident and well-formed is precisely what survives review. The errors most likely to be caught are the ones least likely to occur.
It propagates into the record. The case management system isn’t just an operational tool — it’s the firm’s account of what it knew and when it knew it. Every downstream artifact inherits from it: the deadline report, the attorney’s queue, the discovery response, whatever a court eventually sees. A bad value doesn’t sit still.
Weigh that against what the write actually buys you. It saves a person a field entry. The asymmetry is not close.
The shape that comes out of that decision
Two processes, one database.
A pipeline adopts a read-only copy of the case data into a normalized schema. A deterministic rules engine reads that copy and produces flags. A language model turns flags into English. A web application serves the result, governed per user. Hub-owned data — the audit log, triage state, saved views — is written locally. The system of record is never touched.
The important property here isn’t a policy. It’s an absence. There is nothing to misconfigure, no permission to get wrong, no environment variable that flips it on. The architecture document for the system says it in one line — it does not write back — and that line is true because there’s no code that could make it false.
That absence is what pays for everything else. It’s why the model can be given real matter context without a committee meeting, and why the firm’s answer to “what can it break” is a short conversation rather than a long one.
The deterministic layer decides. The model only phrases.
The rules that decide what deserves attention are plain thresholds in a configuration file. Not a prompt, not a fine-tune, not a judgment the model is asked to make:
# The rules decide WHAT is flagged; the model only ever PHRASES things later,
# and a person always reviews.
stalled: # open matter with no activity for N days
tiers:
- { min_days: 60, severity: high }
- { min_days: 30, severity: medium }
- { min_days: 14, severity: low }
sol:
window_days: 30 # flag a provided deadline within N days
high_within_days: 14
statute_years: # YEARS from incident date — PLACEHOLDERS the
default: 2 # firm's attorneys MUST confirm; used only to
mismatch_tolerance_days: 30 # ESTIMATE and cross-check, never to advise
high_value_stalled: # an expensive matter that has ALSO gone idle
min_est_value: 100000
stalled_days: 14
severity: highWhether a matter has been idle for thirty days is arithmetic. It is not an inference, and asking a model to make it buys nothing and costs determinism: the same data on Tuesday yields a different answer than on Monday, and no one can tell you why. A threshold in a file can be changed by the firm, reviewed in a diff, and reproduced exactly.
The deadline logic was the case where the boundary had to be written into the code itself, in a docstring, because the code is where the next engineer looks:
Estimated and cross-checked dates are an assist to verify — never legal advice. Statutory periods are attorney-confirmed placeholders.
The engine will flag an approaching deadline, estimate one when the field is empty, and tell you when the date on file disagrees with the computed one. It will not tell anyone what their deadline is. That distinction is the difference between a tool and a liability, and it lives in configuration and comments rather than in a policy document nobody reads.
So the division of labor is: rules decide what, the model decides how to say it. That’s a smaller job than it sounds. It is also the only job in the system where being occasionally clumsy has no consequence.
One gate, applied in one place
Every governed read passes through the same four steps: check the domain, narrow the rows, run parameterized SQL, record the access. The gate itself is eleven lines:
// Domain gate: if the user's department isn't granted the domain, log a denied
// access and short-circuit to empty — the data never leaves the DB.
async function gate<T>(
user: CurrentUser, domain: Domain, resource: string, run: () => Promise<T[]>
): Promise<T[]> {
if (!canSeeDomain(user, domain)) {
await logDenied(user, resource);
return [];
}
return run();
}Breadth within a permitted domain is a second, separate decision, and it returns both the SQL fragment and the string that will describe it in the audit log:
export function dataScope(
user: CurrentUser,
opts: { attorneyCol: string }
): { clause: string; params: unknown[]; summary: string } {
// attorneyCol is interpolated as a raw SQL identifier, so it MUST be a trusted
// constant — never request-derived. assertSqlIdent is the guardrail against a
// future caller passing user input.
assertSqlIdent(opts.attorneyCol);
const base = `dept=${user.department},level=${user.level}`;
if (user.level === "individual") {
return {
clause: `${opts.attorneyCol} = ?`,
params: [user.name],
summary: `${base},own=${user.name}`,
};
}
return { clause: "1=1", params: [], summary: `${base},all` };
}Two details matter more than they look.
The scope decision and the audit description are produced by the same function call, so they cannot drift. There is no version of this where the log says a user saw their own matters and the query returned the whole book of business.
And the permission matrix — which department reaches which data domains, who may see clinical detail, who may see money — is data, in one module, with no imports beyond types, so it can be unit-tested directly. Governance spread across twelve call sites is governance that’s wrong at three of them. I’ve seen the version where each screen does its own checking. It passes review and it is not correct.
Where the model is allowed to run
The confidentiality problem in a legal AI system is mostly a question of where the text goes. Florida’s opinion is direct about it: an in-house system that doesn’t share information with outside parties substantially reduces the concern that a third-party provider’s self-learning model surfaces one client’s information in another user’s session.
So the backend allowlist is enforced in a constructor that throws, not a convention in a README:
ALLOWED_BACKENDS = {"stub", "bedrock"}
class LLM:
def __init__(self):
self.backend = os.environ.get("LLM_BACKEND", "stub")
if self.backend not in ALLOWED_BACKENDS:
raise ValueError(
f"LLM_BACKEND={self.backend!r} is not an approved backend "
f"{sorted(ALLOWED_BACKENDS)}. Client data (possibly PHI) may only "
"go to the local stub or in-tenant Amazon Bedrock."
)Production runs against a model inside the firm’s own cloud tenant. Local development runs against a deterministic template stub with no network and no API key, which means the whole platform runs on a laptop with nothing to leak — and swapping between them is an environment flag, not a code change.
There’s a second constraint on the input side that does more work than the allowlist. Callers pass structured context only — a dictionary of matter facts already minimized (name: "Matter 4471", never a client name), rendered into fact lines. End-user free text is never concatenated into a prompt; the question set is fixed and parameterized. That closes the prompt-injection path, and as a side effect the model never sees a client’s name at all.
The audit log is a surface, not a debug tool
"""Observability: record every agent action to the audit trail.
This is the backbone of the platform's "nothing happens in a black box" promise.
PHI-SAFE RULE: input_summary / output_summary are for COUNTS, IDs, and config —
never client free-text or medical content. Keeping the audit trail free of PHI
keeps the log itself out of disclosure scope.
"""Two things I’d argue for in any system like this.
Log the denials. A governance layer that only records successes cannot demonstrate that it works. The denied read — user, resource, dept=X,denied, zero rows — is the evidence that the gate is load-bearing rather than decorative.
Keep content out of the log. It is very tempting to log the model’s output so you can debug it later. Do that and you’ve created a second copy of the sensitive data, in a table with weaker access controls, that nobody thinks of as a data store. Identifiers, scope, and counts are enough to reconstruct what happened. And audit logging never breaks a read — a failure to log is an operational signal, not a reason to fail the user’s request.
What the ethics opinions push toward
Both of the relevant opinions — the Florida Bar’s 24-1 and ABA Formal Opinion 512 — read on the surface as permission with conditions. Lawyers may use these tools; they must protect confidentiality, verify output, supervise appropriately, and bill honestly.
Read as an engineer rather than a lawyer, the conditions have a shape.
Verify the output. Then the system must produce drafts, and the interface has to say so, and there has to be a moment at which a person could have caught it.
Supervise the tool the way you’d supervise a nonlawyer assistant. Then there must be a review step and a record that it happened. Supervision that leaves no trace is indistinguishable from no supervision.
Know how the tool uses your data, and don’t rely on boilerplate consent buried in an engagement letter. Then the fewer places the data goes, the smaller the problem gets — and “it runs inside your own tenant” is a much easier sentence than the alternative.
None of these say don’t write to the system of record. What they say, collectively, is that the lawyer remains responsible for the output. And responsibility you have no mechanism to discharge is a design defect. If a model writes to the system of record and no human ever sees the write, there is no point in the process at which review could have occurred. The obligation still exists; the architecture has just made it impossible to meet.
That’s the connection I find most useful, and it’s an engineering observation, not a legal one: read-only isn’t a compliance concession. It’s the design that makes the review requirement mechanically satisfiable.
I’m not a lawyer, and none of this is legal advice. What a firm’s obligations are is for that firm and its bar counsel to determine. I can only tell you what I designed against.
What would have to exist before it writes
I want to be exact about the status of this section: for this system, it’s designed, not deployed — no part of it runs against a firm’s case data. But the first piece already runs, just elsewhere.
A proposal is a value, not a call. The model’s output would be an object — target field, current value, proposed value, the evidence it’s derived from, a trace id — and not an invocation of anything. Nothing executes by virtue of existing. This sounds like a technicality. It’s the entire safety property: the difference between a system where a bug causes a write and one where a bug causes a queue entry nobody approves.
That part isn’t hypothetical — I ship it in a different tool. My own internal project-management system (no client case data anywhere in it; it tracks my own delivery process, not a firm’s matters) has an assistant that proposes tool calls the same way: each one lands as a row with status='proposed', and nothing runs until I approve it — at which point it goes through the exact service layer the UI itself calls, so there’s no separate “the assistant did it” code path:
// POST /api/chat/actions/:id/apply — owner approves a proposed action; runs it
// through the gate-enforcing service layer. 400 (still 'proposed') on a hard error.
chatRouter.post('/chat/actions/:id/apply', (req, res) => {
const id = Number(req.params.id);
const a = db.prepare("SELECT * FROM chat_actions WHERE id = ? AND status = 'proposed'").get(id) as any;
if (!a) {
res.status(404).json({ error: 'action not found or already resolved' });
return;
}
let input: any = {};
try { input = JSON.parse(a.input || '{}'); } catch { input = {}; }
try {
const result = applyAction(a.tool, input, a.project_id ?? null);
db.prepare("UPDATE chat_actions SET status='applied', result=?, resolved_at=datetime('now') WHERE id=?").run(result, id);
res.json(db.prepare('SELECT * FROM chat_actions WHERE id = ?').get(id));
} catch (e: any) {
res.status(400).json({ error: e?.message ?? 'could not apply the action' });
}
});What it lacks is everything below. Every proposal gets the same one-click approval regardless of what it touches — fine for logging a decision, wrong for a limitations date.
Commit tiers keyed to the field, not the confidence. Attaching a received document to a checklist: auto-commit. Advancing a matter’s phase: attorney review. Anything touching a limitations date, a settlement figure, or a client-facing communication: named sign-off, recorded, non-delegable. The tempting design is to tier by model confidence, and it is a trap — it lets the model promote its own work, and confidence is highest exactly where the error is most plausible. The field’s blast radius doesn’t move. The model’s self-assessment does.
A disclosure gate on the way up, not only the way down. Before a matter’s facts reach the model at all, they should pass the same domain and sensitivity check a human read passes. Privileged or clinical content shouldn’t get a lower bar because the reader is software. In the current system this is partly free — the model only ever receives minimized structured context — but a real write path means richer context, and richer context needs the gate made explicit rather than incidental.
I haven’t built the tiering or the disclosure gate — for this system or any other. The honest reason is that the read-only system does what it was scoped to do, and no one has yet wanted the write badly enough to accept the review load. That may change. If it does, the queue mechanic above is already proven; the tiering and the disclosure gate are where I’d start, and I’d expect to throw away a third of it on contact with the first real workflow.
What it costs
Read-only means a person still does the data entry. The system will tell you a matter has been idle for thirty-one days, that the document checklist has a gap, that the deadline on file disagrees with the one computed from the incident date. It will not fix any of it. Firms notice this, and some of them are disappointed.
What they get in exchange is that nobody has to audit the AI’s edits, because there are none. There’s no reconciliation process, no quarterly review of what the agent changed, no incident where a field moved and nobody can say who moved it. For a firm where the deadline calendar is the business, that trade has been worth more than the typing.
Which brings it back to the question at the top. Notice that it isn’t a question about the model — it’s a question about the system the model sits in, and it has an answer before you’ve evaluated a single vendor. If nothing in your process would surface a wrong write, then accuracy isn’t the variable to tune. Detectability is. Buy the model that’s good enough and spend the rest of the budget on being able to tell when it’s wrong.
Most of the code above is from a system I built for a personal injury firm — the excerpts are the generic governance and observability layer, nothing in them specific to any client, and the threshold values shown are illustrative. The approval-queue excerpt near the end is from a separate internal tool of mine (project management, not case management) with no client data in it at all. Every demo on this site runs on synthetic data.
Sources: Florida Bar Ethics Opinion 24-1 · ABA Formal Opinion 512