Method
How AI is a tool I work with and an accessibility feature itself
I decide what gets built, I am the imaginer, and I am the final check. The AI drafts and does the legwork on my ok. This page is how that holds together across many sessions, and the rules I keep because AI is still evolving.
One loop, every session
The brief is mine. The spec says what good means before anything is built. The agent builds. I review the copy block by block and rule on each one. A correction worth keeping becomes a rule, the rule goes into the next handoff, and the next session starts from there.
A written handoff, not a memory
An AI session starts without the last one’s context. So every session that changes anything ends by writing a numbered handoff: where things stand, what is still open, and what nearly went wrong. The next session reads it before it touches anything. The numbers run in one straight line; when the work forks, the open items get a tag instead of a second series of files. The rules for writing one are copied word for word into every handoff.handoff-017 §5
- **State, not narrative.** "The X is broken because Y" beats a story about discovering it. Nobody needs your journey. - **Traps are worth more than successes.** If something cost you an hour, write it down in one sentence so it costs the next agent nothing. The stale-clone and `index.lock` notes in handoff-001 ยง2 are the model. - **Never claim something is done that you have not verified.** "Deployed" means you saw it serve. "Written" means it is on disk. Say which.
handoff-017.md §5. Curated: the section has more.Rules I keep because an AI broke them
These are the rules I hold AI work to, across all of my projects. Each one was drawn from misses written down in handoffs, reviews and audits, and each row links one of them.
| The rule | A miss behind it | Where |
|---|---|---|
| Measure the thing itself, not a stand-in for it: drag the slider, Tab through the page, load it logged out | Three layout sliders shipped doing nothing. Each force measured correctly in a script; in the app, none of the sliders had any effect. I caught it by using them. R-050 | the research graph |
| “Verified”, “clean” and “deployed” are claims. Check them from outside | A security policy passed on a local server and blocked the analytics script on the live site. R-028 | this site |
| The checker can be wrong too. Test the test | A validator check reduced, algebraically, to a constant: it could never fail. R-045 | the research graph |
| Say what you read, not what exists: “I could not see X”, never “X is missing” | For about ten sessions a legal database was recorded as blocked to every tool. A real browser loaded it normally. R-080 | the research graph |
| A quote is verbatim, or it is empty | A research agent reported a footnote with an exact title and date. The footnote does not exist; it was assembled from true parts. R-075 | the research graph |
| A recommendation, a premise or a summary is a hypothesis until it is checked | Three of one session’s four recommendations rested on something false. R-072 | the research graph |
| Never invent a date, a figure or an organisation. Unknown means ask me | A site analysis invented a registered charity for my rare-disease site, with made-up social-media links to go with it. R-140 | bareyourrare.org |
| No selling: no claims that go stale, no vanity numbers, no commitments that are not the AI’s to make | The contact page promised I would reply “within a working day”, a commitment only I can make. I set it to “within a day or two”. handoff-003 | this site |
| One copy of each fact. A claim repeated in three notes is still one claim | An agent said three times that pushes were safe because there was no automatic deploy. The wrong belief was in three notes, so every check agreed with it. R-007 | gpresidentialsociety.com |
| Whoever extracts does not also decide. A doubt gets reported, never quietly overridden | An outside research model noticed its own quote pointed the opposite way, wrote that down, and kept the claim anyway. R-071 | the research graph |
Where AI is weak, and what I do about it
-
Plausible, wrong facts
It fills a gap with something that sounds right: a charity that does not exist, a footnote built from true parts.
What I do: an unknown is a question for me, and a quote is checked against its source or left empty · R-140 -
Drift across sessions
A wrong belief written into one note gets copied into the next, until it looks settled.
What I do: one current handoff, one copy of each fact, and anything about deploys checked against the live site · R-007 -
Numbers
Exact counts go out of date, and counts from memory are simply wrong.
What I do: count with a command, and round in copy · R-011 -
Over-polish
It writes like marketing: promises in my name, impressive numbers, claims that go stale.
What I do: no copy ships until I have ruled on it, block by block · handoff-003 -
“Done” that is not done
A passing script, a clean checker report or a working local copy stands in for the real thing.
What I do: use it: drag the slider, Tab through the page, load it live. And make sure the check can fail · R-050
In the research graph, the checks are code. Its rule is “No document, no edge”: a link between two reports goes in only if something published says it exists. Dozens of checks run on every report and every link, and a change to the data is not finished until they pass. A link that cites a homepage instead of the document is flagged. A link graded A must carry the exact words it was found from. A report cannot depend on itself, or on a report that is not in the graph. The checks are checked too: one once reduced to a constant that could never fail, and it was rewritten so that it can.R-045
The files this page is drawn from
-
The twenty rules, drawn from the inventory
-
How the work is divided between me and the AI, and the checking routine for each kind of change
-
Every miss, how it was caught, and my ruling on each
-
The rules for writing a handoff
-
The copy review this page went through
reviews/copy-review-003.md M1–M8 ·
-
Two case studies built this way