Case Study 02 — How I Use AI

The easy things easier. The hard things harder.

4 minute read

Honesty means not lying. It does not mean volunteering everything you know.

The chatbot on this site answers from 2,119 documents spanning twenty years, not from a flattering summary of me. It is live, it is in beta, and this is the honest version — including the parts where it was wrong and the parts where I was worse.

Year
2025 — Present
Corpus
2,119 documents, twenty years
Built for
This site
Status
Live. Rewritten whenever it breaks.

First, the work had to exist somewhere a machine could read

It didn’t. Twenty years sat across a Google Drive, an iCloud account, a Gmail account going back to 2006, three Time Machine backups of machines I no longer own, an external drive, and a public tobacco-industry archive at UCSF. Finding the files was not the hard part. Every tool I wrote lied to me by succeeding. One script reported 2,253 empty files; a quoting error had broken every path containing a space, and 84% of the run had failed while the script exited successfully.

Figure 1 — Documents extracted, by source
Google Drive
1,091
iCloud Drive
678
MacBook Air backup
507
MacBook Pro backup
29
5,030
files discovered in Drive alone
2,509
duplicates collapsed
2,119
documents in the finished corpus
Plus 248 catalogued job codes across 18 clients, and 181 public UCSF documents that corroborate the Camel work independently of anything I hold.

A privacy scanner that is never questioned

The first scan flagged 284 files. Ninety were false alarms. It quarantined 116 for containing the tax identifier “EIN” — most of them contained the word protein, in an archive built from meat-industry work. A scanner that is never questioned deletes the evidence instead of the exposure. Both failures look identical in the log.

How an answer gets made

The evidence base decides what is true. The rules decide what is sayable. The voice layer decides how it sounds. Forcing a structured response means it fills a defined shape or fails visibly. A system that can fail to answer beats one that will always answer.

Figure 2 — How an answer gets made
The path a question takes through the system A question passes through an evidence base of 2,119 documents, a written rule set, and a voice layer of 251 lines drawn from his own writing, then through a forced structured output step that shapes the answer it gives back. The rules and the voice layer can block or reshape an answer, but neither one can add facts. Question or a whole JD Evidence base 2,119 documents 248 job codes Written rules what it may claim, and what it may not Voice layer 251 lines he wrote, from 455 candidates Structured output it cannot reply freehand Answer short, sourced The rules and the voice can stop an answer. Neither can invent one.
Provenance is not only for claims. The voice layer is 251 lines he actually wrote, kept from 455 candidates.

Then it started costing me interviews

Ask about a job I could plainly do and it would volunteer, unprompted, what my file did not cover. It flagged caution to people who had not asked for it. I had written the rules for an evidence archive and then pointed them at a buyer. Every factual restriction stayed exactly where it was; what changed was separating two things I had collapsed together — not asserting something, and announcing its absence.

SYSTEM_PROMPT.md excerpt — the never-say table
Never say thisBecause
“That’s a documented gap.”You just disqualified him, unprompted
“Where I’d flag caution…”Nobody asked you to flag caution
“Not claimable.”Internal vocabulary. Never say it out loud
“The evidence doesn’t show him at that altitude.”Not your call. That’s the hiring manager’s judgement

Then it was true, and it still didn’t sound like me

A dozen questions, a dozen answers with the same silhouette. The facts were right; the shape never moved. My own prompt said the shape is always the same, and the whole personality rested on twenty-six quotations of me against twenty years of my writing. Provenance is not only for claims. A sentence in my voice that I never said is a fabrication too.

Figure 3 — The voice sweep, end to end
455
candidate lines mined from my own material
203
rejected under adversarial review
251
shipped into the voice layer
Sources: a working knowledge base, a 9,925-word recording of me talking through my whole career, drafts I wrote about myself, a 114,000-word working document, my cover letter, a letter to a departing intern, thirty-six published comedy pieces and five published, bylined journalism clips.
Figure 4 — Why 203 lines were thrown out
Crossed a privacy line
79
Not actually my words
68
Could have been anyone
29
Smuggled in a claim
14
True, but not for this room
11
Could not be found in the source
2
Sixty-eight lines read exactly like me and were not mine — mostly an assistant’s summary of my position, or an answer given by someone I was interviewing twenty years ago. Two could not be located in any source file at all, which means they were assembled rather than found.
Forty-five percent did not survive. Sixty-eight lines sounded enough like me to pass the first test and belonged to someone else — an assistant’s summary, or a person I interviewed twenty years ago. Sounding right is not evidence.
The closing

The easy things got easier. I am still doing the hard part myself.

Every fact has to be sourced or flagged. All 203 rejected lines are kept with the reason attached, which is the only way to tell later whether the filter was right. It is still in beta. It still gets things wrong. The site says so.

Next case study
TruStage — You can’t say that
→
↑ Top