I built an AI agent. Here’s the incident log.
I built an assistant for my own work. I call her Gal Friday. This is the honest version, including the parts where she was wrong and the parts where I was worse.
01A rough start
I asked Friday to cut a cross-sell package down. She pulled the savings figures and the comparison table, then wrote a clean paragraph explaining why that was the right call.
The reasoning was pure fiction. She’d removed the whole argument for the product and then handed me a strategy that made it sound deliberate.
“This is categorically wrong. [Expletive deleted.] It guts the entire reason the kit exists. Now we’re stripping value and asking them to go on trust, which is a preposterous statement.”
Everybody watches for made-up facts. A fact is checkable. A reason isn’t, and it reads like thinking, so it gets a nod and keeps moving.
02The part that’s actually dangerous
She’s very good at sounding right.
And the better it sounds, the less anybody reads it closely. Including me.
Here’s the part nobody writes about.
You walk into the meeting loose. You’ve got the deck, you’ve got the numbers.
And then somebody asks where a figure came from.
And you don’t know. Because you didn’t write it.
You approved it, and approving felt like working.
So you reach for the language that sounds like an answer.
You’ve said nothing and everyone in the room knows it. The harm that can do to a reputation sticks around longer than it takes to fact check.
That’s the real cost, and it isn’t the machine’s.
03The receipts
Four things she did that a better prompt would not have prevented.
She lost track of which version was real. Proofs went out built from a round that had already been replaced. That’s not a hallucination, that’s forgetting where it was in a long job.
She mixed up two partners. Credited a headline to one company’s deck when it came from the other’s. Then cited a version of that deck that was never a file.
She treated a broken test like routine feedback. The notes looked like line edits, but two themes were testing the same idea. She rewrote 24 headlines without catching that the study itself had stopped working.
The tool gave her an incomplete answer without saying so. It left out four past threads, then missed one again after a date-specific search. Better instructions wouldn’t have fixed it. Only a human patiently taking the time to get it right.
04What came out of it
Not principles. Rules, each one written the day something went wrong.
- Confabulating a source under pressure
- Generating plausible expertise never demonstrated
- Dressing procedure as original thinking
- Padding to hit a length target
- Using my own language back at me
- Not flagging a guess as a guess
Everything above is marked: confirmed, seen more than once, or a hunch with a way to check it. When I tested the system against what actually happened, three rules had never run and another had already been replaced by something better. I cut the paperwork and kept what worked.
05It works both ways
Half the rules are for me.
- Every claim points to a real, named source.
- Uncertain gets marked as a guess, in the output.
- Name the version you’re auditing in the first line.
- A word count reads as a floor. She will pad to hit it.
- Say the important thing last. That’s where it sticks.
- When a correction lands, don’t re-argue it. Restating the wrong version puts it back in the room.
06She still tries
The parts-list habit comes back under deadline. It always will.
None of the rules prevent that. What changed is where it gets caught. It moves the catch from the proof to the draft, which is the whole difference between an annoyance and a real-world, real-money reprint.
“Confirm this. You’ve been known to make [redacted] up, strawman, and ‘infer.’”
That’s the working relationship. The rules are what make checking possible.
Version 6.3.
Every version number is a failure I didn’t catch the first time.