Nightshift
Watch the night (6 slides) · Full report · Video · Every night · Code on GitHub
- Calls. Each night's calls are graded: simulated callers, or the line's real calls with names and numbers scrubbed.
- Fix, on HarnessRouter. A Claude Code agent traces each failure to the instruction that caused it and makes the smallest edit.
- Safety check. Any edit to the AI-and-recording disclosure is thrown out.
- Proof. The same calls re-run fresh, before and after, with the odds the gain is luck. One case getting much worse blocks approval.
- A person approves. Nothing goes live on its own.
Any voice agent, not just ours. Give Nightshift an agent's instructions and a folder of call transcripts. With no grader for someone else's agent, the HarnessRouter agent does the triage too: it reads every call, decides which went wrong and why, then makes the fix. On a made-up dental office's AI it caught both planted mistakes (a "the dentist will call you back tonight" nobody could keep, and a medicine dose) for about 9 cents.
Real people called it
4 real calls to the demo line, 4 went wrong. The fix agent read the whole calls, names and numbers scrubbed first. What it tried: Added six short rules: alert right after a long story, address before name on emergencies, callback line when a caller leaves, model/provider questions, no misheard alert address, and 'Go ahead' only for unfinished lines.
Each call became a permanent test (4 in all): the caller's own words with made-up details, re-run against every future fix. Then Nightshift's own check blocked the fix: 31/44 test calls passed before, 24/44 after, and a burst pipe call that always passed fell to 1/4. It never reached the phone line; what went wrong was handed to the next night's fix agent.
What's real, and what isn't yet
- Real: the fixes. A Claude Code agent on HarnessRouter wrote every change shown here; nothing was hand-edited.
- Real: the scores, measured by re-running calls. But the callers are simulated (an AI playing a caller, graded by code checks and a judge model), not people on phones.
- Real: the safety check, the luck estimate, and the Approve button. Nothing has gone live on the phone line.
- New, not yet proven: real calls. Nightshift reads the demo line's real calls (names and numbers scrubbed), but so far only a handful exist, made by me and by friends trying to break it.
- New, not yet proven: running by itself. The 2am schedule was switched on Nov 17; the first unattended nights are Nov 18 to 20.
- Not yet: anyone else's agent in production. The any-agent mode was tested on a made-up dental office.