11/19 2026
504
Series: "Writing High-Quality Code with AI: AI Werewolf Development Record." This article corresponds to version v1.13.0 → v1.13.1 | Date: 2026-10-23
Preface
The previous eight articles were all about "building features." This final installment addresses something more fundamental: how do you know it's correct?
My verification method had always been "play a round and see." A round takes eight minutes, so I couldn't verify many times a day, and I often failed to hit the branch where problems occurred. Then I had AI modify a spectate interface, and ten minutes after going live, the smoke script turned red—it caught the problem before I did.
That was the moment I truly understood the value of testing: it's not a process burden; it's the only means of making AI's output trustworthy.
Problem: AI Won't Doubt Itself for You
AI has a characteristic when writing tests: it writes "tests that pass."
If you ask it to write tests, it will infer assertions from the implementation—if the implementation calculates wrong, the tests calculate wrong too, everything goes green, and you feel more reassured. This is the most dangerous state: a false sense of security.
So testing is not "verifying the implementation"; it's "writing down in advance what you believe is correct."
Write the behavior you expect first, then let AI implement. Otherwise, tests become a mirror of the implementation.

Figure: Three gates. Unit tests are millisecond-level, covering only a single function; contract tests are second-level, governing interface shape and permissions; online smoke tests are minute-level, running in the real environment. The further you go, the slower it gets—and the closer it gets to what users will actually encounter.
Why "Changing the Contract Makes Old Tests Red" Is a Good Thing
When that change went live, two old tests plus one smoke assertion turned red simultaneously.
At first I was a bit annoyed, but then I understood: what turned red was exactly the behavior that had just been changed. If the contract changed and no test turned red, that would be terrifying—it would mean nothing is guarding this contract.
Testing is the ledger of changes. You make an entry, it records it, forcing you to confirm "this entry is intentional."
Solution: Three Gates. Gate One: Contract Testing (the most cost-effective kind)
Before writing tests, first think "what's the worst way this feature could break," and the tests will have direction. The risk of a tracking interface isn't "recording one less event," but "recording what shouldn't be recorded": event names must be narrow, identities must be irreversible, and summaries must require login to view.
it('unknown event returns 400', async () => {
expect((awaitpost('/api/track', { event: 'whatever' })).status).toBe(400);
});
it('all four known events are accepted', async () => {
for (const e of ['share_click', 'landing', 'login', 'invite_bound']) {
expect((awaitpost('/api/track', { event: e })).status).toBe(200);
}
});
it('summary interface returns 401 when not logged in', async () => {
expect((awaitget('/api/track/summary')).status).toBe(401);
});
Contract testing has very low cost (no need to spin up a real database—just use fake objects) and high benefit: once the interface shape is pinned down, AI won't dare to casually change the response body when modifying the implementation.
Gate Two: Online Smoke Testing
The essence of a smoke script is: turn acceptance criteria into assertions that will turn red, and run them in the real environment.
This project has 24 items, covering game start, speaking, voting, settlement, spectating, sharing, and tracking. Here's one example:
await check('spectating requires no credentials, but undisclosed identities are not sent', async () => {
const { status, body } = await json(`/api/games/${gameId}/spectate?since=0&client=smoke`);
assert(status === 200, `HTTP ${status}`);
const finished = Boolean(body.snapshot.winner) || body.snapshot.phase === 'over';
if (!finished) {
const leaked = body.snapshot.players.filter((s) => s.role !== null);
assert(leaked.length === 0, `leaked ${leaked.length} identities during ongoing game`);
assert((body.snapshot.timeline ?? []).length === 0, 'should not send replay timeline during ongoing game');
}
});
Three details worth learning: when an assertion fails, point out "who leaked," not just "failed"; distinguish game states (ongoing / finished), since correct behavior differs between the two; return a human-readable summary so you can see at a glance what was verified this time. The common goal is: failure information must directly point to the cause.
Gate Three: Repository Hygiene npm run repo:check # Check whether junk files were committed, or ignored files snuck into the repository
What it blocks is debug screenshots, temporary scripts, and local configs (containing keys) being committed. Sounds like a small thing, but once a key enters git history, it's very troublesome to deal with.
Case Study: That "Contract Change"—What Exactly Changed
Background: The spectate interface is auth-free (spectating is a product selling point—it can't require login), but it originally sent full identities—6 seats, and who's the werewolf is obvious at a glance. Players could click "Spectate" during the game, paste the link into another window, and make decisions while watching everyone's identities.
The fix: spectating only discloses identities that have "already been revealed."
spectateSnapshot(): SpectateSnapshot {
const finished = this.winner !== null || this.aborted !== null || this.phase === 'over';
const seats = this.players.map((p) => {
const revealed = finished || p.revealRole; // exile reveal / idiot reveal / elder punished
return { ...p, role: revealed ? p.role : null };
});
return { players: seats, timeline: finished ? this.timeline : [] };
}
The timeline entry is the easiest to miss. It carries each night's "who was killed, whether the witch saved them, who the guard protected"—not rendered in the UI, but readable via packet capture, which is enough to infer identities. "The UI doesn't display it" doesn't equal "no leak."
The cost of this change: two old tests turned red (they were locking in exactly "full disclosure"), and one online smoke test also turned red. Plus 6 new tests were added.
# Result
Test Files 35 passed (35)
Tests 324 passed (324)
This process of three reds and two greens is "the ledger of changes" at work.
Effect: After the change, I dare to run it; wherever it's red is where to look. Gate | Scale | Time per Run | What It Catches. Unit + Contract Testing | 35 files / 324 cases | ~20 seconds | Rules, interface shape, permissions, prompt defenses. Online Smoke Testing | 24 items | ~2 minutes | Real-environment main path (game start→speaking→voting→settlement→spectating). Repository Hygiene | 1 item | Seconds | Debug screenshots, temporary scripts, local configs sneaking into the repository.
The most illustrative effect was that spectate change: ten minutes after going live, smoke turned red first—before I noticed the problem, it had already pointed out "leaked 6 identities during ongoing game." After fixing, the same interface:
Since then my habit when submitting changes has changed: no longer "hope nothing broke," but "run it; wherever it's red is where I should look."
Content Summary
1. The value of testing isn't coverage; it's "the ledger of changes": each entry is recorded, forcing you to confirm it's intentional.
2. Write expectations first, then let AI implement. Otherwise, tests become a mirror of the implementation, and all green is meaningless.
3. Contract testing has the best cost-effectiveness: low cost, blocks "shape drift," especially suitable for projects where AI frequently makes changes.
4. Smoke testing must run in the real environment, and failure information must directly point to the cause (who leaked, which step).
5. "The UI doesn't display it" doesn't equal "no leak." Auth-free interfaces must only send public information; this must be written into design constraints.
6. Is it transferable? Completely transferable. Even writing just three assertions—one for core rules, one for interface shape, one for permissions—is far better than zero tests.
Series Wrap-Up: Human-Machine Division of Labor
After nine articles, the pattern is very clear: humans handle "judgment," AI handles "execution." Judgment includes three things—where the boundaries are, what counts as correct, and when to switch approaches.
For requirements, humans set boundaries and write acceptance criteria; AI fills in details and writes documentation. For architecture, humans judge "who owns the truth"; AI moves implementations and writes glue. For acceptance, humans decide "what counts as correct"; AI writes tests and runs smoke tests.
The stronger AI becomes, the more important these three things are—because the more it can write for you, the greater the cost of one wrong judgment.