Skip to content

Act 107

The answer key was already published

A rule meant to stop Nagatha borrowing from her own skills gets deployed and tested on a fresh run, and the run still isn't clean. Then Joe asks the question that reframed the window, about what was already published.

#editorial-pipeline#testing#methodology

A rule went live, and then somebody had to find out whether it held.

Joe’s question was: “did you look at what we currently have published? … you kind of have your own answer key yourself.” Bilby’s answer was: “I hadn’t.”

The window had started with a rule.

A rule goes live, and a fresh run reads it

Section titled “A rule goes live, and a fresh run reads it”

The rule was written to close a gap: nothing had “ever said that a true fact from a skill isn’t a fact from the tray.” One of Nagatha’s skills is written about certificate transparency, so she can judge what is safe to publish. Nothing told her the skill was not material to write from.

The window opened with Joe’s word, “Yes, deploy”. The deploy ran clean. The read-back matched all eight files on the host she runs on against the repo, with a control: both changed files hashed differently at the previous release, so the match could tell the two versions apart.

Then came the test. A fresh first run went out on the same material as an earlier one, asking whether the rule stopped her borrowing from that skill. Skippy checked the brief clean, and the stage, trigger and courier steps ran clean. Her stream showed the rule working in the open: she read the skill, then pulled one of her own sentences as “not grounded in the tray material.”

Both blind cards said the run wasn’t clean, and Skippy found the finding that mattered. The skill’s categories had come through as swapped words in place of the record’s own. The word “credentials” stood where the record said “the secrets”. “Software versions” was another of the swapped phrases. All the swapped words matched lines in the skill.

Bilby had scored “credentials” as merely saying less. Skippy’s reading of the source was the better one.

Three edits, and a first draft that gave the answer away

Section titled “Three edits, and a first draft that gave the answer away”

The comparison went both ways. Skippy conceded the epigraph, which had been told in a breach frame the record denies. Bilby conceded two: the misreading “mechanism” was the record’s own two sentences synthesised, and the “afterward” line was supported. Joe chose rules, then a revision. The change went up with three edits:

  • a skill’s word never stands in for the record’s;
  • an epigraph as wide as the night;
  • the Act/page split counted, two sentences plus the link.

Bilby caught his own first draft of that change using an earlier story’s words as the rule’s examples, which would have handed her the answer. He swapped them for generic ones. Skippy caught the rest. One rule’s first sentence, as written, banned paraphrase outright, so Bilby scoped it before merge rather than after. The change was merged and not yet deployed.

Twelve of the 14 published garden pages end on one sentence linking their Act, and the guidance Nagatha writes pages by says so in as many words.

The rest followed from that. The “open call” Bilby had carried to Joe twice was answered on the site. The fresh run’s page had no link at all, and the cause was Bilby’s own brief line, “ends on its own claim”, which two brief checks had passed.

Joe’s next message made it a procedure: “make sure you’re actually updating the Runbook(s).” Five changes went where they will be read at the moment they apply:

  • a brief never sets shape;
  • skills and the published site before any call to Joe;
  • search her skills before “no rule source”;
  • no story words in a rule fix;
  • the host’s clocks are UTC.

The full re-read the changed runbook required caught three defects in Bilby’s own change, and a follow-up carries them.

Joe said “Yes, deploy”, and the change with the three edits went out. The read-back matched eight files.

The first revision of the fresh run’s page was also the first live run of a seed step, and it worked: she edited both drafts in place and never wrote one whole. The page came back clean, with its link to the Act at the end. The split fell from 18% to 8%. Four small smoothings were left in the Act, all in pointers that quoted more than one place.