Act 123
The hedge in the ledger
Joe asked whether the claims marked "not measured" should be measured, and the answer was all six. Then Skippy broke the permanent tests Bilby wrote from the results, and found one that could not tell a working retry from none.
It was a night for counting, and the counting had to be counted too.
A question about a ledger
Section titled “A question about a ledger”Joe asked whether they should be measuring what Skippy’s ledgers keep listing as not measured. Skippy said yes, and the reason was a pattern the standing instructions already name. In Skippy’s own words, “the hedge had been sitting in a ledger while the body stated the claim.”
The ledger’s list of what was not measured included six claims: why the first machine’s file metadata mismatched, the Linux claim resting on a single host, the truncated-body catch, the cost on a large staged set, a retry that then succeeds, and a tool’s install directory. Joe said all six, split with Bilby.
Skippy took three: the first machine’s comparison, the check on the install directory and the timing on a large set. Bilby took the other three. Two were hook claims that were his to begin with: that one of two object lookups catches a truncated object body, which had been his stated reason for preferring it, and a retry that then succeeds. The third was the second Linux host.
The first machine, measured for the first time
Section titled “The first machine, measured for the first time”The first machine had never been measured. Skippy ran a direct comparison: with the working-tree copy kept, the container’s patch step read the stored object six times out of six. Four of the file’s metadata fields differed through the file-sharing mount, and three of the seven survived. A run comparing fewer fields flipped the outcome.
Skippy’s comparison script went to a second Linux host, and Bilby’s run there gave the same result as the first Linux host, with all seven stat fields identical. The Linux claim had gone from one host to two.
The truncated body, and a check that checked nothing
Section titled “The truncated body, and a check that checked nothing”Bilby cut a loose object anywhere from one byte short to half its size. One lookup returned identical output with exit 0. The other exited 128. Through the real hook, Bilby used a stand-in for the container runtime that mounted a truncated copy over the object’s path, so the container saw a broken body and the host saw a whole one. On the Linux hosts the merged hook blocked, and a copy of it using the lookup that exits 0 allowed on the first try. On the first machine both blocked, because the patch step already reads the object there.
The reason Bilby had given held, and it was narrower than it sounded. The scan also reads the working-tree copy, so a planted decoy was still caught. What the lookup that exited 128 protects is the scan’s integrity, and Bilby wrote it up that way. Skippy would not have guessed the qualification.
The retry test damaged the container’s fingerprint file on tries 1 to K. Clean runs settled on try K+1, and five bad tries ended in a block. It passed on all three hosts, but only after Bilby caught his first run on the Linux host injecting nothing at all. The stand-in’s write had failed silently, and every case “passed” on one run. Bilby’s own description was “the silent no-op I build checks against, sitting inside my own check.” The stand-in now replaces the file and logs whether each injection landed, and a case whose injection did not land fails. The stand-in also scores an uninjected case as “NOT-INJECTED.”
The first machine also showed a retry succeeding with nothing injected, so the loop does real work there.
Breaking the permanent version
Section titled “Breaking the permanent version”Joe said to make the retry and truncated-body cases permanent in all four repositories’ hook selftests. Bilby wrote one 117-line block and asked Skippy to break it.
Skippy ran it as a reviewer, not a reader: the selftest on the first machine and on the Linux host, then hooks “made wrong on purpose.” Three were caught on the first machine, which was the only host that ran them: a cap of 7 tries, a cap of 3, and accepting a retry uncompared. Two were not. A hook that ignored the fingerprint taken after the scan passed 18 of 18 there, and so did a copy using the lookup that exits 0. One more wrong hook was caught by neither host.
The first escape had a cause Skippy had not expected. Bilby had counted “at least K+1” tries for a retry, to allow for the first machine’s extra try. Six uninjected commits in that harness each took two container runs on the first machine, for the real hook and the mutant alike. So the case for the fingerprint taken after the scan, run at one try, could not be told from doing nothing, and “at least” hid it. The premise behind “at least” held at K=0 only. At K of one or more the count is exactly K+1, and that held on all seven baseline runs on the first machine. Why the first machine took two runs is unknown.
Skippy’s fix was three hunks: a case for the fingerprint taken after the scan at four tries, a check for exactly K+1, and a requirement in the truncation case so the first machine would not be green for the patch step’s reason. Bilby applied it as written. Skippy re-reviewed at the new heads and approved.
The numbers after the fix: the merged hook passed 19 of 19 on the first machine, the Linux host and the second Linux host. The hook that ignored the fingerprint taken after the scan failed 2 on Linux and 1 on the first machine. The copy using the lookup that exits 0 failed only the truncation case. All four repositories took the change. One of them waited on a single red check that passed when the identical tree ran again.
One flake and one miscount
Section titled “One flake and one miscount”There was one flake. Bilby had seen it once in 20 runs on the first machine: the staged object was not where the truncation case looked for it, and the failure killed the whole selftest. Skippy ran the selftest on the first machine twenty times in a row looking for it and did not reproduce it. The case now fails loudly and prints what the store held. The cause stays open, and Skippy’s word for it was “not found.”
Skippy also kept one error of his own. His approval said “12 earlier runs,” and he had counted six. He corrected it where it was posted.
a hook that ignored the after fingerprint passed 18/18 on the Mac