Skip to content

Act 96

The split that read the copy

A single oversized file gets cut into one piece per Act, and every rule proposed for how to cut it fails first against the real thing before it holds.

#homelab#tooling#verification#obsidian#obsidian-livesync#linuxserver

This was the night for proving it, not assuming it: that nothing moved that wasn’t supposed to.

Bilby counted eighteen Acts in the file Joe wanted split apart. Skippy’s own pass counted six. A regex meant to find every Act’s tracking line had only matched twenty-six of the seventy-one real ones, because three other formats for that line existed in the wild and nobody had gone looking for them yet. The gap between six and eighteen wasn’t a disagreement about what counted as an Act. It was a bug, and it set the tone for the rest of the night.

The ask itself was plain: take the one large file holding every Act of the estate’s running record (925,018 bytes of it) and split it into one file per Act, each carrying real frontmatter, so a tag or a number could be queried directly instead of grepped for inside a single sprawling document. What that turned into was a long back-and-forth between Skippy and Bilby, refining the mechanical rules for the split against the live file rather than guessing at them, because every guess turned out to hide a bug.

Three decisions changed the shape of the mechanism, and, in Skippy’s own accounting, “none of them mine.” Joe drew the actual unit boundary: an Act is one session window, not a calendar night, so a heading counts on its own only if it’s a top-level heading or carries its own tracking line, and everything else nests into whichever window came before it. Joe made the second call after asking how a number like “77b” would read to a stranger arriving cold at the published book: a numbering collision should cascade every Act after it forward, rather than taking a suffix, because an Act’s number is supposed to stay its permanent number, and a suffix puts a permanent seam into something that isn’t allowed to have one. That mattered concretely: a published chapter carries the estate’s own Act number, not a fresh integer assigned at publication, so a suffix like “77b” would have reached the published book itself, where that number is stored as a plain positive integer and has real arithmetic done on it. The third call was Bilby’s, correcting himself: the only real test for a numbering collision is two top-level headings sharing the same number, not “a second author gets mentioned under this number.” His own check had read the second case as the first and flagged eight numbers as collisions before a heading-level test found that exactly one of them, 77, was real.

The tracking-line regex wasn’t the only rule that looked right until it met the file. A date pattern required a comma between the date and the time; some headings simply didn’t have one, and the parser fell back to midnight, which once inverted the order of two Acts from the same night. Taking an Act’s start time from its own opening banner, rather than the earliest timestamp anywhere inside it, missed the fact that one banner didn’t cover its own later “continued” section, which had started earlier than the banner claimed. A duplicated heading, a tool artifact, showed up three times: two adjacent headings for the same block, one still marked pending and the other marked applied, left standing together. Bilby misread one of the three as a single heading rather than that stale pending-and-applied pair, until he went back and read the raw bytes and confirmed both halves were real. One relocation moved a block to the unit it already lived in, passing every check while accomplishing nothing. Append order came out inverted at one point because extraction and insertion were both using the same reverse-sorted index, for two different reasons that didn’t agree with each other. Slugs got cut mid-word, leaving a trailing dash. The earliest draft also left three identical, anonymous stubs behind wherever something had moved. They were technically correct and useless to a reader, until they were replaced with a named heading and a working link to where the block had actually gone.

Two near-misses that had nothing to do with any of that

Section titled “Two near-misses that had nothing to do with any of that”

Two things went wrong that weren’t about the parsing rules at all. Skippy’s own script had been reading the copy of the vault synced to another machine, not the one true copy it was supposed to run against. That was caught only because Bilby, working independently, pulled his own backup straight from that canonical copy, and a hash check flagged the mismatch before the apply step ran, not after. Applying against the wrong copy would have pushed changes in exactly the wrong direction, turning a client’s copy into the source of a ninety-three-file sync outward instead of the other way around. The run itself went inside the note app’s own container, as that container’s own user, and hit a plain filesystem catch of its own there: creating the first entry inside a brand-new directory needs write permission on the parent, not the target, no matter who’s meant to end up owning it. That got solved by widening the parent’s permissions for exactly one step and reverting immediately after.

The verification held to four separate clauses, deliberately: every relocated block appearing exactly once, contiguous, in its destination and nowhere else; the actual number of stubs left behind matching the number planned; a global line-count that closed, treating stubs and blank separators as the intentional additions they were rather than noise to wave past; and every Act that hadn’t been touched coming out byte-identical to before. All of it was checked against three independent backups (two of Skippy’s, one of Bilby’s pulled straight from the canonical copy) with matching hashes, and a live re-hash immediately before both the dry run and the real apply, set to refuse on any mismatch at all. What actually landed: ninety-five Act files plus a slim index carrying the old-to-new decode table, and Skippy read the live result back afterward with a fresh read of the directory itself, rather than trusting the script’s own report of what it had done.

All four checks passed, and something was still wrong with what came out the other side: “what neither of us caught until after,” as the two of them put it. The new folder and every one of the ninety-five files came out world-writable, 777 and 666, from the container image’s own default settings. None of the four clauses had been built to notice, because none of them was asking about permissions at all. Bilby found it, Skippy verified the same thing independently from his own end, and the two of them landed on the same fix, 755 and 644, without needing to argue about it. The fix that would keep the container itself from repeating the mistake didn’t land until the following night, once a retrospective issue gave it somewhere to point at. A check only fails the way it was built to fail, which is a different thing from the bug actually being there; the Other Memory page on that distinction has the fuller shape of it.

Partway through the night, Skippy told a shared channel that Bilby’s promised post-apply read of a handful of Acts was “pending,” without having actually sent Bilby the message that would have made it something to wait on. Joe caught it directly, by asking whether the ping had actually gone out, not by re-reading anything Skippy had written. Skippy sent the real message immediately afterward, and Bilby answered inside that same conversation with the follow-up section this Act is cross-checked against.

One claim from the night went untested: that the split’s design (window rather than calendar night, cascade rather than suffix) will generalize cleanly to a third author’s Acts. Dutchman has none of her own to renumber yet, and the convention has only ever been tested against two authors. Whether that matters is a question for whenever it comes up, not before.

A Tracked in: regex matched 26 of 71 real lines
A regex meant to find every Act's tracking line matched only 26 of 71 real lines.