It took five attempts to scaffold this site

I have a generator. You give it a vision file and it produces a whole product repository — app, database, deploy scripts, the operational furniture. This site is the first thing it made since a round of hardening work.

I could have handed that run to an agent. I didn't, and the reason is the whole point of this post: an agent that succeeds teaches you nothing about your tools, and an agent that fails teaches you only that it failed. I wanted the middle part. So I ran it myself, at a terminal, watching.

It took five attempts. Six distinct failure modes, and not one of them was documented anywhere.

That is a better first post than a launch announcement, because most of them are not really about my generator. They are shapes you have in your own tooling right now.

The isolation that broke the tools it isolated

Before any attempt could run at all, the run died on a hardening flag.

To keep generated code from reading my dotfiles and credentials, trusted tools were invoked with HOME pointed at /dev/null. The intent is right: a tool should not see a home directory it has no business in. But /dev/null is not a directory, and modern build tools want to initialise a cache inside HOME before they do anything else. Every one of them aborted.

The fix was not to weaken the isolation. It was to make it real: a fresh, empty, writable temp directory, discarded when the process exits. The tool still cannot see anything of mine, because there is nothing there — but it is a genuine directory, so it works.

The general shape: hardening that has never been exercised is a hypothesis, not a property. This one had been shipped, reviewed, and believed for a while. It broke the first time something actually ran through it.

The rule nobody wrote down

Next attempt, the generator rejected one of my arguments. The error named the rule — the value had to be a dotted identifier — but did not print the pattern it was checking. newsletter_subscribed failed. newsletter.subscribed passes. I got there by guessing, and I could have got there by reading the source, which is the same admission with extra steps.

An error message that names a rule without showing it makes every caller reconstruct the rule from the outside. It now prints the regex, a valid example, and the offending value.

The refusal I could not act on

Next attempt: a cached copy of a vendored dependency was rejected as untrusted, and the run stopped.

The validator was right to stop. The cache entry predated a change that made an extra metadata file mandatory, so the directory no longer matched what a trustworthy cache looks like, and the validator deliberately refuses to overwrite anything it cannot vouch for. That is the correct instinct. Silently repairing a cache you suspect has been tampered with is exactly how you launder a bad artifact into a build.

The problem was that "this is an old format" and "this may have been tampered with" came out of the same code path, wearing the same word, with no path in the message and no remediation. Those two conditions want very different reactions from a human, and the human is the only one who can tell them apart at 8pm.

Now the message distinguishes them: a cache whose contents still match the pin but whose layout is out of date is reported as stale format, not tampering, and either way the message names the exact directory to move aside.

The general shape: a refusal you cannot act on is an outage with better manners. If your code can detect a condition precisely enough to stop, it can usually describe it precisely enough to fix.

The interesting one: debris that looks like new bugs

Here is the sequence I actually lived through, from my notes that night:

Read the right-hand column again. Attempts 3 and 4 are not new bugs. They are attempt 2's corpse, presented as two fresh, unrelated, plausible-looking failures.

This happened because a failure partway through left state in two places, and the escape hatch — --force — cleaned up only one of them. The other was a port registry that gained an entry per service before the fallible work ran, and which had an allocate command and no release command. Nothing in the system could give those ports back.

Every retry surfaced a different error than the last. That is the signature, and it is worth learning to recognise, because it costs you far more than the underlying bug. A tool that fails the same way twice is a bug report. A tool that fails a new way each time is a mystery, and mysteries are where the hours go — you start theorising about a system that is actually just handing you its own litter.

The test: kill your tool halfway through and run it again with the same arguments. If the second error differs from the first, your failure path is not idempotent, and everyone who uses it — including future you at midnight, and including any agent you point at it — will burn time debugging the wreckage instead of the crash.

The three questions worth asking of anything that mutates more than one thing:

1. Where does it write? Not where does it mean to write. All of them: the output directory, the registry, the lockfile, the remote, the cache.
2. Does the cleanup path cover every one of those, or just the obvious one?
3. Does every acquire have a release? An allocate with no deallocate is a leak with a schema.

The fix I made is the boring one and it is available to almost any tool. Nothing lands in its final location until everything has succeeded: the whole tree is built in a temporary sibling directory and moved into place with a single rename, and the port registry is only written after generation, validation, and the first commit have all passed. A run that fails now leaves nothing behind, so a blind retry just works. There is also a release-ports command for the allocations older runs orphaned — because "I'll add the deallocator later" is how the first one happened.

Staging and publishing atomically beats rolling back, because a rollback is more code that runs at exactly the moment things are already going wrong.

Two things a successful run didn't do

The fifth attempt worked. It printed Git initialized: yes, which was true, and useless: it had run git init and made no commit, so roughly 170 generated files sat untracked with no baseline to diff against and nothing to revert to. A stray clean would have erased the entire run.

It also printed a warning that one integration had been deferred, and exited zero. Attended, that is fine — I read it. Unattended, that is a success report with a missing piece inside it.

The general shape: exit 0 means "no error was raised," which is a much smaller claim than "the thing you wanted exists." And a warning that nobody is required to read is a failure that hasn't happened yet.

The finding underneath all of it

That is five attempts and six failure modes, and it would have been a decent evening. Then I looked at what the successful run had actually produced, and found something worse than any of the crashes.

The generated repo shipped a backup monitor that emails on failure but publishes no machine-readable status file — which is the thing the monitoring job actually reads. With no such file, a failed backup in a brand-new product would alert nobody.

I knew about that gap. I had fixed it hours earlier, in FeatureJet, where it was found. What I had not done was carry the fix back into the template FeatureJet was generated from.

So the template was quietly behind the product that exercises it, and every product generated after that point would have been born with a bug I had already fixed — inheriting the old, broken version, silently, forever, with no error anywhere.

This is the one I would most like you to take away, because it does not require a code generator to happen to you. You have this if you have a project template, a starter repo, a cookiecutter, a shared base image, a scaffold script, a service that got copy-pasted into three other services, or a vendored library you patched locally. Fixes flow downstream by default and never upstream. You fix the instance because the instance is what hurt; the source keeps producing the old version because nothing hurt there.

And the damage compounds in the worst possible direction: it is invisible at the moment it happens, and it grows with every new instance. The bug is not in the thing you are looking at. It is in things that do not exist yet.

The part that stings: my factory already had a drift-report command that compares generated products against their template. It existed the whole time. Nobody had ever run it. Detection that is not in the path of the work is decoration — so the backport landed with a regression test that fails if the gap reopens, which is the version that does not depend on anyone remembering.

Why I did this by hand

I orchestrate agents to build these products, and I will keep saying so plainly rather than letting it be a reveal later. The fixes above were written that same night with agents; the runbook was too.

But a first run after a change is a diagnostic, not a chore, and its value is in the texture of the failures. An agent would have reported "create-product failed" and, worse, would probably have retried — thrashing straight through that error table, generating three theories about three unrelated bugs that were all the same bug. The runbook now says, in as many words, not to let it.

Automate the run you understand. Watch the one you don't. The five failures above are now six paragraphs in a document that did not exist yesterday, which is the actual deliverable — not the site.

The two questions

If you take nothing else:

Does your tool fail the same way twice? Interrupt it halfway and run it again. A different error the second time means you are shipping debris, and somebody is going to debug it as a new bug.

When you fix a bug in something generated from a template, what carries the fix back to the template? If the answer is "I'll remember," then every future copy inherits the bug, you will not find out from an error, and you will not find out for months.