The comic went quiet in June. The pipeline is a home-built comic-book
generator: a story idea and a few reference photos go in, and a lettered,
multi-page book comes out, with a language model writing the script, a
diffusion model painting the panels and a Python director above both of them
deciding what each is allowed to do. It had been producing talky, over-written
books for weeks. Then I added two things I wanted for continuity — a mood
field per panel, so the painter would draw the right face, and a settings
registry, so a recurring place would look the same each time it appeared — and
the next book came back with 56 panels and almost no dialogue. Characters stood
in beautifully rendered rooms and said nothing.
I blamed the model first, because that is the cheap explanation. It was not the model: with the new fields in place, one writer produced a talking book zero times in four runs, a second once in ten, and the best of the three about one time in three. I blamed context next and doubled the window; a two-by-two of context length against schema made no difference at all. I blamed the brief and hand-wrote a perfect beat sheet with every line of dialogue implied; the best writer still came back mute in three of three. The only variable that moved the result was the shape of the JSON the writer was asked to fill.
Diagram: read it left to right — the writer answers a deliberately thin schema, and a second pass rebuilds everything the schema no longer asks for from the prose the writer did write.
The principle
A language model does not fill a schema for free. Every field you put in front of it is a demand it will meet, and it meets the demand out of whatever it would otherwise have spent on the parts you did not constrain. Dialogue was never a required field — it cannot be, honestly, since a panel may legitimately be silent — so it was the first thing to go. Add a field, lose a line.
Why it hid
The failure was invisible for a structural reason, not a careless one. The writer emits one JSON document against a typed schema, and the decode is grammar-constrained, so a malformed document cannot exist; a wrong one is rejected by validators and retried. That machinery is genuinely good, and it is exactly what hid the bug. A book with no dialogue is a valid book. Every one of the silent books passed validation on the first try, rendered cleanly, lettered cleanly and packaged into a PDF with a manifest. Nothing in the pipeline had an opinion about how much talking there should be, because talking is optional, and an optional field’s absence is not an error.
The second reason it hid is that the cost was paid in a different place from where it was charged. The fields I added were about faces and rooms. The thing that vanished was speech. No log line connects those, and the diff that introduced the fields touched a schema file, not the prompt, so the obvious “what changed” question pointed nowhere. Silence that validates is the worst kind of failure a typed contract can produce: the contract keeps its promise and the promise was the wrong one.
The same mechanism showed up again, later, from the other side. When I asked the outline pass to emit a list of side characters — marked required, in capitals, in the prompt — it came back empty book after book. Optional and nearly-optional fields are under-emitted by the same economy that makes demanded fields over-cost the prose. It is one behaviour with two faces.
The shape of the fix
The fix was to stop asking. The writer’s schema now carries story, dialogue and
layout hints and nothing else; every metadata field I wanted for continuity was
stripped out of it — two environment toggles, both defaulting to off, one
afternoon. Then, instead of demanding the metadata, the director rebuilds it
from the prose the writer did write. A second, cheaper pass reads the finished
script and relocates each panel’s emotion from the art description into a
mood field. Another reads the same prose and derives the settings registry:
which panels share a place, and one canonical description of that place
injected into every one of them so the painter draws it consistently. A third
does the same for recurring props, pinning a colour and a count so a boat stops
changing type between panels. A fourth resolves every named non-lead across the
book — “the ranger”, “her”, a surname — into one registered side character with
a text anchor, so a face that has no photo still holds.
The measurement after: the mixture-of-experts writer produced three talking books out of three, at the same context length and the same brief that had failed. The relocation pattern then became the rule for every cross-panel registry the pipeline has, because the outline under-emits all of them the same way. Nothing is demanded of the model that the prose can be read for afterwards.
The costed version, for a reader with their own schema. First, list every field the model is asked to fill and mark the ones a reader could derive from the free text with a second call; those are candidates to remove. Second, look for the field you care about most that is not required — for me it was dialogue, for a support bot it might be the actual answer — and measure its presence across a handful of runs before and after adding any new field. That is a count, and counts catch what validators cannot. Third, when a field is under-emitted despite a forceful prompt, treat it as evidence of the same economy, not as a prompting problem; move it out of the schema and derive it. The whole change here was four relocation functions and two toggles, and the pipeline has not produced a silent book since.
Where it stands
The four relocation passes are the mechanism now, and the diagram’s guided views walk them in order: the thin schema, the silence, why validation could not see it, and the rebuild from prose. The same pipeline has since rendered a fourteen-page, 56-panel book with 87 lines of dialogue, identity held across every panel, and its known failures are all in the painter, not the writer. What is not done is a wardrobe registry — a coat still vanishes between panels because clothing is the one recurring attribute nothing pins — and it will be built the same way, read out of the prose rather than asked for.
Never demand from a model what you can derive from what it already wrote.