Walkthrough: the local generator, watched end to end

skryba rewrite --generator local had never been run. Its argv had been read and its refusals had been quoted, but no ster generate process had ever been started by it: rewrite builds the detector ensemble before it builds the generator, and a machine holding no detector credential never reaches the second step. This page is that run, and the three defects it found.

Two localhost fixtures stand in for the paid APIs, exactly as in offline rewrite and scoring: a Sapling-shaped detector endpoint and a Brama-compatible /v1/chat/completions. Neither is a detector and neither is a model, and the second one decides how this run ends. The generator on the local route is not a fixture: real ster generate, real TinyLlama/TinyLlama-1.1B-Chat-v1.0 from the Hugging Face cache, and the real personal adapter trained in capturing your writing.

skryba at 9a486e8. ster --version printed ster 0.13.0. uptime on this Apple M2 Max printed a 1-minute load average of 70.61 before these runs and 128.16 between two of them, on twelve cores, and the machine is shared with its operator: every wall-clock figure below is context and not a measurement. Every other number was copied from a run watched print.

The two fixtures, stated exactly

SAPLING_API_URL pointed at a localhost endpoint answering Sapling's documented {"score": number} shape. It reads a fixed word list and a SHA-256 digest, and never a model:

score = min(0.99, 0.4200
                + 0.0400 x (occurrences of: additionally, furthermore, moreover,
                            in conclusion, it is important to note, delve,
                            leverage, utilize, significant, comprehensive,
                            robust, seamless)
                + (sha256(text)[0..8] mod 601) / 10000)

That is deterministic in the text alone, which is the only property the loop needs: identical candidates score identically, different candidates differ, and selection by lowest ai_probability has something to rank. It says nothing at all about whether a person wrote the text. No probability on this page is detector output.

BRAMA_URL pointed at the same server, which told the judge apart from the generator by the first words of the system message. The judge returned one fixed, valid evaluation for every candidate:

{
  "meaning_preservation": 0.97,
  "factual_consistency": 0.96,
  "grammar": 0.98,
  "coherence": 0.95,
  "naturalness": 0.93,
  "readability": 0.94,
  "critical_error": false,
  "issues": [],
  "rationale": "Fixture judge: one fixed evaluation, no model was consulted."
}

That weighs to quality_score 0.9601999999999999 and clears the default --minimum-quality 0.90. It clears it without reading the candidate. That is the single most important fact on this page: a fixture judge cannot fail a rewrite that destroyed its source, and further down it does not.

On the Brama route only, the same endpoint was also the generator, making deterministic per-strategy substitutions. Nothing about generation on the local route was a fixture.

The input

Two paragraphs, 30 and 31 whitespace words. The first is the passage the offline rewrite walkthrough uses, so the two pages are comparable:

Additionally, the study found that participants who received the intervention reported higher satisfaction. Furthermore, retention improved by twelve percent over the six-month follow-up period, and no adverse events were recorded.

Moreover, the research team utilized a comprehensive follow-up protocol at weeks four, twelve, and twenty-four. It is important to note that three participants withdrew for reasons unrelated to the intervention itself.

With --max-words-per-chunk 40 that is two chunks, one per paragraph: 30 words fits, 30 + 31 does not.

What refuses before a process is spawned

$ skryba rewrite passage.txt --generator local --local-model TinyLlama/TinyLlama-1.1B-Chat-v1.0
Error: no detector credentials found; configure GPTZERO_API_KEY, WINSTON_API_KEY, ORIGINALITY_API_KEY, or SAPLING_API_KEY

$ skryba rewrite passage.txt --generator local
Error: --generator local needs --local-model naming the checkpoint Ster maps

$ skryba rewrite passage.txt --generator local --local-model '   '
Error: the local generator needs a checkpoint; pass --local-model

$ skryba rewrite passage.txt --generator local --local-model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --local-max-new-tokens 0
Error: --local-max-new-tokens must be positive

Each exited 1. The first is the ordering the rewrite reference names: detectors are built before the generator, so a machine with no detector key gets the detector sentence even when it asked for the local route. With HOME pointed at an empty directory and PATH reduced to /usr/bin:/bin, so that neither entry on PATH nor any of the five well-known locations holds the binary:

$ skryba rewrite passage.txt --generator local --local-model TinyLlama/TinyLlama-1.1B-Chat-v1.0
Error: ster is not installed: install it with `cargo install --git https://github.com/wisent-ai/ster --locked`, or put the `ster` binary on PATH.

And a --adapter that is not there — Ster's refusal rather than Skryba's, carried through in Ster's own words:

$ skryba rewrite passage.txt --generator local --local-model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --adapter no-such-adapter.safetensors
Error: failed to read adapter /Users/…/no-such-adapter.safetensors

The run

skryba rewrite passage.txt \
  --generator local \
  --local-model TinyLlama/TinyLlama-1.1B-Chat-v1.0 \
  --adapter ~/.stado/work/skryba-corpus/my-voice.safetensors \
  --local-device cpu --local-precision f16 --local-max-new-tokens 128 \
  --json --candidates 8 --max-words-per-chunk 40 --temperature 0.9 --seed 42

Exit status 0. Stdout was 20,392 bytes. The run took 7m41.247s. Repeated later without changing a flag it took 3m29.182s and wrote byte-identical stdout, because each ster generate is seeded with its own candidate index and nothing on this route is drawn from a clock. The identical work taking 461s and then 209s is the load average on a shared machine, and the reason no number here that is measured in seconds should be read as a property of the product.

16 ster generate processes ran: 8 candidates times 2 chunks. The process table was sampled every 100ms for the length of the run. Those 16 were the only ones, none was ever observed in state Z, and none outlived the run:

lifetimes, seconds: 31.12 49.03 45.36 33.49 58.35 29.58 34.29 26.81
                    32.85  4.73  4.81 13.07 35.44 13.71 13.26 26.02
min 4.73   max 58.35   mean 28.25   sum 451.92

Never more than one at a time, which is the whole point of the route and the reason it is chosen rather than default: each process maps the checkpoint again, and a group generated in parallel would hold one copy of TinyLlama's weights per candidate. 451.92s of process lifetime inside 461.25s of wall clock is that sequence, observed rather than asserted; the repeat run's 16 processes summed to 201.32s inside 209.18s. The same passage on the Brama route took 2.046s for the same 16 candidates: on this machine the weights being local cost between 100 and 225 times the wall clock, depending on what else was running.

The detector fixture was called 15 times and the judge 15 times, for 15 distinct texts: every candidate that survived generation was scored by the ensemble and evaluated by the judge, exactly once each. 15 and not 16, because of the line Skryba printed on stderr:

candidate 6 (specific) came back as whitespace from `ster generate`; dropping it from the group

The report

Both chunks selected a candidate; neither fell back to its source. --seed 42 drew the same strategy indices here as on the Brama route, because strategy sampling happens before the generator is asked for anything: 3 3 3 2 0 2 4 5 for the first chunk and 0 0 5 3 2 0 3 1 for the second. Across the run all six built-in strategies were sampled. specific, index 4, was drawn exactly once — at offset 6 of the first chunk, which is the candidate that came back as whitespace, so four distinct strategies reached scoring in that chunk and five in the other.

chunk 0   source 30 words   used_source_fallback false
  [0] 3 rhythmic         ai_probability 0.4699   quality_passed true
  [1] 3 rhythmic         ai_probability 0.45     quality_passed true
  [2] 3 rhythmic         ai_probability 0.4418   quality_passed true   <- selected
  [3] 2 scholarly-human  ai_probability 0.5137   quality_passed true
  [4] 0 natural-editor   ai_probability 0.4652   quality_passed true
  [5] 2 scholarly-human  ai_probability 0.5388   quality_passed true
  [6] 5 light-touch      ai_probability 0.4729   quality_passed true

chunk 1   source 31 words   used_source_fallback false
  [0] 0 natural-editor   ai_probability 0.4408   quality_passed true
  [1] 0 natural-editor   ai_probability 0.4637   quality_passed true
  [2] 5 light-touch      ai_probability 0.4658   quality_passed true
  [3] 3 rhythmic         ai_probability 0.4493   quality_passed true
  [4] 2 scholarly-human  ai_probability 0.4706   quality_passed true
  [5] 0 natural-editor   ai_probability 0.4393   quality_passed true   <- selected
  [6] 3 rhythmic         ai_probability 0.4735   quality_passed true
  [7] 1 plainspoken      ai_probability 0.4407   quality_passed true

quality_score was 0.9601999999999999 on all 15, because the fixture judge returns one object. Every cached field was false: all 15 texts were distinct, so nothing here could come from the score cache.

The selected rhythmic candidate for the first chunk, quoted from the report:

Introduction

Despite the high impact of ground-up strategies for improving patient satisfaction and retention, few clinicians have access to the potential benefits. The pilot study was designed to recruit experienced clinicians with limited funds and resources, in order to present an effective and achievable protocol that could be further modified for efficient implementation. This study reviewed the literature to determine the best strategies for improving satisfaction and retention for the acute surgical subspecialty, specifically for the introduction of practice-changing interventions and in real-time intuitive feedback.

Background & Purpose

And the selected natural-editor candidate for the second chunk, quoted in full:

Conclusion:

Eleven characters, one word, and the lowest ai_probability in its group of eight. Joined, result.text was:

Introduction

Despite the high impact of ground-up strategies for improving patient satisfaction and retention, few clinicians have access to the potential benefits. The pilot study was designed to recruit experienced clinicians with limited funds and resources, in order to present an effective and achievable protocol that could be further modified for efficient implementation. This study reviewed the literature to determine the best strategies for improving satisfaction and retention for the acute surgical subspecialty, specifically for the introduction of practice-changing interventions and in real-time intuitive feedback.

Background & Purpose

Conclusion:

Not one claim of the source survives. Both paragraphs are gone, replaced by an invented clinical abstract and a section heading. The loop did exactly what it is specified to do — the least detectable candidate that passes the quality gate — and the quality gate was a fixture that passes everything. A real Brama judge would have scored meaning_preservation near zero and set critical_error on all 15, no candidate would have passed, and both chunks would have kept their source. That is the guarantee the product rests on, and a stand-in judge is exactly what switches it off.

The other candidates in the same two groups show the rest of what a 1.1B checkpoint does with this instruction. Three began by echoing the prompt's own delimiter:

Passage:
These results suggest that the intervention was successful, that the potential risks were manageable, and that the participants experienced a significant improvement in satisfaction and retention. Additionally, the study found that the intervention was a reliable measure of satisfaction.

One answered the instruction instead of following it:

Could you perform the following task for me?

One restated it:

However, full statistical analyses and detailed program implementation plans were not feasible for this project. To keep the rewritten passage authentic, omit all demographic scores, methods, dates, and qualifications.

Three defects the run found

None of these was found by reading the source. All three needed the process to actually run.

One empty completion aborted the whole document

The first attempt exited 1 after 2m12.620s and seven ster generate processes, six of which had produced a candidate, with nothing written — no report, no output file, no partial text:

$ skryba rewrite passage.txt --generator local …
Error: `ster generate` returned nothing for a 81-word prompt; raise --local-max-new-tokens

Reproduced by hand with the exact argv Skryba builds for candidate 6 of the first chunk — strategy specific, --seed 6 — ster generate exited 0 and wrote 129 bytes, every one of them a newline. The completion was not truncated for want of budget; the model spent all 128 tokens on \n, so the remedy the sentence named would not have helped. Worse, the failure landed before any candidate was scored, which is upstream of the branch that keeps a source chunk when nothing beats it — the one invariant that makes the product safe to run at all.

A group is sampled with replacement and the source chunk is already the fallback when nothing in the group survives, so one degenerate sample is worth one candidate. It is now dropped and named on stderr, and the chunk is selected from what is left. When a whole group comes back that way the group is empty, the quality filter selects nothing, and used_source_fallback is true — the existing branch, reached without adding a new one. That is the fix in 9a486e8, and it is why the run above finished with 7 candidates in one chunk and 8 in the other.

The group paid twice for identical text

On the Brama route the same passage cost 16 detector requests and 16 quality evaluations for 9 distinct candidate texts. A group of 8 regularly draws the same strategy more than once — 3 3 3 2 0 2 4 5 above draws rhythmic three times — and a deterministic generator returns byte-identical text for it. Two different strategies can collide as well: light-touch and plainspoken produced the same text for the second chunk, which is why 16 candidates held only 9 distinct texts.

The score cache cannot catch this. The whole group is scored concurrently, so every request is in flight before the first result is written and all of them miss. The judge has no cache in front of it at all.

Identical text has an identical score by definition, so the group is now collapsed before it leaves: one measurement per distinct text, handed back to every position that asked for it. The same command now costs 9 and 9, and the report is unchanged — same 8 candidates per chunk, same probabilities to four decimals, same selection, same 19,046 bytes of stdout. Generation is not deduplicated and must not be: a real model asked the same prompt twice does not answer identically, and GRPO needs its group to be the size it asked for.

Ster's refusals arrived behind two Error: frames

A missing adapter used to read:

Error: Error: failed to read adapter /Users/…/no-such-adapter.safetensors

Ster prints its own Error: frame, Skryba passed the line through verbatim, and anyhow printed a second frame in front of it. The leading frame is now dropped and the sentence is untouched, including any Caused by: chain below it.

The same passage on the Brama route

Same loop, same seed, same eight candidates per chunk, same fixture judge and fixture detector; the generator is the fixture instead of Ster.

skryba rewrite passage.txt --generator brama --json \
  --candidates 8 --max-words-per-chunk 40 --temperature 0.9 --seed 42

Exit 0, 2.046s, 19,046 bytes of stdout. It selected scholarly-human at ai_probability 0.4321 for the first chunk and light-touch at 0.4246 for the second, and result.text was:

In addition, the study found that participants who received the intervention reported higher satisfaction. Beyond that, retention improved by twelve percent over the six-month follow-up period, and no adverse events were recorded.

Also, the research team used a full follow-up protocol at weeks four, twelve, and twenty-four. Note that three participants withdrew for reasons unrelated to the intervention itself.

What differs

Not better or worse. The two generators were asked the same thing and what came back differs in kind.

	Brama route (fixture generator)	Local route (real Ster and adapter)
Rewrite of chunk 0	32 words, every claim intact	88 words, no claim of the source present
Rewrite of chunk 1	27 words, every claim intact	Conclusion:
What changed in the source	three connectives, one hedge, two word choices	everything
Distinct candidate texts per group	5 of 8, then 4 of 8	7 of 7, then 8 of 8
Wall clock	2.046s	7m41.247s
Cost per candidate	one HTTP POST	one process, one mapping of the checkpoint
Selected ai_probability	0.4321 and 0.4246	0.4418 and 0.4393

The fixture generator's determinism is why its groups collapse to 5 and 4 distinct texts while the local groups do not: each ster generate is seeded with its own candidate index, so the same strategy drawn three times produces three different candidates. The fixture generator preserves meaning because all it does is substitute words — Additionally to In addition, Furthermore to Beyond that, Moreover to Also, It is important to note that to Note that, utilized to used, comprehensive to full. The local generator does not preserve meaning at all.

When nothing passes the gate, the source is kept

The branch that makes the loop safe was exercised on the local route, with real generation, by raising the gate above what the fixture judge returns:

skryba --minimum-quality 0.98 rewrite passage.txt \
  --generator local --local-model TinyLlama/TinyLlama-1.1B-Chat-v1.0 \
  --adapter ~/.stado/work/skryba-corpus/my-voice.safetensors \
  --local-precision f16 --local-max-new-tokens 128 \
  --json --candidates 2 --max-words-per-chunk 40 --seed 42

Exit 0, 48.797s, 4 ster generate processes. All four candidates came back with quality_passed false; both chunks reported used_source_fallback true with strategy source-fallback and strategy_index null; the detector fixture and the judge were each called 6 times, four candidates plus two sources; and the printed text was the input, byte for byte. The source chunks scored 0.549 and 0.6282 against the fixture — higher than every one of the four candidates it rejected, which scored 0.4699, 0.45, 0.4639 and 0.4776. Skryba kept them anyway, which is the rule: quality is a gate, not a ranking preference.

What this proves, and what it does not

It proves the local route runs as specified. Chunking split the passage in two; the policy sampled eight strategies per chunk and all six built-in strategies over the run; one ster generate process ran per candidate, 16 of them, one at a time, each mapping the checkpoint again, all of them reaped and none left behind; a degenerate sample cost one candidate instead of the document; every surviving candidate was detector-scored and judged exactly once; the least detectable quality-passing candidate was selected for each chunk; and with the gate raised the source was kept instead. --candidates is honoured exactly: 8 produced the 16 processes counted above, and on the Brama route, where a sweep is cheap, 1, 3 and 12 produced 1, 3 and 12 candidates per chunk and 2, 6 and 24 generator calls. Nothing on this path writes a temporary file — the prompt travels in argv, which is why the sampled process table holds the whole system prompt — and the only files any of these runs created were the score cache and what --output and --report named.

It does not prove the local route produces a usable rewrite. On this input it produced the opposite, and the run says plainly why. A 1.1B checkpoint at --temperature 0.9 with a rank-8 adapter is too small for an instruction that requires holding every claim of a passage while rewording it, and the fixture judge that let the result through does not read text. The judge is the component that would have refused this, and replacing it with a stand-in is what allowed a one-word rewrite to be selected over its source. Read the page as the shape of the plumbing, verified; not as evidence about what a personal adapter writes like, and not as a detector score of anything.