skryba benchmark

Evaluate rewrite quality either on passages collected from documentation roots or on a verified curated corpus. It generates candidates and quality-judges them; it does not call detectors.

Invocation

skryba [GLOBAL OPTIONS] --minimum-quality <F> benchmark [ROOTS]... [--corpus <PATH>] --output <PATH> [--policy <PATH>] --samples <N> --candidates <N> --min-words <N> --max-words <N> --temperature <VALUE> --seed <SEED>

Supply one or more documentation ROOTS, or supply --corpus <PATH>, never both. --output <PATH> is required and names where the complete report is written; no path is assumed. --samples, --candidates, --min-words, --max-words, --temperature and --seed are required too: Skryba assumes none of them, and a missing one is refused by the parser with exit 2. A policy file is optional. Global Brama generator/judge settings and the required --minimum-quality precede the command.

skryba --minimum-quality <F> benchmark docs --output build/docs-quality-benchmark.json \
  --samples <N> --candidates <N> --min-words <N> --max-words <N> --temperature <T> --seed <N>

Output and state

The complete schema-versioned report is written to --output; stdout prints its summary JSON. Documentation mode reads a named file as it is and a named folder where the judge model, shown one folder's entries at a time without following symlinks, names the person's prose; a folder it cannot judge stops the run with its cause. It then takes one passage per file: the first run of consecutive prose lines holding at least --min-words and at most --max-words words. A single line longer than --max-words gives its first --max-words words, in order, so no passage exceeds the maximum. Passages are then sampled in an order seeded by --seed. Corpus mode verifies declared words and checksums before sampling. No detector cache is read or written.

Refusals

• --minimum-quality is required: the quality score a rewrite must reach is the caller's to state.
• benchmark needs --corpus or at least one documentation root.
• benchmark accepts either --corpus or documentation roots, never both.
• benchmark sample and candidate counts must be positive.
• benchmark word bounds are invalid when documentation minimum is zero or exceeds maximum.
• no qualifying passages found.
• temperature must be a finite number of at least 0; minimum quality must be between 0 and 1.
• reading <PATH> with the file system's cause when a prose file cannot be read as text, for example invalid UTF-8; no report is written. A file removed after the folder walk is skipped.
• Curated corpus refusals match corpus. Brama and policy parse failures are returned; an unwritable report is surfaced as writing <PATH>.