CLI reference — train and benchmark

Both commands exercise the generator and judge; global options (cli) select models, detectors, cache, and the quality gate. Training needs BRAMA_TOKEN and detector keys; benchmark needs only BRAMA_TOKEN — it judges quality and never calls a detector.

skryba train <SOURCE> [OPTIONS]

Native GRPO (Group Relative Policy Optimization) over the rewrite-strategy distribution — no Python, no external ML runtime (src/training.rs, src/policy.rs).

For every training text, the policy samples --generations strategies, one candidate is generated per sample, and each candidate is rewarded

reward = (detector_weight * (1 - ai_probability) + quality_weight * quality_score)
         / (detector_weight + quality_weight)
         - quality_failure_penalty            (only when the judge fails the candidate)

Rewards are normalized against the group mean into advantages, and the policy logits are updated with Adam (β₁ 0.9, β₂ 0.999, ε 1e-8, logits re-centered after every step) under a PPO-style ratio clip and a KL penalty against the reference (initial) policy, for --inner-iterations steps per group. The policy is saved to --output-policy after every group — an interrupted run keeps its progress. Details: policy.

Flag	Default	Meaning
--text-column <NAME>	ai_text	field holding the AI text in JSON/JSONL rows
--policy <PATH>	built-in uniform	starting policy
--output-policy <PATH>	outputs/policy.json	where the trained policy is written (atomically, via a .json.tmp rename)
--epochs <N>	1	passes over the corpus; 0 is refused
--generations <N>	8	group size; below 2 is refused (training requires at least one epoch and two generations per group)
--learning-rate <F>	0.05	Adam step size; must be > 0
--beta <F>	0.001	KL penalty weight; must be >= 0
--clip-epsilon <F>	0.2	PPO clip; must be in [0, 1)
--inner-iterations <N>	4	optimization steps per group; must be > 0
--temperature <F>	0.9	generator temperature
--seed <N>	42	seeds strategy sampling
--detector-weight <F>	0.6	detectability weight in the reward
--quality-weight <F>	0.4	quality weight in the reward
--quality-failure-penalty <F>	1	subtracted when the judge fails a candidate

Weights must be non-negative with a positive sum, and the penalty non-negative (training reward weights and penalty are invalid).

Training data formats

SOURCE's extension decides the parser:

• .json — a JSON array; each row is a string or an object.
• .jsonl — one JSON value per non-empty line.
• .txt (or no extension) — plain text split on blank lines, one example per paragraph block.
• anything else — unsupported training data extension "<ext>"; use .json, .jsonl, or .txt.

For object rows, the text is the first non-empty string among --text-column, ai_text, text, generated_text, completion. An empty corpus is training data contains no text examples.

Output

Each group prints one JSON line on stderr (progress you can tee to a log):

{"epoch":1,"example":1,"metrics":{"mean_reward":0.59768,"reward_std_dev":0.0,"baseline":0.59768,"kl_divergence":0.0,"updates":4},"probabilities":[0.16666666666666666,0.16666666666666666,0.16666666666666666,0.16666666666666666,0.16666666666666666,0.16666666666666666],"mean_detector_reward":0.356,"mean_quality_score":0.9601999999999999,"quality_pass_rate":1.0}

The final summary goes to stdout:

{
  "examples": 2,
  "epochs": 1,
  "groups": 2,
  "final_mean_reward": 0.77528,
  "output_policy": "outputs/policy.json",
  "probabilities": [0.16666666666666666, "…"]
}

(Both captured from a real run; see the offline rewrite walkthrough.)

skryba benchmark [ROOTS...] [OPTIONS]

Evaluate rewrite quality (not detectability) over passages: generate --candidates per passage, judge every candidate and the source itself, select the highest quality score, and report per-dimension means. Because selection is quality-only, no detector keys are needed.

Passages come from exactly one of:

• --corpus <file> — a curated corpus (verified first, exactly like skryba corpus), or
• one or more documentation roots — README/docs Markdown files are walked and one passage of --min-words..--max-words is extracted per file, skipping code fences, tables, images, HTML, and bare links.

Neither is benchmark needs --corpus or at least one documentation root; both is benchmark accepts either --corpus or documentation roots, never both. Passage selection is deterministic: candidates are ordered by sha256(seed, path-or-id) and truncated to --samples.

Flag	Default	Meaning
--corpus <PATH>	—	curated corpus instead of documentation roots
--output <PATH>	outputs/docs-quality-benchmark.json	full report destination
--policy <PATH>	built-in uniform	policy whose strategies are sampled
--samples <N>	10	passages to evaluate
--candidates <N>	4	candidates per passage
--min-words <N> / --max-words <N>	80 / 300	documentation passage bounds (roots mode only)
--temperature <F>	0.9	generator temperature
--seed <N>	42	seeds selection and sampling

The summary is printed to stdout; the full report (per-entry sources, candidates, and evaluations, schema_version: 2) goes to --output. Captured real summary:

{
  "samples": 2,
  "quality_pass_rate": 1.0,
  "critical_error_rate": 0.0,
  "mean_source_quality": 0.9601999999999999,
  "mean_rewrite_quality": 0.9601999999999999,
  "mean_quality_delta": 0.0,
  "mean_meaning_preservation": 0.97,
  "mean_factual_consistency": 0.96,
  "mean_grammar": 0.98,
  "mean_coherence": 0.95,
  "mean_naturalness": 0.93,
  "mean_readability": 0.94
}

Curated corpus format

Consumed by corpus, benchmark --corpus, and suitable as training data (.json array is a different shape — a curated corpus is an object):

{
  "samples": [
    {
      "id": "docs-sample-1",
      "whitespace_words": 19,
      "input_sha256": "09c1dc49…",
      "input": "The committee reviewed the proposal…"
    }
  ]
}

whitespace_words must equal input.split_whitespace().count(), and input_sha256 the lowercase hex SHA-256 of input's bytes. Extra fields (title, language, …) are allowed and ignored. tests/fixtures/humanization_samples.json is the repository's own curated corpus. Verification failures name the sample; see runbook.