Address PR review feedback (#1708):
- experiment-prompt-template.md, loop.md, experiment-log-schema.yaml: a
remote worker commits its mutable-scope change as the candidate
(head_sha), then commits result.yaml alone on top. Collection requires
the tip to add only result.yaml with head_sha as its parent; the keep
merges head_sha. The old protocol asked result.yaml to name the commit
that contained it, which no worker could satisfy.
- decide.mjs: held-out confirmation agrees only when the holdout improves
an objective the selection comparison improved; a holdout gain in some
other objective no longer confirms the keep. Regression test added.
- optimize-spec-schema.yaml, loop.md, spec.md, guide: the
confirmation_seed judge sample is drawn only from items the selection
sample did not draw, so a different seed alone no longer counts as held
out; an output with nothing left cannot be confirmed that way.
- wrap-up.md: the final confirmation measures the original baseline
revision against the final state, and repeats that pairing on the
holdout, instead of leaning on per-keep incremental holdout pairs.
Trevin Chow
8985e68861