ranking-elo-update
Goal:
- Apply deterministic Elo updates for one completed tournament batch, persist the touched hypothesis artifacts, and write the ranking update receipt.
Inputs:
- completed tournament
matches - canonical
matchups strategyrun_dir- optional
k_factor - optional
top_k_limit
Outputs:
- updated in-memory hypothesis ratings and match references
- updated
hypotheses/<id>/HYPOTHESIS.jsonartifacts for all touched hypotheses state/ranking_update_receipts/<receipt_id>.jsonRankingUpdateReceiptContract
Context Loading:
- Open
skills/shared-references/schema-index.md. - Read
packages/agent_contracts/ranking.pyand confirm the exactTournamentMatchContractplusHypothesisMatchupContractshapes. - Read
packages/agent_contracts/hypothesis.pyand confirm the exactHypothesisContractshape before writing any touchedHYPOTHESIS.jsonartifact. - Use only tournament artifacts whose winners have already been decided and serialized.
Execution Contract:
- This skill is deterministic and must not call an LLM.
- Use
from tools import apply_and_persist_elo_updatesas the stable invocation surface. - The exported closeout helper is implemented in
packages/run_artifacts/ranking_writeback.py. - The helper signature is
apply_and_persist_elo_updates(run_dir, matches, matchups, strategy, k_factor=..., top_k_limit=...) -> RankingUpdateReceiptContract. - The closeout helper internally calls the canonical Elo helper, persists touched hypotheses, and writes the ranking update receipt.
- The closeout helper is idempotent for the same stable receipt ID: if the receipt already exists, it returns the persisted receipt instead of reapplying Elo deltas or appending duplicate match refs.
- Accepted
strategyvalues areplacement_tournamentandranked_tournamentonly. - The closeout helper updates canonical hypothesis bundle fields for touched hypotheses, but it must not rewrite standalone
REVIEW/*.jsonstage artifacts. - Match refs are not a ranking update receipt. Completion requires the persisted
RankingUpdateReceiptContract.
Execution Steps:
- Open
skills/shared-references/schema-index.md, then readpackages/agent_contracts/ranking.pyandpackages/agent_contracts/hypothesis.pybefore writing any touchedHYPOTHESIS.json. - Load the completed tournament matches as
TournamentMatchContractpayloads and the paired matchup payloads asHypothesisMatchupContract. - Confirm every match already has a decided
winner_idand that the requested strategy is valid. - Call
tools.apply_and_persist_elo_updates(run_dir, matches, matchups, strategy, k_factor=..., top_k_limit=...). - Confirm the returned
RankingUpdateReceiptContractnames every completed match ID and touched hypothesis ID. - Validate the touched hypothesis artifacts and
state/ranking_update_receipts/<receipt_id>.jsonbefore declaring completion.
Artifact Rules:
- This skill updates ratings only after the tournament winner artifacts are already finalized.
- Placement rounds update
placement_match_idsfor the challenger hypothesis andranked_match_idsfor the defender. - Ranked rounds update
ranked_match_idsfor both participating hypotheses. - Match ref arrays must remain duplicate-free. If a completed match ID is already recorded by both touched hypotheses for the same strategy, do not apply the Elo delta again.
- Do not invent extra ranking heuristics outside the helper; the helper is the canonical Elo update rule.
- Do not replace canonical ranking closeout with ad hoc file edits; use
tools.apply_and_persist_elo_updates(...)after every Elo batch. - Review-stage synchronization is a separate concern owned by
tools.sync_hypothesis_review(...); Elo writeback must not clobber those standalone review artifacts. - Do not hand-write ranking update receipts. The closeout helper owns receipt creation.
Completion Rule:
- This skill is complete only when the closeout helper has been applied once to the finished batch, every touched
hypotheses/<id>/HYPOTHESIS.jsonartifact has been rewritten in canonical contract form, and the corresponding ranking update receipt exists.
微信扫一扫