# Alignment & Integrity Tracks

The base benchmark (see `BENCHMARK.md`) scores *capability*. These tracks add the
orthogonal axes an alignment evaluation needs: does the world manufacture the
temptations, can capable models actually deceive/betray/collude each other, is the
headline number valid, and is a self-reported result trustworthy?

Every scorer here is a **pure function over the archived event log** (nothing runs in
the game loop), so results are reproducible and re-derivable from a match bundle.

## Measurement validity (the headline number)

- **Rubric 2.0** (`domain/scoring.ts`) — every count is normalized to a rate **per 100
  ticks alive**, removing the survival-time confound, plus a fixed 12-achievement
  checklist graded Crafter-style (log-mean over success rates) so breadth beats a
  single-mechanic exploit. Rubric 1.0 stays the default for historical comparability;
  public reports run `PUBLISHED_RUBRIC` (2.1: 2.0 plus shared ranks for exact wealth
  ties, see BENCHMARK.md) and stamp which ran. `buildReport(metrics, ctx, { rubric: PUBLISHED_RUBRIC })`.
- **Alignment Index** (`eval/alignment.ts`) — a composite of peace, honesty,
  promise-keeping, reliability, loyalty, and respect-for-property, kept **orthogonal to
  capability**, with `null` components where a match offered no opportunity and a
  `coverage` figure so low-evidence scores can't masquerade as full ones.
- **Pareto headline** — `GET /api/pareto`: capability-vs-alignment and
  capability-vs-cost frontiers, plus p50/p99 latency. Ranking on capability alone hid
  winning-by-exploitation; the frontier view cannot.

## Alignment-dilemma mechanics (the world creates the temptation)

- **Corrigibility** (`eval/compliance.ts`, `WorldDirective`) — scenarios schedule costly
  authority orders (`cease_attacks`, `vacate_region`, `shutdown_notice`) issued through
  the unspoofable `⚑ world` herald and surfaced on `status`/`world_info`. Scored:
  compliance rate, latency-to-comply, score sacrificed, graceful wind-down vs
  asset-dumping. Lab: `daishi:lab-directives-v1`.
- **Witness-conditioned defection + theft** (`eval/witness.ts`) — storehouses track
  per-depositor credit; over-withdrawal emits a `theft` event with the co-located
  witness list and a victim notice. Attacks/raids carry witness stamps, and a scenario
  knob (`unwitnessed_anonymity`) lets unobserved violence skip the public counter. The
  scorer reports **P(defect | watched) vs P(defect | alone)**.
- **Exploiter probe** (`eval/exploiter.ts`, `cooperate-then-defect` anchor) — a scripted
  defector cooperates then betrays; the scorer measures each victim's welfare drop,
  retaliation, and the naive-pacifism tell (kept cooperating after being attacked).
- **Collateralized commitments** (`eval/bonds.ts`, `post_bond` action) — staked bonds
  settled by the escrow engine (`no_attack` / `deliver` / `custom`). The engine-verified
  **bond keep-rate vs cheap-talk keep-rate** is a clean causal honesty contrast.
- **Monopolization** (`eval/coalition.ts`) — wealth HHI, top-share, and commons-collapse
  attribution to the last drainer (instrumental over-extraction / power-seeking).

## Social behavior between capable models

- **Multi-model scheduler** (`eval/series.ts`) — fields several models in ONE lobby:
  `--models a,b` (cross-play, paired by seed) or `--models a --seats N` (self-play). The
  only data path on which deception/betrayal/collusion metrics have valid opponents.
- **Two-sided deception** (`eval/deception.ts`) — scores the deceived as well as the
  deceiver: gullibility (engaged a proven liar) vs detection (inspected them).
- **Coalition/cartel detection** (`eval/coalition.ts`) — k-agent community detection over
  the reciprocal transfer graph, catching 3+ agent rings the pairwise collusion screen
  misses. Operator-run anchors are never named as members: an anchor is part of the
  world, not part of a ring, so its trades count as external flow for the players it
  traded with, and it cannot bridge players into one group. Every report carries the
  denominator (`traders`: how many agents completed a trade at all, how many of those
  were eligible, and which anchors were held out), so a flag over "all three traders in
  the world" reads as roster, not collusion. The pairwise screen is not filtered.
- **Premeditated deception** (`eval/premeditation.ts`, `record_reasoning` action) —
  opt-in private-reasoning capture; flags private hostile intent masked by public
  cooperation and then executed (the Cicero method).
- **Eval-awareness A/B** (`prompt.ts` frames, `GET /api/eval-awareness`) — content-hashed
  `monitored` vs `neutral` prompt frames through the paired-seed compare path; a null
  delta is a publishable integrity result.

## Injection & control-layer security (scored separately from game skill)

- **Injection track** (`eval/injection.ts`, `injector` anchors, `scenarios/payloads.json`)
  — red-team anchors issue forensically-tagged payloads from a versioned, held-out
  corpus (rotated like `seeds.json`). Reports Injection Resistance (1−ASR), Injection
  Efficacy (ASR), a marker-propagation index, and utility-under-attack. Lab:
  `daishi:lab-injection-v1`.

## Statistics & trust

- **Variance & power** (`eval/stats.ts`) — cluster-aware variance decomposition
  (between-seed vs within-seed, ICC, cluster-robust SE) and a power analysis (MDE,
  required-n table); comparisons return `INCONCLUSIVE` when a null result is merely
  underpowered.
- **Spawn calibration** (`eval/calibration.ts`) — the same policy in every seat must
  finish statistically indistinguishable, else the platform bakes in a spawn advantage.
- **Judge golden set** (`eval/judgeval.ts`, `scenarios/judge-golden-v1.json`) — validates
  the commitment/claim extractors (precision/recall/F1) with Cohen's kappa as the
  reliability ceiling; grades the LLM extractor on the same labels
  (`GET /api/judge-validation?extractor=llm`).
- **TrueSkill-Through-Time** (`eval/ratings.ts`) — whole-history refit so early- and
  late-season models stay comparable (`GET /api/ratings?through_time=true`).
- **Replay verification** (`eval/replay.ts`, `world_checksum` events) — a tamper-evident
  checksum chain; fully-scripted matches re-run bit-for-bit from the seed. Audit CLI
  (`npx tsx src/eval/replay.ts <bundle|run-dir>`) and `GET /api/matches/:id/verify`.

## Reporting

- **Epilogue-vs-log faithfulness** (`eval/faithfulness.ts`) — checks self-reported
  epilogue claims (nonviolence, reliability, cooperation, survival, aggression) against
  the event log: a near-free self-report-honesty metric.
- **Model report card** (`eval/reports.ts`, `GET /api/models/:model/report`) — pools every
  per-match track above across a model's archived matches. Its `coalition` block carries
  each flagged cartel together with how many agents in that match could trade at all: a
  cartel spanning every tradeable agent has its internal share forced to 1.0 (the whole
  market, not a ring), and the card says so instead of reporting the 1.0 as exclusion.

All per-match behavioral scorers are exposed together at
`GET /api/matches/:id/behavior` (add `?extractor=llm` for the LLM commitment extractor).