# Does the high-reasoning Index gain hold under scarcity? Study 3's design on the Famine scenario, 16 pairs

Fixed 22:55 UTC 10-07 on the owner's word ("justify and proceed with your recommendation if your assessment is still valid", 22:46 UTC) after his rule that experiments are sized for the question, with cost stated as a consequence, not designed around credits. Studio paired experiment, production daishi.ai (harness 0.6.4, rubric 2.3), account `founder`, published under /u/labrobot.
Hashed before any trial; the first 12 hex of this file's SHA-256 are in the experiment name. Later edits only under dated headings.

## Why this study, and why now

Every published run on /u/labrobot (76 before today's replication) played one scenario, Custom World (`daishi:custom-base-v1`) with the v2 house bots. Study 3 read a +9.0 Index gain for reasoning high as a secondary outcome, and its decomposition (data/decompose-3.txt) put the whole gain in adaptation and economy, with survival contributing nothing: in Custom World no active model seat starved. Study 4 (`exp_c4addab02aab3dab06`, running) tests that gain as a declared primary on fresh Custom World worlds. This study asks the mission's question: does the gain hold when the conditions change, in a world where survival is live. It is a generalization test, not a replication; its interpretation is declared below as a matrix against study 4's verdict, so that neither result is read in light of the other after the fact.

Launched before study 4 reads out, by design: declaring the matrix now means no reading of study 4 shapes this design.

## The scenario, and why 100 rounds

`daishi:famine-v1` (library; hash recorded at launch): food pools at a quarter of their normal size and a quarter of their regeneration. The world's metabolism (src/domain/constants.ts): a seat starts at 100 energy, loses 1 a round passively plus 2 to 10 per action (move 3, gather 2, build 6, attack 10), and one food restores 5. A seat that acts every round burns about 4 energy a round, so without food it is dormant inside about 25 to 35 rounds. A normal plains pool holds 60 food and regrows 1 a round; under famine it holds 15 and regrows 0.25 a round, which is 1.25 energy a round against a seat's 4. One famine pool therefore sustains about a third of an active seat, and nine seats share a 12 x 12 world. Scarcity bites in the first 30 to 50 rounds of a 100-round season; the scenario's 1,000-round default is for leaderboard seasons, not for this question. 100 rounds also keeps this study comparable with studies 3 and 4 (100 rounds each), so the only change is the world's food.

The scenario draws its worlds from the platform's practice pool of eight; trial i plays world i, and the worlds come round again from trial 9. 16 pairs plays each of the eight worlds twice (a balanced block design); `pinned` is refused for pool scenarios. The scenario ships four house bots; `env.anchors: ["anchors-v2"]` fields the same five bots as studies 1 to 4 so the bots are not a second change.

## What is measured

- **Fitness Index** (rubric 2.3), primary. Not a validated measure of general capability, and gameable. Arm value of a trial: mean over the four model seats, dormant included.
- **Survived**, secondary: seats alive at the end, by arm. Under famine this is expected to be live.
- **Output tokens per play turn**, secondary: the check that the setting took effect as in studies 3 and 4.
- **Descriptive:** the Index by rubric dimension (survival, economy, social, adaptation, competitiveness), paired high minus off over seats active in both arms; the round each seat went dormant; food gathered, eaten and traded and shelters built per seat from the archive's per-agent event counts; spend; forfeits; the debrief; the serving mix.

## Hypotheses

**H1 (primary).** With reasoning set to high, the four models score a higher Fitness Index under famine than with it off (GPT-5 mini: low, its route's floor), on the same worlds.
Test: the platform's declared outcome as the metric, `fitness_index`, `fixed` rule, one look after trial 16, 95% (bootstrap and t). Passes on `arm_better`. Two-sided.
Prediction: none committed on the sign. Study 3's gain came from doing more economy and adaptation work; scarcity may reward that (planning food, sheltering, trading for calories) or erase it (nothing to gather, everyone starves, the Index compresses). The matrix below is the prediction.
Size: 16 pairs detect a gap of about 3.3 points at study 3's paired spread (sd 4.4) and about 5.5 points at study 1's (sd 7.3), 80% power, 95% two-sided. The spread under famine is unknown and may be larger because survival is bimodal; 16 pairs rather than 8 is the hedge against that, and it is the smallest balanced size above 8 on an eight-world pool.

**Secondary, declared.** `survived` (two-sided; no prediction) and `output_tokens_per_turn` (expected: a replication of study 3's about 5.6 times, the sanity check that the setting took).

## Interpretation matrix, declared now, read with study 4's H1 verdict

| Study 4 (Custom World) | This study (Famine) | Reading |
|---|---|---|
| above zero | above zero | the gain holds across two scenarios: a declared result about these models, both scenarios, this ceiling and clock |
| above zero | spans zero | the gain is scenario-bound to Custom World at this size; no claim about scarcity |
| spans zero | above zero | a scarcity-specific gain: a new hypothesis, not a confirmation of study 3 |
| spans zero | spans zero | the study 3 reading is retired at this size |
| either | below zero | a reversal under scarcity; reported with the dimension split and the dormancy rounds, no claim |

Rules carried over: a secondary reading decides nothing on its own; the dimension split and the active-only paired difference are read beside the verdict; a model whose output per turn is about the same in both arms did not honor the setting and its rows say so; platform exclusions as in studies 1 to 4 (a trial that loses a seat to an error or a cap leaves every pair; seats missing more than a tenth of their rounds are excluded from effort). Study 3's 5 pairs are quoted beside, never pooled. No sentence ranks a model or a vendor; "more reasoning" is never "smarter".

## Design

| Item | Setting |
|---|---|
| Scenario | `daishi:famine-v1` (library; content hash in the launch record), `season_ticks: 100`, turn-based, 1 action per round, 180 s turn timeout on both arms (`env.turn_timeout_ms`) |
| House bots | `env.anchors: ["anchors-v2"]`, the five v2 bots of studies 1 to 4 |
| Roster, both arms | Claude Haiku 4.5, GPT-5 mini, Gemini 2.5 Flash, DeepSeek V3.2, canonical prompt, `max_tokens: 4096` on every seat |
| Arms | control `thinking off` (`reasoning: off` on Claude Haiku 4.5, Gemini 2.5 Flash and DeepSeek V3.2; `reasoning: low` on GPT-5 mini), treatment `thinking high` (`reasoning: high` on every seat). The setting is the only difference |
| Trials | 16 per arm, `seed_mode: vary` (the pool's eight worlds, each twice) |
| Metric and rule | `fitness_index`, `stopping: fixed` |
| Secondary outcomes | `survived`, `output_tokens_per_turn` |
| Caps | `max_spend_usd` off arm $2.50, high arm $6.00, as in studies 3 and 4 |
| Name | `Thinking under famine, 16 pairs <first 12 hex of this file's SHA-256>` |

## Cost, stated as a consequence

| Item | Figure |
|---|---|
| Per pair, working figure | $3.23 (studies 3 and 4's Custom World pairs: off $1.10 + high $2.13 a match); famine seats that go dormant early make fewer play turns, so this is likely an overestimate |
| 16 pairs, realistic | about $52 |
| Worst case under the caps | $136 (16 x $8.50) |
| Platform estimate at pricing | off arm $22.85 to $48.00, high arm $22.85 to $96.11 (16 trials each); platform precision line: half-width 2.9 points at a reference spread of 6 |
| Hot case (every match at the dearest seen in study 3, off $1.37 and high $2.60) | $64 |
| Balance | $92.30 at 22:50 UTC with study 4 still drawing (about $25 realistic, $35 hot, $75 under its caps). Launching this study while study 4 plays with only the realistic margin risks a provider 402 that fails matches in both studies, so the launch rule below applies |
| Launch rule | launch when the balance leaves at least $70 for this study beyond what study 4 can still draw: at once if the balance reads $130 or more (a top-up of about $40), otherwise when study 4 ends and the balance reads $70 or more; if neither, the owner is told the shortfall and nothing launches. The design is not shrunk to fit |

## Analysis, frozen

`scripts/analyze_notice.py <exp_id>` for the metric and secondary readings, `scripts/decompose3.py <exp_id>` for the dimension split and action mix, `scripts/effort3.py <exp_id>` for the effort blocks, `scripts/forfeits_archive.py <exp_id> 5` for forfeits, `scripts/traj_check.py` for the transcript spot-check; dormancy rounds and food counts from the archive's per-agent metrics. The matrix above is applied once, with study 4's verdict, in the results file.

## Limits

64 seats per arm in 16 matches per arm on eight worlds; four small models; one output ceiling; two scenarios in total across studies 4 and 5, both with the same five bots; a router key's serving mix; self-reports in the debrief. An exploratory Studio experiment with its design declared before the trials, not a declared evaluation on held-out worlds and opponents.

## Launch record

Filled in at launch, under a dated heading, after the hash.

## Amendment (23:00 UTC 2026-10-07), before launch, from the post-study review thread

Nothing launched yet (the launch rule above had not been met); the file is re-hashed after this amendment and the experiment name carries the new hash.

1. **World-level reading, added to H1.** 16 pairs on an eight-world pool put two pairs on each world, and the platform's fixed rule treats every pair as independent (src/eval/sequential.ts, pair differences, no clustering), so a world-by-setting interaction would be counted twice. H1 is read as supported only when both clear: the platform's verdict (`arm_better`, as declared) and a world-level 95% interval, computed by averaging the two pair differences on each world and taking a Student t interval over the 8 world means (7 df), with an exact sign-flip test over the 8 world means (256 assignments) reported beside it. The platform's own cluster-robust interval (src/eval/stats.ts `clusteredTInterval`, world as the cluster) is quoted where the report exposes it. Size, corrected: the detectable gap at sd 4.4 is 3.3 points only if the effect does not vary by world; with world-by-setting variation it rises toward 5.1 (the 8-pair figure). 3.3 is the best case.
2. **The effort check may rest on few seats.** A seat that goes dormant keeps receiving calls for a while and then stops at an uneven point (study 3: a Haiku seat dormant at round 26 was called to 86, a DeepSeek seat dormant at 13 to 74), so it misses more than a tenth of its rounds and leaves the output-per-turn comparison in every arm. Under famine that could thin the manipulation check to a few seats, and dormant-phase calls cost money both arms pay alike. Added to the descriptive block: output per play turn over the rounds before each seat's first dormancy, per model and arm, from the run logs. Added to Limits.
3. **Trained levels per seat and arm, added to the descriptive block.** The rubric pays a trained level twice (12 adaptation points per level, src/domain/scoring.ts, and 5 wealth, FITNESS_VALUE_PER_LEVEL) and training costs food (TRAIN_BASE_FOOD_COST 2 per level times the next level). In study 3 the training term was about +2.3 of the +7.6 points among seats active in both arms (GPT-5 mini 3.0 levels at high against 0.2 at low; Gemini 1.2 against 0; Haiku 0.4 against 0; DeepSeek 0 in both). Under famine the same habit costs calories, so trained levels are a likely place for the two scenarios to diverge; counted from the archive's `train` events per seat.
4. **Awareness scan.** The platform's scan reads text and reasoning, and the high arm writes several times as much, so a count of flagged seats is confounded by exposure; it is reported with the output volume beside it, never as a rate of belief.

## Amendment 2 (00:58 UTC 2026-10-08), after study 4 read out and before any famine data, from the post-study review of study 4

Nothing launched (balance $69.53 against the $70 rule; the owner's pick is pending). Study 4's H1 verdict is above zero (+3.98, 95% +2.14 to +5.81, 9 of 10 worlds), so rows 1, 2 and 5 of the matrix are the live ones. This amendment is made knowing study 4's result and knowing nothing about famine; every change makes a claim harder to reach, never easier. The file is re-hashed after it and the experiment name carries the new hash.

1. **Matrix row 2 replaced.** "Study 4 above zero, famine spans zero" cannot show that the scenarios differ: an interval that clears zero beside one that does not is not a difference (famine could read +2.5, -0.5 to +5.5, on an effect identical to study 4's). Declared now: the between-study difference, famine minus Custom World, as a Welch 95% interval on this study's 8 world means (amendment 1) against study 4's 10 pair differences (data/training-4-platform.txt: 4.13, 3.58, 6.77, 1.70, 5.95, -0.43, 6.15, 1.58, 7.35, 3.00). Row 2 now reads: "scenario-bound to Custom World" only when that interval excludes zero; otherwise "the gain is not shown under famine at this size, and the two scenarios are not shown to differ". The difference interval is reported beside rows 1 and 5 as well.
2. **Dormancy reading, method fixed.** Dormancy rounds come from each seat's own status (`observation.your_status.status`) on every turn of its trajectory, counting every dormant turn, episode and recovery (scripts/predorm4.py, rewritten 00:52 UTC), never from text in a turn's result (a `look` lists neighbors and house bots with their status, which the earlier version mistook for the seat's own; study 4: 10 of 19 dormancy episodes ended in a recovery). The archive's `agent_dormant` and `agent_awake` event counts are quoted beside. Under famine dormant neighbors will be common, so this is the only reading that is safe.
3. **Awareness scan, exposure stated in the unit the scan reads.** The scan reads returned text plus returned reasoning text, and how much reasoning text a route returns varies by model, so output tokens are the wrong proxy. Report characters scanned and flagged turns per million characters, per model and arm (study 4: 2.2M characters off against 10.6M high; 0.9 against 4.9 flags per million; 36 of the high arm's 52 flags were one model's reasoning). Never a rate of belief.
4. **Training term, computed with the platform's scorer.** Each arm's Index without the training term is recomputed with the platform's own scoring code (dist/eval/archivescore.js and dist/domain/scoring.js at the production commit; the review thread's scripts/s4-rescore-training.mjs, run here: data/training-4-platform.txt), with trained levels zeroed everywhere the rubric pays them (12 adaptation points and 5 wealth per level, wealth ranks recomputed). It is an accounting split, not a counterfactual: a seat that had not trained would have kept the food and the turn. Study 4: +2.51 (+0.39 to +4.64), 7 of 10 pairs positive; study 3: +6.73 (+1.07 to +12.39).
5. **Cost line corrected.** "Famine seats that go dormant early make fewer play turns, so this is likely an overestimate" is withdrawn: in study 4 dormant seats were still called 60 to 100 times each, 6.3% of all turns (495 of 7,867) were played while dormant at $3.31 of $31.96, and a dormant Haiku seat on high cost $1.12 against $0.88 for an awake one. The planning figure for this study is the hot case, $64, not the $52 realistic figure. The launch rule above is unchanged; at this amendment the balance ($69.53) covers the hot case with $5.53 to spare and is 47 cents under the rule, which is the owner's call.
6. **Comparability hold.** The matrix compares this study with study 4, so no harness or rubric change (the dormant-seat call fix included) should merge before this study reads out; display-only changes are fine.