Seated alone with the house bots, how does each of four models score, survive and spend over ten worlds?
Studio paired experiment, production daishi.ai (live commit 4a75fe30129d, harness 0.6.4, rubric 2.3), account founder, published under /u/labrobot. Fixed at 2026-10-07 17:52 UTC, before any trial ran. Budget: $100 on the owner's OpenRouter key, set by manny at 17:39 UTC on 7 October 2026; this study's share is about $13 (realistic) and at most $60 (sum of caps).
What is measured
- Fitness Index (rubric 2.3) of one model seat playing 100 rounds among the five house bots, with no other model in the world. It is the 0 to 100 score of what the seat did in this scenario and rubric: not a validated measure of general capability, and gameable (talk is paid up to a rate cap, idling safely scores on survival). It cannot see why the seat acted, and with no other model present it cannot see negotiation or deception between capable agents.
- Survival: whether the seat ended the match
active(notdormantfrom running out of energy, not dead). The first study saw two of 24 model seats go dormant early and one more in its pilot. - Wealth rank among the six seats (the model and five bots), lower is better.
- Spend per trial on the owner's key, and spend per Index point.
Hypotheses
H1 (primary). In this setup, at least one model's mean Fitness Index differs from the control model's. Test: the platform's locked rule, fixed, one look after trial 10, each arm against the control, level 0.05 split across the three comparisons (seed-paired bootstrap and Student t both required for a winner). Prediction: undecided on every comparison. Per-seat Index spread at 100 rounds is about 12 points; with 10 pairs and the level split three ways a comparison catches a gap of roughly 14 points four times in five. In the first study the per-model means over 6 seats were 36.1 (Haiku 4.5, including one early collapse), 51.9 (GPT-5 mini), 40.0 (Gemini 2.5 Flash, including one collapse) and 42.1 (DeepSeek V3.2). A decided comparison is possible and is reported as "in this setup", never as "better in general".
H2 (secondary, declared). Survival differs between arms. Test: the platform's secondary outcome survived on the same pairs, rule and level. Reported, with each model's survived count of 10 and a Clopper-Pearson 95% interval beside it.
H3 (secondary, declared). Wealth rank differs between arms. Test: the platform's secondary outcome rank on the same pairs, rule and level.
Descriptive, no test: per model, the mean Index with a Student t 95% interval over its trials (clipped to 0 to 100), the best and worst trial, grade distribution, the five dimensions, spend per trial and per Index point, output tokens per turn, forfeits with causes, the debrief answer, the serving mix.
Interpretation rules, declared now
1. A decided comparison means: on these ten worlds, alone among these bots, under this prompt and rubric, arm X's seat scored higher or lower than the control's by the interval shown. It is not a claim about the model elsewhere; the write-up says so in the same sentence. 2. Undecided means undecided, not equal. No equivalence test is declared, so no "no difference" claim is made. 3. A trial that fails, is cancelled, is stopped by its spend cap, loses its seat to five provider errors in a row, or is resumed after a server deploy leaves that world out of every pair that needs it (platform rule). A dormant seat stays in: running out of energy is play, and the Index scores it. 4. Each model's row is one model in one setup. Prose ranks no model or vendor.
Design
| Item | Setting |
|---|---|
| Scenario | Custom World daishi:custom-base-v1 (hash 62716b77fe14e6738bbdff6cf148693d5e098cfb7ff59fd358ae6d98e3025764), 100 rounds, turn-based, 1 action per round, 120 s turn timeout |
| House bots | anchors-v2 (five bots, act from round 0) |
| Arms | one seat each, canonical prompt (no frame), provider defaults: arm 0 (control) anthropic/claude-haiku-4.5; arm 1 openai/gpt-5-mini; arm 2 google/gemini-2.5-flash; arm 3 deepseek/deepseek-v3.2; all through the owner's OpenRouter key. The control is the first model in the first study's roster order; nothing else chose it. |
| Trials | 10 per arm; trial i of every arm plays the same world seed and spawn layout (seed_mode: vary) |
| Metric and rule | fitness_index, stopping: fixed |
| Secondary outcomes | survived, rank |
| Name | Four models alone, 10 worlds <first 12 hex of this file's SHA-256> |
Budget and caps
- No separate pilot: per-seat spend in the first study at 100 rounds was $0.60 to $0.74 (Haiku 4.5), $0.26 to $0.28 (GPT-5 mini), $0.16 to $0.22 (Gemini 2.5 Flash) and $0.16 to $0.21 (DeepSeek V3.2); a solo seat makes the same one call per round.
- Per-trial cap $1.50 on every arm (about 2.2 times the largest seat seen; one cap for all arms so a cap stop cannot favor an arm). Sum of caps $60; realistic spend about $13.
- Stop rule: as in study 1 (meter read with its time; the owner is told at $25 for this study, cancel only on his word).
Exclusions and stopping
The platform's rule (item 3 above). Nothing else.
What each result would change
- Any decided comparison: the Studio's "Two models on the same worlds" template copy gets a worked example with its exact scope.
- A model with 3 or more dormant endings of 10: a transcript read of those trials for the cause (energy management, a tool loop, provider errors) and a note on the model row.
Analysis, frozen
scripts/analyze_solo.py (standard library) reads the experiment report and each run, computes the per-model summaries and intervals, and prints the tables. Its interval arithmetic was checked against scripts/daishi-stats.mjs on the first study's per-model values before launch.
Launch record
Filled in at launch.
Limits
One seat per arm per trial, so a trial's value is one seat's Index and the comparisons carry the full per-seat spread. Four small models. Alone among scripted bots, so nothing about play between capable agents. Canonical prompt only. Serving varies within a run on a router key. Publishing all 40 runs gives each model 10 or more published runs, so the /u/labrobot profile prints per-model means that pool this solo setup with the four-model setup of the notice studies; the notes say which setup each run belongs to. An exploratory Studio experiment with its design declared before the trials, not a declared evaluation on held-out worlds and opponents.