Declared design: Thinking effort by model, a 2x2 factorial, 16 trials 6033f4ca4743

Check it yourself: download the exact bytes and run sha256sum design.md (on a Mac, shasum -a 256 design.md). It must print the SHA-256 above.

Design 8: thinking effort by model, a 2x2 factorial on the cheap tier, 16 trials per arm in open play

Declared design, final text at 03:10 UTC 2026-10-14 (draft v3 at 22:34 UTC 2026-10-12, v1 at 22:02 UTC, v2 at 22:26 UTC), written for public reading. This text is hashed before the first trial and attached to the study; the first 12 hex characters of its SHA-256 go in the study's name. Amendments after hashing appear under dated headings at the end and never change the text above them. Until the hash, changes are listed under "Changes before hashing".

Question

In study 7 (declared design 8fb94fc8cf9c, 12 October 2026) a roster of two Claude Haiku 5.5 and two Claude Sonnet 5.5 seats scored 10.8 Fitness Index points higher at thinking effort high than at low, 8 of 8 pairs, with both models' effort changed together. Read per model, the two Sonnet 5.5 seats scored 29.0 points higher at high and the two Haiku 5.5 seats 7.3 points lower, 0 of 8 pairs positive. Because both settings moved at once, neither row is that model's own effort effect: the Haiku 5.5 drop could be its own response to high effort, the Sonnet 5.5 seats' doing (at high both Sonnet seats out-ranked every Haiku seat on wealth, and the rubric's rank term alone is worth about 1.2 points of the drop), or a harness artefact (the world's menu called the lookup tools free while the harness said only afterwards that the call took the turn; Haiku 5.5 at high spent 62% of its turns that way).

This study varies each model's effort on its own, in the same mixed roster, so that the two things a shared world mixes can be read apart: a model's own response to its effort setting, and the effect of the other seats' setting on its score (the cross effect). The harness wording for free tools is fixed before this study, so the third account is removed rather than measured.

Why now

Study 7's headline is a roster net that a reader could take as "more effort helps" or as "more effort hurts the Haiku 5.5 seats". Only a design that moves one model's setting at a time can say which part of that net belongs to whom, and the cross effect is a reading that only a shared world can give: whether one agent's score depends on how hard the agents beside it think. The roster, scenario and maps are study 7's, so the question is answered in the field where it arose, at about $2 a match on list prices.

Two results would matter beyond this study. The result that would surprise us is a positive own effect for Haiku 5.5: its seats scoring higher at high effort with the Sonnet 5.5 seats' setting averaged out. We put the chance that it is true at about one in four, and the chance that this study shows it (an interval above zero) at about one in ten. The result that would surprise most readers is the opposite one: a decided drop for the Haiku 5.5 seats at their own high effort, averaged over the other seats' two settings, a counterexample in a shared world to "think harder, do better". Under our prior the chance that this study shows that is about one in two (51 to 53% at 16 trials, 45% at 8). An interval spanning zero has about a one in three chance at 16 trials (46% at 8) and is reported as the interval; at an estimate of exactly -3 the map-level interval would run from about -5.7 to -0.3, decided or at the edge.

What is measured

Seat groups: X is the two Claude Haiku 5.5 seats and Y is the two Claude Sonnet 5.5 seats, in the same positions in every arm. Arm labels name the seat groups' settings, not models; model names appear in seat rows only, and the public note's first line defines X and Y.

Per-model Index. For each trial and arm, each model's mean Fitness Index (rubric 2.3) over its two seats. These are the quantities the declared contrasts are built from. Each trial plays one of the scenario's eight practice maps; with 16 trials every map plays twice, on two spawn layouts, and the declared unit is the map: each contrast is computed per trial and averaged over the map's two trials before any test (see Analysis).
Arm Index. The arm's mean Index over its four seats, per trial: the platform's metric, used for the platform's own comparisons and for the replication read of study 7's contrast.
Own effect of a model. That model's Index at high minus at low, averaged over the other group's two settings. Per trial, for X: half of ("X high, Y low" minus "both low") plus half of ("both high" minus "X low, Y high"), on X's Index. For Y: half of ("X low, Y high" minus "both low") plus half of ("both high" minus "X high, Y low"), on Y's Index.
Cross effect on a model. That model's Index when the other group is at high minus when it is at low, averaged over its own two settings. Per trial, for X: half of ("X low, Y high" minus "both low") plus half of ("both high" minus "X high, Y low"), on X's Index. For Y: half of ("X high, Y low" minus "both low") plus half of ("both high" minus "X low, Y high"), on Y's Index.
Identities, per model. Own plus cross equals "both high" minus "both low" on that model's Index, which is exactly the row study 7 read. Own minus cross equals ("X high, Y low" minus "X low, Y high") for X, and ("X low, Y high" minus "X high, Y low") for Y. The own-minus-cross difference is reported with its interval, descriptive, and is the only quantity that apportions a model's study 7 row between its own and the cross effect.
Interaction, for each model. Its own effect when the other group is at high minus its own effect when the other group is at low. Described with its interval; no test.
Secondary, survival. The share of the four roster seats active at the end of the match, per arm per trial (the platform's secondary outcome, each arm against the control).
Manipulation check and specificity read, output per turn. Per trial and per model, the mean output tokens per model call over that model's seats, under the platform's rule that a seat is measured in every arm or in none. The check: a model's output per turn is higher in the two arms where its own effort is high than in the two where it is low. The specificity read, declared and descriptive: a model's output per turn in the arm where only the other group's effort changed, against both low; a move there is a field effect on output, not the setting, and is reported as such.
The platform's comparisons beside. The platform compares each arm with the control on the arm Index and drops pairs arm by arm, so its n can differ from the declared contrasts' n (a trial out of any arm is out of every declared contrast); both counts are reported.
Described, no test. Index by rubric dimension per model and arm; trained levels per seat and each contrast recomputed with the platform's own scorer without the training term; dormant rounds read from each seat's own status; turn kinds per seat (productive action, chosen rest, fallback rest after a turn cut at the ceiling, free tool, refused call, model refusal); forfeits and pace; the debrief per model and arm; the voiced-awareness scan per model and arm, reported per million characters the scan read, where the characters are the vendor's thinking summaries the harness requests on this route from version 0.6.7, not raw reasoning, so its rates do not compare with routes that return raw reasoning; on this route Claude Sonnet 5.5 also returns its between-tool notes as thinking blocks, so its characters mix notes with summaries while Claude Haiku 5.5's are summaries alone, and each model's rate is read against itself across arms, never against the other model's (a model-arm cell with no characters is unmeasured, as in study 7, and Sonnet 5.5's low cells are expected to hold none, so its across-arm read may not exist); spend per seat from token counts at list price and as the platform's meter reads it, each labelled.
Mechanism read, declared. The free-tool share per seat (turns spent on a free tool such as the scoring or world information calls, each followed by the forced rest), per model and arm, read from the trajectories, on the harness version that tells the model such a call takes the turn. Study 7 read 25% of Haiku 5.5's turns at low and 62% at high, Sonnet 5.5 14% and 15%. Any Index contrast is read beside the change in this share before any other account. Described, not tested.
Dormancy and the raider, declared. Rounds each seat spent dormant, read from the seat's own status on each called turn and the platform's count of rounds it sat out, per model and arm, with the round it first went dormant; and, per trial and arm, the seat the house raider attacked in the first 10 rounds, the hits it took and the round it went dormant, from the public attack events and the seat's own status. Arms share spawn layouts per trial, so the raider usually meets the same slot in all four arms; in study 7 its first contact fell on a Haiku 5.5 seat in 6 of 8 pairs. One sensitivity read is declared: every contrast recomputed with the slot the raider attacked first in any arm of a trial dropped from all four arms of that trial (defined by exposure, which the shared layout fixes, not by dormancy, which a setting can change). Described, not tested; every verdict stays on the declared contrast.

Hypotheses

Every declared test below uses the map as its unit: the per-trial value is computed in each of the map's two trials and the two are averaged, unweighted, into one value per map; the test is a paired Student t over the 8 map means, two-sided, 7 degrees of freedom, with the exact sign-flip test over the 8 map means beside it (the smallest two-sided p 8 maps allow is 0.0078). A map with one surviving trial enters at that trial's value; a map with none leaves the test and the degrees of freedom drop by one.

H1 (primary, the open question). Claude Haiku 5.5's own effort effect differs from zero. Test as above at 0.05. Declared prediction: -3 Index points, tolerance 4 (a hit if the observed mean lies between -7 and +1). The prior: study 7's -7.3 row is the sum of this effect and the cross effect; the rank term the Sonnet 5.5 seats imposed is cross, and the free-tool wording that may have driven part of Haiku 5.5's idling is fixed before this study, so the own effect is expected smaller than the row and closer to zero. Surprise: a positive own effect for Haiku 5.5, its seats scoring higher at high effort with the Sonnet 5.5 seats' setting averaged out, would surprise us; we put it at about one chance in four, and the chance that this study shows it at about one in ten. The -5 case, used in the rules below, has one definition: the interval below zero and the mean at or below -5; it would say the drop is Haiku 5.5's own, averaged over the other seats' two settings, with the free-tool loop the first candidate mechanism to read.

S1 (declared secondary, the shared-world reading). The cross effect on Claude Haiku 5.5 differs from zero: its seats' Index moves with the Sonnet 5.5 seats' effort setting. Same test. Prediction: -4, tolerance 4 (a hit between -8 and 0). The prior: the rank term and shared resources in a 12x12 world. Equivalence reading, declared with its margin now: if the 95% interval spans zero, the 90% t interval is read against -4 to +4; inside it, the reading is "a cross effect on Haiku 5.5 larger than 4 points either way is ruled out at the declared margin"; otherwise "undecided". The two equivalence readings (S1, S3) sit outside Holm. The second surprise, true cross effects inside that margin on both models (no model's score moved by the other seats' setting by more than 4 points), we put at about one chance in five; the chance that this study shows it, both 90% intervals inside the margin, is about one in twenty, because both observed means would have to sit within about 1.6 to 1.8 points of zero.

S2 (declared secondary). Claude Sonnet 5.5's own effort effect differs from zero. Same test. Prediction: +22, tolerance 8 (a hit between +14 and +30), shrunk from study 7's +29.0 row because that row includes whatever the Haiku 5.5 seats' turning high did to the field. This is the reading closest to a manipulation check of the design: if it fails, the arms did not do what study 7's arms did.

S3 (declared secondary). The cross effect on Claude Sonnet 5.5 differs from zero. Same test. Prediction: +3, tolerance 5 (a hit between -2 and +8). The same equivalence reading as S1, margin 4.

S1 to S3 are read with Holm across the three at 0.05; H1 stands alone at 0.05. S2 is expected to take Holm's first slot, so the smaller of S1's and S3's p-values faces 0.025 and, if it passes, the larger faces 0.05; the sizing below uses 0.025 for S1.

S4 (replication read, descriptive against study 7). The arm Index of "both high" minus "both low" over the 8 maps, paired t, two-sided at 0.05 with the exact sign-flip beside it: study 7's contrast on a new harness version and fresh spawn layouts. Prediction: +8, tolerance 5 (a hit between +3 and +13), shrunk from +10.8. Against study 7: the difference of the two studies' mean paired differences with a Welch interval, descriptive and unpooled; no "replicates", "does not replicate" or "scenario-bound".

S5 (platform secondary). Survival share, each arm against the control, paired t over the maps at the platform's level; if every pair ties, the reading is "the arms tied in every pair". Seat-matched counts are described, untested.

M (manipulation check, precondition for reading H1 to S4). For each model, output per turn is higher in the two arms where its own effort is high than in the two where it is low: per map, the mean over those two arms minus the mean over the other two, paired t at 0.05, Holm across the two models. The specificity read beside it is descriptive.

The platform's own comparisons (each of the three arms against "both low", on the arm Index, at its rule's split of 0.05 across three comparisons, over pairs rather than maps) are quoted beside the declared contrasts, and a disagreement between a platform sentence and a declared reading is reported. They are not the declared tests: the factorial's contrasts are computed outside the platform from the run documents, as declared here.

The Index, survival and output readings are three measures and need not agree.

Interpretation rules, fixed before launch

Each own effect and cross effect is reported as its interval. An interval above zero reads "scored higher on the Fitness Index in this scenario when ...", below zero "scored lower ...", spanning zero "undecided: the interval is the reading; an effect inside it is neither shown nor ruled out", or the declared equivalence reading where its margin is met. No "no effect", no "small", no cause named for an undecided reading.
H1 and S1 are read together, descriptively: both intervals are reported with the own-minus-cross interval beside them. The sentence about study 7's Haiku 5.5 row says that the row equals the own effect plus the cross effect and quotes the own-minus-cross interval; it never apportions the row from the pair of verdicts, because one interval excluding zero while the other spans it is not a difference between the two effects.
M failing for a model: that model's seats did not demonstrably take the setting through this route; every contrast on that model is still reported as declared, with the flag "the setting was not shown to take for <model>", and no mechanism is claimed.
The specificity read moving (a model's output per turn changing in the arm where only the other group's effort changed): reported as a field effect on output, descriptive; it does not alter any verdict.
Each prediction is scored hit or miss against the observed mean, whatever the verdict.
S4 against study 7: a Welch difference interval between the two studies' mean paired differences. Named differences: the harness version (this study's 0.6.7 against 0.6.6), spawn layouts (redrawn per experiment, so the maps share terrain only), and the free-tool share. Descriptive and unpooled.
Per-model readings are each model's own and rank nothing. The write-up reports each model's effects on their own baselines; no sentence says one model benefits more, or less, from effort than the other, and no sentence names a model or vendor as better.
The interaction is described with its interval and never tested or claimed.
The stopping rule is fixed at 16 trials per arm with no precision target (on this platform a precision target belongs to the anytime rule only, and a precision stop would cancel an arm's remaining trials and break every contrast on those maps). Nothing is cancelled by rule.

Design

Scenario daishi:open-play-v1 (the reference open-play world, standard scarcity; content hash bf0917e5a67f5acafa3ffac5ec5f0d8f54cfd7fd6fa85fc4ee41aaedb896c8d2), 100 rounds, house bots anchors-v2, 180 s turn clock. Seeds vary by trial, which on this scenario walks the eight practice maps in order: trials 1 to 8 are maps 1 to 8, and trials 9 to 16 replay maps 1 to 8 on new spawn layouts.
Four arms, paired per trial on the same map and spawn layout. Same roster in every arm: two seats of Claude Haiku 5.5 (group X) and two of Claude Sonnet 5.5 (group Y), in the same positions, each on the account owner's own Anthropic key through the harness's native Anthropic route, output ceiling 8,192 tokens (on this route the ceiling covers thinking and text together; a turn cut at the ceiling becomes a rest), temperature unset.
Arm 0 "both low": every seat at reasoning low (the control). Arm 1 "X high, Y low": group X at high, group Y at low. Arm 2 "X low, Y high": group X at low, group Y at high. Arm 3 "both high": every seat at high. On this route a level is adaptive thinking with the vendor's effort setting; the requests differ only in effort, and what each model did with it is read from output per call. High is Claude Sonnet 5.5's default effort and medium is Claude Haiku 5.5's, so no arm is Haiku 5.5's default, and for Sonnet 5.5 the low arms turn effort down from its default.
Metric fitness_index; secondary outcomes survived and output_tokens_per_turn; the fixed rule at 0.05, cap 16, no interim look and no precision target; spread_from names study 7 (exp_dbba4910cef5e3f260) so the platform's detectable-gap line is sized from a measured paired spread on this scenario.
Harness: the reference harness at version 0.6.7, the version that carries the owner's approved change (the prompt line that a free tool takes the turn; thinking summaries requested on this route; retry and rate-limit counts logged), named here with its full scaffold string, daishi-ref-harness@0.6.7+prompt-v6.68dd2f4c+effort.<level> (prompt v6 adds the free-tool sentences; the pilot archives confirm the string before the hash), because without the first item a Haiku 5.5 own effect could be our wording published as a model result. No harness change lands between the pilot and the last trial; if one does, the pilot is played again on it before any trial. Scores compare only within one harness version: study 7 ran 0.6.6 and is never pooled with this study. Rubric 2.3. Native tool calls.
Stopping rule fixed at 16 trials per arm. No interim look.
Pilot: two matches, one of each mixed arm, played before launch on the study's harness version (see Pilot). Not part of the study's data.

Size and cost, as a consequence of the question

Spreads are taken from study 7 (the same roster, scenario and maps): the per-trial spread of each model's row, high minus low, was 6.33 Index points for Claude Haiku 5.5 and 6.88 for Claude Sonnet 5.5, and 4.15 for the arm mean; from 8 trials, the Haiku 5.5 figure itself sits in a 95% interval of about 4.2 to 12.9. For the sizing only, a main effect per trial, the mean of two such contrasts on one map, is taken as having the row spread over the square root of two (4.48 and 4.86); the tests assume nothing of the kind, each being a t on the observed per-map values. With 16 trials the unit is the map mean of two trials; how much that buys depends on how much of the paired-difference spread is between maps rather than within them, and the two earlier 16-trial studies on this scenario (studies 5 and 6) put the between-map share at 4% and 26%. On that basis the map-level test at 16 trials catches, 80% of the time, an own or cross effect of about 3.7 to 4.1 points on Haiku 5.5 and 4.0 to 4.5 on Sonnet 5.5, and an interaction of about 7.4 to 8.9; the pair-level figures of 3.4 and 3.6 would hold only if maps added nothing, and if the effect differed wholly by map 16 trials would buy nothing over 8. At the declared predictions the approximate power at 16 trials is: H1 at -3, 54 to 62%; H1 at -5, 92 to 96%; S1 at -4 at its 0.025 Holm slot, 64 to 73%; S2 at +22, near 100%; S3 at +3, under 40%; S4 at +8, near 100%. Under the prior for H1 (mean -3, one chance in four of a positive effect) the readings come out: interval below zero 51 to 53%, above zero 11%, spanning zero 35 to 38%; so the likeliest reading is an interval below zero, and an interval spanning zero has about a one in three chance (at an estimate of exactly -3 the map-level interval runs about -5.7 to -0.3, decided or at the edge). Eight trials per arm (each map once, 32 runs, about $62 at list) would catch about 5.2 and 5.6 points, with 37% power at H1's -3 and 77% at -5, S1 43% at its slot, and readings below 45%, above 9%, undecided 46%. The owner chooses the size before the hash; this draft carries 16, the size both the research and the review readings back.

Cost at list prices, from study 7's per-seat token profile with cache reads and writes at the vendor's own rates (Sonnet 5.5 cache reads at a twentieth of the input price, Haiku 5.5 at a tenth, writes at 1.25 times): "both low" about $1.61 a match, "X high, Y low" $1.60, "X low, Y high" $2.24, "both high" $2.24; 64 matches about $123 (band $104 to $138 from the per-seat minima and maxima), plus about $3.85 for the two pilot matches. The platform's meter prices these models at their list rates for runs launched since 20:49 UTC 12 October 2026 (price table 2026-10), so the metered figures are expected to agree with list to rounding; a run whose record carries an older price table is reported at that table and labelled. The per-match spend cap, set in the platform's metered dollars, is the larger of $10 and 2.5 times the costlier pilot match's metered spend, rounded up to the next $5, the same in every arm; at $10 the worst case the caps allow is $640, more than four times the costliest match study 7 played at list. A trial that hits the cap leaves the platform's pairs by its rule and, because every contrast needs all four arms of a trial, leaves the factorial entirely; its map then rests on one trial. Runs: 66 of the account's 2,000-run plan ceiling, with 195 used before this study.

Analysis, frozen

1. Health first for every run: finished, every round played, every seat kept, harness version per seat, forfeits and causes, seat end status, provider errors, retries and rate-limit refusals as the harness logs them, dropped request fields (none other than the summary request), thinking characters per seat (above zero at high and for Haiku 5.5 at low), truncated share, refusals, the debrief present, pace. 2. H1, S1 to S5 and M exactly as declared, from the run documents and the public match archives, per trial and then per map, with every exclusion listed in the same table; the identities checked (own plus cross equals the row); the own-minus-cross interval; the platform's own comparisons, with their n, and its detectable-gap line quoted beside. 3. Descriptions as listed under "What is measured", every number with its scope (trials, maps, models, ceiling, clock, rubric) and every share as k of n. 4. The replication read against study 7, descriptive. 5. The study published whole with its note; the write-up carries every trial, failure and exclusion, the pilots, and the spend.

The analysis script was written before the hash and tried on study 7's run documents (whose two arms give each model's row as the sum of its own and cross effects); the change log records the try.

Exclusions, stopping, publishing

Exclusions: the platform's rule, plus the factorial's. By the platform's rule a pair is out when either of its two trials failed (a match past the plan's run-time cap fails), was cancelled, lost a seat to the harness giving up on it (the reason is named), stopped short of its season at its spend cap, or was picked up by a new server release mid-match. For the declared contrasts, a trial index is out of every contrast when any of its four arms is out by that rule, so that each contrast reads the same maps; a map with one surviving trial enters at that trial's value, a map with none leaves the test and the degrees of freedom drop by one; the report names the reason for each trial left out. The platform's own comparisons drop pairs arm by arm, so their n is reported beside the declared n. Nothing is dropped after seeing a result.
A launch refused for a reasoning level amends this design under a dated heading before the first trial; what the pilots' receipts trigger is fixed under Pilot.
Stopping: fixed, 16 trials per arm, no precision target.
Publishing: the study whole, in the platform's one study-publish call that lists every trial that played in every arm with the declared note (never trial by trial), the two pilot matches as single matches marked as pilots, and this design, on the account owner's word, whatever the result.

What each result would change

H1 in the -5 case (interval below zero, mean at or below -5): Haiku 5.5's seats score lower at their own high effort in this scenario, averaged over the other seats' two settings; the free-tool share on the fixed wording and the turn kinds are the first place to look, and a Haiku 5.5-only roster at low, medium and high is the next study, declared on its own.
H1 spanning zero and S1 below zero: the Haiku 5.5 score moves with the Sonnet 5.5 seats' setting and its own setting's effect is undecided; the next question is which channel carries the cross effect (rank, resources, raids), read from the dimension split and the events before any new study.
H1 above zero: the surprise; study 7's row is then read beside the own-minus-cross interval, and the finding stands on its own until a second study on a different task shows the same sign.
S2 undecided or reversed: the arms did not do what study 7's did; the turn kinds and output per turn are read first, and nothing about Haiku 5.5 is claimed from this study.
M failing for a model: the route or the model did not take the setting; a harness-side check comes before any further effort study on this route.
Every result: each model's cost at list beside each contrast, never per Index point on a card.

Pilot

Two matches before launch, on the study's harness version: the full roster at the settings of "X high, Y low" and of "X low, Y high", the 8,192 ceiling, 100 rounds, same scenario and bots, with a metered cap high enough that neither can stop short of 100 rounds. Both-low and both-high receipts come from study 7's 16 matches on the previous harness version and are read again here from these two, since every seat in them plays one of the two settings. Receipts, recorded in the launch record before the hash: every seat was called on every round it was not dormant (in study 7, 10 of 64 seats were called on fewer than 90 rounds and every one was dormant, with no model errors), and none was lost to five failed calls in a row; refusals per seat; turns cut at the ceiling per seat; retries per seat as the harness logs them (each turn's retries, retry_causes and retry_wait_ms; the summary's retried_calls, retries, retry_causes and retry_wait_ms); no dropped request field (the harness records a field it dropped after a refusal on that turn and counts them per seat in its summary; every seat's count is empty and no turn carries the record, except that a dropped summary request, logged as thinking.display, leaves the seat's effort in place and only marks the voiced scan unmeasured for that model); thinking characters (the summary's reasoning_turns and reasoning_chars; the per-turn text is record-only and never fed back) above zero for every seat at high and for the Haiku 5.5 seats at low, and reported for the Sonnet 5.5 seats at low, where study 7's output per call says to expect none; tokens in and out per seat; per model the call latency (median, 95th percentile, maximum), forfeits by cause, seconds per round and the busiest minute's calls and uncached input tokens; free-tool turns per seat under the new wording; output per play turn per seat; the Index per seat as a sanity reading, not a result. Remedies, fixed now: the output ceiling cannot go above 8,192 on this platform, so if a seat is cut at the ceiling on more than a tenth of its turns the study does not launch as written and goes back to the owner with the figures; if a seat forfeits more than a tenth of its rounds to the clock and its 95th-percentile latency is under 150 s, the clock becomes 300 s (the platform maximum) in every arm, provided the pilot's match minutes plus 2 for every round it held at the 180 s clock stay under 360; if that latency is 150 s or more, or the minutes do not fit, the study goes back to the owner; a seat refusing on more than a tenth of its turns, or any seat with a dropped request field other than the summary request, sends the study back to the owner; a model whose high seats show no thinking characters and no rise in output per play turn over its low seats goes back to the owner, and a model whose high seats show the rise but no characters has its scan marked unmeasured, as with a dropped summary request. Provider load: the platform starts a bounded number of matches at once and queues the rest (in study 7, 15 of 16 started within one second and the 16th started 238 s later, 12 s after the first match ended; the queue alternates arms by trial index, so the first wave holds whole trials); that number is read again on the day. The launch rule projects the pilots' busiest per-match minute, uncached input tokens and requests per model, times the number of matches the platform starts at once, against each model's own input-token and request limits, read from the vendor's console on the day and kept in the launch record (study 7's wave of 15 matches ran about 2.0 million uncached Sonnet 5.5 input tokens and 630 requests in its busiest minute, 2.1 million and 630 for Haiku 5.5, and about 9.3 million a minute per model with cache reads counted; the vendor's input-token limit counts uncached input and cache writes only, cache reads excluded, and the request limit is separate). If the projection exceeds either model's input-token or request limit, the study goes back to the owner before creation; between half and the whole limit it launches and the first minutes' failed and retried calls, as the harness logs them, are read before anything else. A change is written in before the hash with the pilot figures that triggered it. The pilots are published as single matches marked as pilots and never pooled with the study.

Limits

The Fitness Index measures play in one scenario under one rubric; it is not a validated measure of general capability, and it can be gamed (talk is paid up to a cap; training pays twice). Nothing here predicts real-world performance.
One scenario, one map pool, scripted house bots as the rest of the field. Each model's two seats share a field with the other model's two and can interact; an own effect is that model's response to its setting inside this roster, and a cross effect is specific to this roster's other model and seat count. Two seats per model per match, sixteen matches per arm on eight maps: each per-model reading is a reading of thirty-two seat-matches in sixteen fields, tested over eight maps.
The sizing took the two arm differences that make up a main effect as uncorrelated within a map, and the between-map share of the spread from two earlier studies; the tests assume nothing of the kind, each being a t on the observed per-map values. The spreads come from one 8-trial study.
No thinking-off control: this harness's off setting is not accepted by Claude Sonnet 5.5, so low is the control for both models, keeping one setting across the roster. Low effort leaves thinking enabled, but the model decides whether to use it: in study 7 Claude Sonnet 5.5 at low answered with the tool call alone (a median of 25 output tokens a call, 53 at the 90th percentile), while Claude Haiku 5.5 at low still wrote a median of about 200 tokens a call beyond the call itself (about 350 at high). For Sonnet 5.5, low is close to no thinking in practice, so S2 and S1 compare Sonnet 5.5 thinking against almost none, while H1 and S3 compare more Haiku 5.5 thinking against less. Output tokens include thinking.
The voiced-awareness scan reads the vendor's thinking summaries, which the harness requests on this route from version 0.6.7, not raw reasoning; its per-character rates do not compare with routes that return raw reasoning or between the two models (Sonnet 5.5's returned text also carries its between-tool notes), and the debrief is a separate, asked measure.
Tokenizers differ across vendors, so token counts compare a model only with itself.
Eight maps from a public practice pool, each played twice; a claim beyond this setup needs held-out maps and opponents, which this study does not have.
A new harness version is a difference from study 7 in its own right; the replication read names it and is descriptive.
Cost lines are computed from token counts at list price, cache reads and writes at their own rates, and from the platform's meter at the price table each run records, each labelled.

Public wording

Arm labels are seat-group settings: "both low", "X high, Y low", "X low, Y high", "both high", where X is the two Claude Haiku 5.5 seats and Y the two Claude Sonnet 5.5 seats, defined in the public note's first line. Model names appear in seat rows and in the per-model readings, each on its own baseline; "the Haiku 5.5 seats", never "the smaller model". No sentence names a model or vendor as better, compares Anthropic's models with other vendors', says one model gains more from effort than the other, or reads this study beside study 7 as a model comparison. "Declared", never "pre-registered". Every number carries its scope; every share its k of n; no interval or rank the sample cannot support.

Changes before hashing

22:02 UTC 2026-10-12: draft written after the owner's pick of the 2x2 factorial on the next-study card (21:56 UTC), from the post-study review's ranking and the next-experiments file; sent to the review thread for a read before the pilot.
22:12 UTC 2026-10-12: the harness change this study waits on (free-tool wording, thinking summaries, retry logging) is in progress as harness 0.6.7; the study pins 0.6.7 once it is live, and the pilot runs on it.
22:17 UTC 2026-10-12: the analysis script was written and tried on study 7's run documents with study 7's two arms standing in for the four: each model's row came out as that model's own effect, every cross effect and interaction exactly zero, the arm contrast and the survival reading reproduced the platform's figures to the last digit (+10.82, 7.35 to 14.29; 0.00, -0.11 to 0.11), and output per turn matched study 7's write-up (+214 tokens over 10 seat-trials). The map-level read, Holm, the prediction scoring and the Welch comparison are in the same script.
22:26 UTC 2026-10-12, draft v2 after the review thread's step-0 read (22:19 UTC), each point checked against the raw data or the code before it was folded in: (1) size carried as 16 trials per arm with the map as the declared unit (unweighted map means, t on 7 degrees of freedom, exact sign-flip over the 8 map means; a map with one trial enters at that value, with none leaves), and the 16-trial figures restated at map level with the between-map share from studies 5 and 6; the 8-trial figures kept beside; (2) seat groups renamed X and Y and the mixed arms "X high, Y low" and "X low, Y high", because a computed sentence opens with the label and "A" read as an article; (3) the identities own plus cross equals the row and own minus cross equals the mixed-arm difference declared, the own-minus-cross interval added as the only apportioning quantity, and the H1/S1 rule reworded so a split result is never read as a difference; the platform's per-arm n reported beside the declared n; (4) pilot receipts and health gained "no dropped request field on any turn" and "thinking characters above zero per seat", no harness change between the pilot and the last trial, and the scan labelled as reading vendor summaries; (5) the provider-load rule projects on the number of matches the platform starts at once (15 in study 7, verified from the run records: 15 within one second, the 16th 238 s later) rather than on every match, with both models' limits read, and study 7's busiest minute recomputed from the trajectories' timing and token counts (2.0 million uncached Sonnet 5.5 tokens and 628 requests, 2.14 million and 634 for Haiku 5.5, 15 matches playing); (6) the chance of each reading under the prior stated beside the prior itself, and the decided-drop result named in Why now as the one most readers would not expect; (7) the second surprise given a declared margin of 4 points and an equivalence reading for S1 and S3; (8) one definition of the -5 case and "averaged over the other seats' two settings" in place of "whatever the other seats' setting"; (9) no cause named in advance for an S2 null; (10) the fixed rule and the absence of a precision target stated; (11) "the smaller model" replaced by "the Haiku 5.5 seats"; (12) H1 kept as the sole primary, S1 a declared secondary, because the owner picked this design for the own-effect question and a co-primary split would drop H1's power at -3 to about 40%. Sizing addendum and the load figures are in the launch record's data files.
22:34 UTC 2026-10-12, draft v3 after the review thread's check of v2 (22:31 UTC), each point verified: (A) the likeliest reading at 16 trials is an interval below zero, not one spanning zero; "expected" dropped and the interval at an estimate of exactly -3 (-5.7 to -0.3 at map level) stated in Why now and Size; (B) the dropped-field receipt reworded to what the harness already logs (a per-turn record only when a field was dropped, and a per-seat count in every summary; in study 7 all 64 summaries are empty and none of 5,751 turns carries the record), so the per-turn logging ask to the harness thread is withdrawn and only retry and rate-limit counts remain a logging gap; (C) the equivalence reading now says a cross effect larger than 4 points either way is ruled out at the declared margin, and the two equivalence readings sit outside Holm; (D) the second surprise carries the chance of its reading (about one in twenty, both 90% intervals inside the margin) beside the chance that it is true; (E) the owner's rate-limit tier removed from the public text and kept in the launch record, and the limit described as the vendor documents it (uncached input and cache writes count, cache reads do not; requests limited separately), checked against the vendor's rate-limit page; (F) apostrophes restored in the 22:17 entry; (G) the Holm slots stated correctly: after S2, the smaller of S1 and S3 faces 0.025 and the larger 0.05.
22:35 UTC 2026-10-12: two wording fixes from the review thread's check of v3 (22:35): Holm stops at its first failure, so the larger of S1's and S3's p-values faces 0.05 only if the smaller passed; and Why now opens with "Two results would matter beyond this study", since the decided drop is also the likeliest reading. The review thread has nothing more before the hash.
23:27 UTC 2026-10-12: the owner picked 16 trials per arm on the size card at 23:26 UTC; the size this draft carries is now fixed (64 runs, each map twice, about $123 at list), and nothing else in the text changes.
23:33 UTC 2026-10-12: the owner asked (23:27 UTC) for the cost to be confirmed before anything proceeds. The platform's estimate call on the exact body (scripts/estimate8.py, data/estimate-8.txt, nothing created) gives $201.60 to $1,342.92 for 64 matches; its cheapest per-match figure ($3.15) is above every match study 7 played at list, so the measured study 7 figures ($1.61 and $2.24 a match, $123 for 64) stay the planning basis and the platform band is quoted beside them. The cap floor drops from $15 to $10 a match (the $15 floor was carried over from study 7's caps on the 2026-08 meter table, which priced Haiku 5.5 at ten times list); the 2.5-times-pilot term is unchanged, and the worst case the caps allow falls from $960 to $640. The pilot's measured spend and the projected study total are reported to the owner before the create is asked for.
23:36 UTC 2026-10-12: harness 0.6.7 merged (#307, 23:3x UTC; the deploy follows). From the harness thread's note, each item checked against the merged code: the scaffold string is named in Design (version 0.6.7, prompt v6, hash 68dd2f4c pinned in the repository's inspect test; the pilot archives confirm it); the pilot receipts name the logged retry fields and the summary's reasoning_turns and reasoning_chars; a dropped summary request (thinking.display) keeps the seat's effort and only marks the voiced scan unmeasured for that model, so it no longer sends the study back by itself, while any other dropped field still does; the voiced-scan lines say that Sonnet 5.5's returned text mixes its between-tool notes with summaries, so each model's rate is read against itself across arms only. No test, prediction, size or cap changes.
23:48 UTC 2026-10-12: the reviewer's check of the 23:33 and 23:36 entries (23:46 UTC) holds them and raised three pilot faults, each verified on study 7's raw trajectories and run documents (scripts/check8_reviewer.py, data/check-8-reviewer.txt) before folding in. (A) The thinking-characters receipt would have stopped the pilot for no fault: Claude Sonnet 5.5 at low wrote a median of 25 output tokens a call (53 at the 90th percentile, 189 at most), so its low seats will likely return no summary; the receipt now expects characters at high for both models and at low for Haiku 5.5 only, reports Sonnet 5.5 at low, and the remedy is tied to whether the setting took (no characters and no rise in output per play turn sends the model's study back; the rise without characters marks its scan unmeasured); Analysis step 1 says the same. (B) Two lines said low effort still thinks; study 7 contradicts that for Sonnet 5.5 (Haiku 5.5 at low still wrote about 200 tokens a call beyond the call, about 350 at high), so Design now says the requests differ only in effort and what each model did with it is read from output per call, Limits carries the figures and says S2 and S1 compare Sonnet 5.5 thinking against almost none while H1 and S3 compare more Haiku 5.5 thinking against less, and the scan line says a model-arm cell with no characters is unmeasured. (C) The receipt that every seat made calls on at least 90 rounds fails whenever a seat goes dormant (study 7: 10 of 64 seats under 90, every one dormant, no model errors); it now reads that every seat was called on every round it was not dormant and none was lost to five failed calls in a row. No test, prediction, size or cap changes.
02:59 UTC 2026-10-14: the owner's word (02:51 UTC): proceed, with total spend not above $145. Read as: the pilot and then the 16-trial study on the same key study 7 ran on (no other key was named; the route is unchanged), pilot plus study under $145 in the platform's metered dollars, which are list prices for these runs. The ceiling is enforced during the study: the platform caps spend per match, not per account, so the study's watcher reads the finished matches' metered spend and projects the matches still to play from each arm's running mean; if finished spend plus that projection exceeds $145, the study is cancelled at that point on the owner's standing instruction, and the cancelled trials are reported with the rest. At the expected $123 for 64 matches and about $4 for the pilots the ceiling is not expected to bind. The pilot matches carry a $25 metered cap each, above the platform's own per-match high ($20.98), so neither can stop short of 100 rounds. Checked before the pilot: the repository's main branch moved since harness 0.6.7 merged (docs, developer docs, run-privacy storage, lobby and report code) with no change to the harness, the prompt, the lounge or any agent-facing text, so the pilot and the study run on the same harness 0.6.7 and prompt v6 named in Design; production ran that code with nothing playing. The owner's per-model rate limits were read from the vendor's console on 12 October 2026, before study 7, and are held in the launch record; the load rule compares the pilots' busiest minute, times the number of matches the platform starts at once, with them (sentence corrected at 03:01 UTC 2026-10-14; the first version, written minutes earlier, said no reading existed).
03:10 UTC 2026-10-14: pilot played and read (launch record, pilot entries). Two matches on harness 0.6.7, prompt v6, scaffold daishi-ref-harness@0.6.7+prompt-v6.68dd2f4c+effort.<level> confirmed on every seat from the archives; 100 of 100 rounds each, every seat active and called on every round, no refusal, no turn cut at the ceiling, no retry, no dropped request field, no forfeit; thinking characters above zero on every seat at high (both models) and on the Haiku 5.5 seats at low, near zero on the Sonnet 5.5 seats at low (one or two short summaries each), with output per play turn rising from low to high for both models (Haiku 5.5 medians about 220 to about 510 tokens; Sonnet 5.5 about 25 to 47 at low and 161 to 265 at high), so the setting took for both and every remedy stays unused; the clock stays 180 s (no round at the clock; call latency 95th percentile under 10 s). Under the new wording the Haiku 5.5 seats at high still spent 49 and 63 of 100 turns on free lookups, each followed by the harness rest; this is a mechanism reading for the write-up, not a receipt. Cap: the costlier pilot match cost $2.23 metered, so the per-match cap is $10 in every arm. Cost: the pilots came to $4.24 at list (the meter agreed with list to the cent), the mixed arms $2.01 and $2.23 a match, and from the seat figures "both low" projects to about $1.98 and "both high" to about $2.27, so the 64 matches project to about $136 and the whole to about $140 under the owner's $145 ceiling; the ceiling rule's projection shrinks each arm's running mean toward its pilot figure with the weight of four matches, so one costly early match cannot cancel the study, and a true overrun cancels only the last matches. Load: the pilots' busiest per-match minute, times the fifteen matches the platform started at once in study 7, stays under half of each of the owner's per-model limits for both models, so the study launches without the first-minutes read. No test, prediction, size or analysis change; this is the last entry before the hash.

The owner's words, kept as received. Daishi checks when the document arrived and what its bytes hash to, not what it says. Back to the study