Scripts, agents and notifications

#Developer access

Scripts and agents act on an account through access tokens (Account, then Developer): the REST API under /api/v1 and the Studio MCP server at /mcp/studio share one validation path with the run builder, so a scenario an agent authors over MCP is held to the same rules as one saved by hand. Tokens are shown once, stored hashed, scoped, and optionally expiring, up to the plan's count; minting one needs a signed-in browser session, so a token never mints a token. An MCP client that supports sign-in needs no token at all: add /mcp/studio to it, approve it on a Daishi page that shows what it asks for, and it is listed under Account, then Developer, as a connected app. A connected app counts toward the same allowance as a token, works on the MCP server only, never sees your email address, and stops working the moment you disconnect it. Sign out everywhere and a password reset revoke every token and disconnect every app at once. The developer docs are the reference, and the Plans table says which plans may write through it. Scripts can skip raw HTTP with the client libraries: pip install daishi for Python and npm install daishi-sdk for TypeScript and JavaScript, one typed method per endpoint, with retries and helpers that wait for a run to finish. To have an AI assistant design, run and read a study for you, install the study design skill.

Onboard your agent, at the top of the Studio's Overview once you are signed in, connects the AI coding tool or chat app you already use without a token. Its marks show the coding tools the setup lines cover: Claude Code, Codex, Cursor, Gemini CLI, VS Code and Windsurf. One click copies a setup prompt. Pasted into a coding tool, it adds the Studio MCP server, starts the sign-in for you to approve, and checks the connection with whoami before it does anything else. A chat app cannot change its own settings, so it lists where to add the server as a custom connector, then makes the same check when you come back and say whoami. The prompt carries no key or token, and it asks your agent to estimate before any launch and to launch, cancel, delete or publish only when you say so. The X beside the button, which appears when the pointer is over it and always shows on a touch screen, puts it away in that browser; the same button stays under Account, then Developer, with the full prompt to read before you paste it.

#Reports for scripts and agents

Everything the Studio shows about a batch or an experiment is also a JSON report a script or an agent can read, and the per-turn record each seat wrote can be paged through the same way:

• Batch reports. GET /api/v1/batches/<id> over the developer API, or the batch_report tool on the Studio MCP, returns a series as one report: per model the 95% interval with the mean, the interquartile mean and survival as a rate, every trial's per-seat outcome and seeds, the cost so far and the verdict sentence, exactly as the batch page prints them.
• Experiment reports. GET /api/v1/experiments lists an account's experiments with their status and verdict, GET /api/v1/experiments/<id> returns the full report (arms, pairs, comparisons, the rule and the decision) with every trial's record grouped by arm, the card's plain sentence (plain) and what each arm changed from the control (what_differed), and POST /api/v1/experiments/<id>/cancel stops one, with your reason and, if you have one, the experiment that replaces it; POST /api/v1/experiments/<id>/stop-reason gives the reason once to an experiment that ended without one. POST /api/v1/experiments/<id>/note adds an entry to your note, POST or GET /api/v1/experiments/<id>/design attaches or reads the design document, the read with the outside witnesses' proofs of its digest and every earlier version, and GET /api/v1/experiments/<id>/creation reads the creation record with the proofs of its digest. The MCP tools are list_experiments, get_experiment, estimate_experiment, create_experiment, cancel_experiment, set_experiment_stop_reason, add_experiment_note, attach_experiment_design, get_experiment_design and get_experiment_creation.
• Runs, seats and models. GET /api/v1/runs pages through the whole run history, newest first: send next_cursor back as cursor for the next page. GET /api/v1/runs/<id>/seats (run_seats on the MCP) returns what a run's Seats section shows, seat by seat, with the trajectory file to read, and GET /api/v1/models (get_models) returns the figures the Models page draws.
• Trajectories. GET /api/v1/runs/<id>/logs lists the files kept for a run (per seat, a trajectory with one line per model call, the observation sent, the reply and the action that ran, and a usage record), and GET /api/v1/runs/<id>/logs/<file> returns one. On the MCP, list_trajectories lists them and read_trajectory pages through one by line offset, up to a thousand lines a call, so an agent can audit a finished run without pulling the whole file.
• Publishing. POST /api/v1/runs/<id>/publish, with an optional note, does what the share block's Publish to profile does, and POST /api/v1/runs/<id>/unpublish undoes it; the MCP tools are publish_run and unpublish_run. POST /api/v1/experiments/<id>/publish and POST /api/v1/batches/<id>/publish do what Publish study does, every trial that played in one call, and answer study_playing while a trial is still queued or playing; each has an unpublish twin, and on the MCP they are publish_experiment, unpublish_experiment, publish_batch and unpublish_batch. All of them, and writing the declared result, need runs:publish, which no token or app gets unless the owner ticks it when minting the token or approving the app. runs:write never includes it, so an agent that may launch cannot speak for the owner in public unless they chose that too.
• Run sets. POST /api/v1/sets creates a set (its name, the setups runs are filed under, a note and whether the card prints each seat's own text), POST /api/v1/sets/<id>/members and .../members/remove file finished runs and whole series under a setup or take them out, POST /api/v1/sets/<id>/setups/<setup>/design attaches a setup's design document, DELETE /api/v1/sets/<id> deletes the set (its runs stay), and GET /api/v1/sets and GET /api/v1/sets/<id> read them, all with runs:write or runs:read. Changing the note, the labels or the seat text switch (POST /api/v1/sets/<id>) and publishing or unpublishing (.../publish, .../unpublish) need runs:publish, and so does any change to the runs or designs of a published set, or deleting one, since it changes what the profile shows. The MCP tools are create_run_set, list_run_sets, get_run_set, update_run_set, add_run_set_members, remove_run_set_members, attach_run_set_design, get_run_set_design, publish_run_set, unpublish_run_set and delete_run_set.

All of it is scoped to the account that owns the runs and reads with a token carrying runs:read; creating or cancelling an experiment, giving its stop reason or attaching its design, and building a run set need runs:write and a paid plan, and publishing and adding to an experiment's note need runs:publish and a paid plan. The shapes, the error codes and the client setup are in the developer docs.

#When a run ends

A long series is something to be told about, not to poll, and an agent operating the lab on its own needs a callback rather than a browser tab. Under Account, then When a run ends, an account can set two things:

• A webhook address (up to 512 characters). The platform POSTs a signed JSON notice there when a unit of work ends. Only https:// addresses on public hosts are accepted: no username or password in the address, and no localhost, private, loopback or link-local host. The host is checked on the addresses its name resolves to, not only on how it is written: a name that resolves to any such address is refused when you save it, and it is resolved and checked again before every delivery, which connects only to the addresses that check approved. Leave it blank to send nothing.
• Email: never, when a series or experiment ends (the default), or after every run. Mail goes only to a verified address.

Send a test (one every ten seconds) posts a test event to the address at once, with the same envelope and signature as a real notice, and shows the answer, so a receiver can be checked before anything finishes. The card also shows the last delivery: when, which event, and the status or the error.

#The events

One notice per unit of work:

• run.ended for a run launched on its own (a trial inside a series is not announced by itself; its series is).
• batch.ended once, when the last trial of a series turns terminal.
• experiment.ended once, when a paired experiment's stopping rule or its last trial ends it. A run inside an experiment is announced through the experiment, never on its own. An experiment created with a callback_url sends this notice to that address instead of the account's; the email follows the account's choice either way.
• test, only from Send a test: its block is test, with a message in place of a run, batch or experiment.

The body is a JSON object with type, delivery_id (unique per notice), at_ms (milliseconds since the epoch), and one block named after the kind of work:

• run: id, name, status (finished, failed or cancelled), stop_reason, batch_id, experiment_id, url.
• batch: id, name, trials, finished, failed, cancelled, url.
• experiment: id, name, status, stop_reason, url.

Batches and experiments also carry verdict: the report's own reading in a few plain sentences, the limitation first, exactly as the report prints it. url is the absolute link to the thing in the Studio. Nothing from inside a match travels in a notice: no seat text, no messages, no scores per turn. A series that ended, for example:

{
  "type": "batch.ended",
  "delivery_id": "dlv_5f0c2a9e7b1d4c38a2",
  "at_ms": 1790526270562,
  "batch": { "id": "bat_ec3a38c28afdfb5c42", "name": "Famine, 10 trials", "trials": 10,
             "finished": 10, "failed": 0, "cancelled": 0,
             "url": "https://daishi.ai/studio#batch=bat_ec3a38c28afdfb5c42" },
  "verdict": "The widest interval here is plus or minus 5.9 index points, about a grade band, so only large gaps between models mean anything yet."
}

#Verifying a notice

Each POST is content-type: application/json and carries three headers: X-Daishi-Event (the event type), X-Daishi-Delivery (the delivery_id) and X-Daishi-Signature, which is sha256= followed by the hex HMAC-SHA256 of the exact request body under your account's signing secret. The key is the secret exactly as the card shows it, as text (its UTF-8 bytes), not decoded from hex. The secret is shown on the card (Reveal) and is derived for your account rather than stored, so Rotate replaces it at once, with no overlap: the new secret exists only from that moment, and a notice sent before your receiver has it fails the check. A refusal is a 4xx, which ends that notice's retries, so rotate when nothing is about to end, paste the new secret into your receiver straight away, and press Send a test to confirm the receiver takes it. A notice lost this way loses nothing else: the run, series or experiment it would have announced is in the Studio and the API as always.

Verify it the same way in any language:

raw      = the request body, as bytes, before any parsing
expected = "sha256=" + hex(hmac_sha256(key = secret, message = raw))
if not constant_time_equal(expected, header["X-Daishi-Signature"]): refuse
notice   = parse_json(raw)
if now_ms() - notice.at_ms > 5 minutes: refuse
if notice.delivery_id was accepted before: answer 2xx and do nothing
remember notice.delivery_id for at least 5 minutes

Compare the two strings in constant time, and compute the HMAC over the raw bytes, not over a re-serialized copy of the parsed JSON. The signature proves who wrote a body, not when: anyone who has seen a notice, in a log or at a relay, can post the same bytes again and the signature still checks. at_ms and delivery_id are inside the signed body, so they cannot be changed without breaking it, and the checks after the signature stop such a replay: a copy older than a few minutes is refused, and a delivery_id you have already taken is not acted on twice. Every retry of a notice carries the same at_ms and delivery_id, and the last retry leaves about three minutes after at_ms, so a five-minute window takes every genuine attempt as long as your clock is right. Answer with any 2xx status once you have taken the notice; a body is not needed.

#Delivery and retries

A delivery times out after ten seconds. On a network error, a timeout or a 5xx answer the platform tries again after 5 seconds, then 25, then 125, four attempts in all; a 4xx answer is taken as your receiver's decision and stops the retries. A host name that does not resolve counts as a network error. One that now resolves to a loopback, private or link-local address gets no connection at all: the delivery is recorded as refused and not retried. Notices are sent once each and may arrive out of order; delivery_id tells a repeated attempt from a new notice. A notice that never gets through is logged on the platform and shown as the last delivery on the card; it never affects the run it reports on.