Control Channel

Overview

Every inspect eval or inspect eval-set process binds a local control endpoint that exposes the live state of the run. The inspect ctl commands connect to it from another terminal, so you can check on a long-running eval — progress, stalled samples, errors, transcript activity — and direct it — cancel a stalled sample or a whole task, retune concurrency limits and log buffering — without parsing log files.

Commands are grouped by resource noun (task, sample, model, process), plus a top-level config command:

Command	Description
inspect ctl task list	List running tasks across all live Inspect processes.
inspect ctl task log-flush	Write buffered completed samples to the log now.
inspect ctl task cancel	Cancel a running task.
inspect ctl task drain	Drain a running task (stop new samples; in-flight samples finish naturally).
inspect ctl task pause	Pause a running task (stop starting new work; --now also holds in-flight samples).
inspect ctl task resume	Resume a paused task.
inspect ctl task score	Score a running task’s samples now and report interim metrics.
inspect ctl sample list	List samples (running, completed, and pending).
inspect ctl sample errors	List samples that errored or were retried.
inspect ctl sample show	Show one sample’s summary and error history.
inspect ctl sample events	Read one sample’s transcript events.
inspect ctl sample messages	Read one sample’s current conversation.
inspect ctl sample store	Read one sample’s current store (shared solver/agent state).
inspect ctl sample cancel	Cancel one sample — running or not yet started.
inspect ctl sample cancel-tool-call	Cancel one hung tool call and let the sample continue.
inspect ctl sample requeue	Re-run one errored or cancelled sample inside the live run.
inspect ctl sample score	Score one sample’s work-so-far now, without ending it.
inspect ctl config	View or retune launch configuration mid-flight.
inspect ctl model pause	Pause one model’s dispatch across the run (--now also holds in-flight calls to it).
inspect ctl model resume	Resume a paused model.
inspect ctl process list	List running Inspect processes.
inspect ctl process anomalies	Show in-flight and anomalous actions from a process’s trace log.
inspect ctl process keep	Make a process stay alive after its eval finishes.
inspect ctl process release	Let a keep-alive process exit.
inspect ctl process pause	Pause a whole running eval or eval-set (--now also holds in-flight samples).
inspect ctl process resume	Resume a paused eval or eval-set.

A bare noun implies list: inspect ctl task ≡ inspect ctl task list, and likewise for sample and process. All commands accept --json for structured output, which makes them straightforward to use from scripts and from coding agents like Claude Code. The list / show / errors / events / messages / store / anomalies commands are read-only. The others direct the run deliberately: config retunes launch parameters (concurrency changes never interrupt in-flight work; the per-sample limit overrides deliberately reach in-flight samples), task log-flush forces a log write that would happen anyway, process keep/release only affect what happens after the eval finishes, task pause/resume, model pause/resume, and process pause/resume stop and restart dispatch reversibly — leaving in-flight work untouched unless pause --now asks for a hard pause (which also holds in-flight samples at their next model call), task cancel / sample cancel interrupt work explicitly, task drain stops new samples while in-flight ones finish naturally, sample requeue re-runs an errored or cancelled sample (all idempotently, with --dry-run support), and task score / sample score score a running task’s samples (or one sample) without ending them (each in-flight sample is briefly held while its work-so-far is scored).

The endpoint is a Unix domain socket under the current user’s Inspect data directory. It is not reachable over the network or by other users on the same machine, and it requires no configuration.

Launch Handoff

Right after launching an eval, an empty inspect ctl task list is ambiguous: the control endpoint may simply not be bound yet. A script or agent that launches an eval and then drives it with inspect ctl should launch with --json:

inspect eval ctf.py --json

This implies --display none and makes stdout machine-readable — the process emits JSON lines (and nothing else) on stdout:

{"event": "launch", "run_id": "Ngkz4viFYq…", "eval_set_id": null, "pid": 17146, "log_dir": "/…/logs", "control": {"socket_path": "/…/control/17146.sock"}}
{"event": "done", "run_id": "Ngkz4viFYq…", "logs": [{"task": "ctf", "task_id": "…", "eval_id": "…", "status": "success", "location": "/…/logs/…_ctf_….eval"}]}

The launch record is printed only once the control endpoint is bound (and before any task work begins), so reading it is a hard guarantee: from then on an empty inspect ctl task list means “no tasks registered yet”, never “no server”. control is null exactly when the control surface is definitively absent (disabled via --ctl-server=false, or the bind failed and the eval degraded to running without it). A process that exits without emitting a launch record failed before the control server came up — the reason is on stderr.

The done record arrives when the eval finishes, with each task’s log location and status — the handoff from live observation to reading logs. A run that crashes (raises out of the eval) emits no done record and exits non-zero. Note that a task error is not a crash: like plain inspect eval, the process still emits done and exits 0 with that task’s status set to "error" — branch on the status fields in logs, not the exit code (or use eval-set, whose success field and exit code do reflect per-task outcomes).

Stdout carries these records exclusively: the eval itself runs with stdout redirected to stderr (at the file-descriptor level, so even output from subprocesses spawned by task or solver code lands on stderr rather than corrupting the stream).

inspect eval-set --json follows the same contract, with eval-set specifics: the records also carry eval_set_id, and the done record adds an overall success field mirroring the exit code. Two deviations from “exactly one launch, then one done”: a set whose tasks are all already complete runs no eval, so stdout carries only the done record (don’t read the missing launch line as a failed launch once done arrived) — except under --ctl-server=keep, where the keep-alive park still binds a control endpoint and reports it with a launch record whose run_id is null; and legacy batch-mode retries (--no-retry-immediate) emit a fresh launch record per retry batch, with the done record carrying the last launch’s run_id — the run that produced the final state.

inspect eval-retry --json follows the contract too, with one wrinkle: each retried log file runs as its own eval with its own run_id, so retrying multiple log files emits one launch record per file (sequentially — each supersedes the previous), and the single done record carries the last launch’s run_id with one logs entry per retried task.

Detached Launch

An eval launched with --json still occupies the terminal until it finishes. To run it in the background — detached from the terminal, surviving the launching shell (or agent session) ending — launch with --detach:

inspect eval ctf.py --detach

This implies --json. The command blocks until the control endpoint is bound, prints the launch record on stdout, and exits 0 — the eval keeps running as a detached background process:

{"event": "launch", "run_id": "Ngkz4viFYq…", "eval_set_id": null, "pid": 17146, "log_dir": "/…/logs", "control": {"socket_path": "/…/control/17146.sock"}, "output_file": "/…/detach/20260716-141530-ab12cd34.out"}

The launch-handoff guarantee carries over unchanged: a --detach command that exited 0 has emitted a launch record and the control surface exists — except when an eval-set’s tasks are all already complete, in which case no eval runs, stdout carries only the done record, and nothing is left running (the set’s results were already final). One that exited non-zero has not started a background eval, and the pre-flight diagnostic (bad task path, missing API key, …) is on stderr. There is no third state to poll for: if the control endpoint fails to bind, the launcher terminates the eval and exits non-zero rather than leave an unmonitorable eval running, and interrupting the wait (Ctrl+C or SIGTERM) likewise terminates the eval before the launcher exits.

The detached process’s stdout and stderr go to a file under the Inspect data directory, reported as output_file in the launch record. The process exits on its own when the eval finishes, and that file’s last line is the completion signal — after the handoff, the eval’s terminal and results are read entirely through the surfaces this page documents:

1. Monitor the running eval with inspect ctl task list --json (and drill down with inspect ctl sample list / errors / events).
2. Intervene if needed: inspect ctl sample cancel, inspect ctl task cancel, inspect ctl config.
3. Detect completion: when the eval finishes the process exits (dropping out of the inspect ctl listings), leaving a done record — overall success plus each task’s status and log_location — as the last line of output_file.
4. Detect a crash: a process that is gone without a done record in its output file died mid-run; the same file holds its diagnostics (stray prints and stderr land there too). For an eval-set, re-running the same command retries the unfinished tasks.

Inspect never deletes these output files: one accumulates per detached run under the data directory until you remove it. Since the file is the completion signal, remove it only after its done record has been read.

To instead keep the process alive after the eval finishes — its state still queryable via inspect ctl until you inspect ctl process release it — pass --ctl-server=keep explicitly (or latch it onto an already-running detached eval with inspect ctl process keep). The done record is then written only when the process is released. Prefer the default exit-when-done unless whatever will issue the release is certain to outlive the eval: a long eval routinely outlives the shell or agent session that launched it, and an unreleased parked process lingers indefinitely.

--detach works the same on inspect eval-set and inspect eval-retry (a multi-file retry hands off on its first launch record; later files’ records go to the output file). Because a detached eval must be observable and cancellable while running, combining --detach with --ctl-server=false is an error.

To make an agent (Claude Code or similar) use this workflow for long-running evals, install inspect-skills, or paste a snippet like this into your eval repo’s CLAUDE.md / AGENTS.md:

For evals that may run longer than a few minutes, do not run
`inspect eval` in the foreground or under nohup/tmux. Instead:

1. Launch with `inspect eval <task> --detach`. It prints a JSON
   `launch` record and returns, leaving the eval running in the
   background; non-zero exit means the launch failed (reason on
   stderr). Never consider an eval launched until you have read
   its `launch` record.
2. Poll `inspect ctl task list --json` to watch progress. When the
   eval finishes its process exits and drops out of that listing;
   completion is confirmed by the JSON `done` record on the last
   line of the launch record's `output_file`, which reports overall
   success and each task's `status` and `log_location`. If the
   process is gone and there is no `done` record, the run died
   mid-flight — diagnostics are in the same file.
3. If samples stall or error, inspect with `inspect ctl sample
   list` / `inspect ctl sample errors`, and cancel with
   `inspect ctl sample cancel` / `inspect ctl task cancel`.
4. Read results from each task's `log_location` (reported in both
   the `done` record and `inspect ctl task list`).

Listing Tasks

inspect ctl task lists the tasks of every running eval on the machine:

$ inspect ctl task
task_id       task                        model                      solver    samples             started
------------  --------------------------  -------------------------  --------  ------------------  --------
ZByxJpK4bKSz  inspect_evals/gpqa_diamond  openai/gpt-5               react     12/40 (3 running)   14:02:11
fR8mWn2cQspD  inspect_evals/humaneval     anthropic/claude-sonnet-5  generate  164/164 (complete)  13:58:40

Each row is one task: retried tasks stay on a single row (with an attempts column showing how many attempts have run), and an errors column appears when any samples have errored. The solver column shows the plan’s terminal solver (the agent name, e.g. react, for an agentic task). With --json, the response is an {as_of, tasks} envelope and each task row also carries pid, socket_path, and log_location (where results are being written — the handle for reading logs after the run).

Two more columns appear only when they have something to report: refusals (model refusals) and http_retries (rate-limit and transient HTTP retries). Both are running totals over the task’s own samples — the finished ones plus the live counts of those still in flight — so they are readable mid-run rather than only at the end. Both are always present in the --json row. They count every attempt, so a sample retried under retry_on_error contributes what each attempt saw; these are counts of things that happened, not properties of a final state. Note http_retries is unrelated to attempts (whole-task retries) and to a sample’s own retries (failed attempts of that sample).

These are the same events the TUI footer tallies, but attributed per eval rather than per process — which is what makes them usable from inspect ctl at all, since one process commonly runs several evals at once and a detached run has no display to print a footer. An event reported outside any sample is counted in the process-global total only, so it appears in the footer and in no task row.

A task is finished exactly when completed_at is non-null; status (running / completed) is derived from it. Don’t infer completion from sample counts — a cancelled or errored eval finishes with completed < total.

Selecting a Task

Commands that operate on one task take a TASK argument that selects a task from this list. It matches a task id (or unique prefix) first, then a task name — anchored at the start of the name or after a /, so gpqa matches inspect_evals/gpqa_diamond. When only one task is running you can omit it entirely.

Task ids are stable across retries, so a command keeps working after a task errors and is retried (per-attempt eval ids are not stable, which is why commands don’t use them).

One task run against several models (--model openai/gpt-5,anthropic/claude-fable-5) makes the task name ambiguous — it matches both rows. Pass --model alongside the selector to pick the row running that model (matched at the name start or after a /, so --model gpt-5 matches openai/gpt-5): inspect ctl task cancel my_task --model gpt-5. The model filter applies within the tasks the selector matches — what other tasks happen to be running can’t change the outcome. If the combination is still ambiguous you get the same candidate-table error, and a combination matching nothing the same not-found error — each qualified with the model. An exact task id paired with a --model its task isn’t running errors by naming the contradiction (the task’s actual model) rather than being silently ignored.

On reads (sample list, sample errors) the selector is a filter: omitting it lists across all running tasks (each row carries its task_id), which makes “what’s erroring anywhere in this eval set?” the zero-argument spelling. On mutations (task log-flush, the task-scoped config knobs) an omitted selector must resolve to exactly one target — the sole running task is the default, and anything ambiguous errors with the candidate list rather than fanning out. Destructive commands (task cancel, task drain) require the selector outright.

Sample Status

inspect ctl sample list lists samples with their live status:

$ inspect ctl sample list gpqa
inspect_evals/gpqa_diamond (ZByxJpK4bKSz)  ·  openai/gpt-5  ·  running  ·  12/40 (3 running)

sample  epoch  status     time   idle  activity         tokens  messages  turns
------  -----  ---------  -----  ----  ---------------  ------  --------  -----
14      1      running    12:40  0:03  bash 0:41        48210   22        11
17      1      running    8:12   6:51  generating 6:51  31055   14        7
21      1      running    0:45   0:33  generating 0:33  2150    3         1
1       1      completed  4:02                          18021   9         4
...

The idle column shows how long since a running sample last showed activity — its most recent transcript event, or streamed progress on an in-flight model call where the provider call streams. A long-running sample with high idle time is the cheap signal that it may be stalled. When an in-flight model call streams, its progress keeps idle near zero (so climbing idle on a streamed call is a sharp stall signal); a non-streamed call produces no signal until it returns, so idle still accumulates there — the activity column is what distinguishes that healthy case from a genuine stall.

The activity column (shown when any running sample has an in-flight operation) names what the sample is doing right now and for how long: generating 6:51 for an in-flight model call (with (2 retries) appended when the provider SDK has retried within the call, and · 1.2k tok when the provider call streams cumulative output tokens), bash 0:41 or 2 tools 1:10 for pending tool calls, retrying in 0:45 when a model call is waiting out a retry backoff (e.g. rate limiting) between attempts, and approval: bash 6:12 or question 2:03 when the sample is parked waiting for a person (see Human Approval and ask_user()).

A pending human interaction takes precedence over anything else the sample has in flight, and it is the only thing that reports the wait at all: an approval is awaited before the tool call’s event is recorded, so a sample parked overnight has nothing pending in its transcript and would otherwise read as silently idle. If you are watching a run from another process, this is the signal that tells “working” from “stopped on you”. The elapsed time is the wait’s own, not the sample’s.

The --json rows carry the underlying activity object — type (model / tool / retry_wait / approval / question), count, started_at, detail (model name, or the tool function an approval is deciding; empty for a question, whose prompt is the request), retries, deadline (when a retry wait elapses), streamed-progress fields tokens / last_progress_at (null for non-streamed calls and providers not reporting them), and — on tool rows, and on a human wait that sits inside a running tool — calls, one entry per pending tool call (see Cancellation) — null on rows with nothing pending.

The turns column counts top-level model generations (blank when unknown, e.g. for samples logged by older versions of Inspect). When any listed sample has a token limit configured, limit usage and limit total columns are also shown: the metered value for that limit — respecting its type (all, output, or a formula) — against the configured ceiling. The --json rows carry these as turn_count, token_limit_usage, token_limit_total, and token_limit_type.

The listing is capped at 100 rows per task by default, keeping the head of the running → terminal → pending sort order (running samples sort first, any queued-but-not-started ones just after, then finished ones — completed, error, and cancelled alike — so the cap keeps the most relevant rows and errored samples survive it alongside completed ones). A capped listing says so — the human output prints a listing capped: showing N of M samples footer, and the JSON envelope sets truncated: true — and the aggregate answer stays complete regardless: the envelope’s counts is the status histogram over all of the task’s samples. Adjust with:

Option	Description
--limit N	Cap the listing at N rows per task instead of 100.
--all	List every sample row (no cap).
--status running,error	Only samples with these statuses (running, completed, error, cancelled, pending, queued). Filters rows only — counts stays whole-task.
--content	Include each row’s error message and limit_reason in the --json rows (agent-influenced free text — withheld by default; see Agent-controlled content below).

With --json the response is an {as_of, counts, samples, truncated} envelope. Pass --active-since <timestamp> to get only the samples that started or changed since a previous poll — feed it the as_of from the prior response (rather than a locally minted timestamp) so nothing that changed mid-read is missed (counts remains the whole-task histogram on a delta poll, so progress tracking rides along for free). The row cap applies to delta polls too, and the rows it drops are typically the terminal ones (running rows sort first and survive the cap) — samples that completed or errored in the window and will never produce activity again, so they won’t match a later --active-since. If a delta poll comes back truncated, re-issue it with the same --active-since plus --all (or a higher --limit) before advancing to the new as_of; otherwise the dropped changes leave the feed permanently.

The cap is enforced by the eval process’s control server, so an inspect CLI from before the cap (≤ 0.3.245) reading a newer eval will see the capped listing without the footer or the truncated flag. Keep the observing CLI at least as new as the eval it inspects (a newer CLI reading an older eval handles the difference automatically).

Errors and Retries

inspect ctl sample errors is a triage view of the samples that errored or were retried (across all running tasks when TASK is omitted):

$ inspect ctl sample errors gpqa --content
sample  epoch  status   retries  error
------  -----  -------  -------  ----------------------------------
9       1      error    2        RuntimeError: tool execution failed
17      1      running  1

Error messages are withheld unless you pass --content — a tool-raised exception can embed the agent’s own output, making the message agent-controlled text (see the callout below). Without the flag the rows still carry status and retry counts, which is enough for most triage.

NoteAgent-controlled content

Free text that originates from the evaluated agent — model completions, tool arguments and results (a bash tool’s stdout is fully agent-controlled), error messages that embed tool output, and a sample’s limit_reason (a bridged agent supplies its own termination reason) — is untrusted content: it can carry prompt-injection payloads aimed at whoever reads it, including an LLM monitor polling these commands. The per-sample reads therefore return metadata only by default (statuses, timing, token counts, event types, tool function names, error presence), and every command that can carry agent text gates it behind an explicit --content (or content=true on the HTTP API) opt-in.

This makes the safe consumer the effortless default: a monitoring agent that never passes --content (and has no other route to the content, such as reading the logs directly) never ingests agent-controlled text, so it cannot be prompt-injected by the evaluated agent. A monitor that does read content should be treated as injection-exposed and sandboxed / capability-limited accordingly.

inspect ctl sample show drills into one sample: its status, timing, token usage, in-flight activity, and score, plus its full error history — including errors from prior attempts (both task-level retries and sample-level retry_on_error). Pass --content for the error messages, or --traceback (which implies --content) for full tracebacks:

$ inspect ctl sample show gpqa 9 --traceback

Transcript Events

inspect ctl sample events reads a running sample’s transcript — the sequence of model calls, tool calls, errors, and scores it has produced so far:

$ inspect ctl sample events gpqa 17 --content
time      event  summary
--------  -----  -------------------------------------------------
14:09:01  model  openai/gpt-5 · 1840 tok · stop · The compound is...
14:09:04  tool   bash(ls /data) → README.md results.csv
14:09:11  model  openai/gpt-5 · generating 2:31

3 events  ·  more
next: eyJuIjoiYWJjMTIzOjAiLCJpIjozfQ  (resume with --cursor)

By default the rows are metadata only — event types, timing, token counts, stop reasons, tool function names, and error presence, with none of the agent-controlled free text (see Agent-controlled content above). --content adds truncated completions, tool arguments/results, and error messages; --full returns the raw serialized events.

An in-flight operation appears as a pending event at the transcript tail — generating 2:31 for a model call still awaiting its response, running 0:41 for an executing tool call — so the tail shows what the sample is doing now, not just what it last finished. The event is completed in place when the call returns: a fresh tail read then shows the finished row, but an incremental --cursor poll that already consumed the pending row does not re-serve it (pending: true in the --json row is the “still in flight when read” marker).

The first (unseeded) call returns the recent tail (the last 20 events; widen with --tail N, or start from the first event with --from-start). Reads are incremental: each page ends with a next cursor, and passing it back via --cursor returns only events that arrived after it. A polling loop reads a page, stores the cursor, and repeats; when the page reports done the sample has finished and no more events will come. Cursors are scoped to one attempt of a sample — if the sample is retried, a stale cursor restarts the read from the beginning rather than misreading the new attempt’s transcript. If the eval process is momentarily too busy to answer, the command fails (non-zero exit, message on stderr) rather than serving an empty page — treat that as “try again shortly”, not as the sample or eval being gone.

Other options:

Option	Description
--tail N	Start N events from the end (default 20 on a fully unseeded read — no --cursor, no --since-time/--until window, no --from-start).
--from-start	Start from the first event and page through the full backlog (cannot be combined with --cursor, --tail, or --since-time).
--limit N	Max events per page (default 500); combines with any start point (e.g. --from-start --limit 15 for the first 15). Counted before the --type filter, so a filtered page may return fewer.
--type model,tool	Filter by event type (all for everything). By default, high-volume structural events are excluded.
--content	Include truncated free-text content (completions, tool arguments/results, error messages) in the summaries.
--full	Return complete raw events instead of compact one-line summaries.
--since-time / --until	Filter to a wall-clock window (unix timestamps).

Note that --cursor takes the opaque next token, never a timestamp — for a wall-clock window use --since-time. Events for samples that have already completed are also readable — they are served from the eval’s log.

Conversation Snapshots

inspect ctl sample messages reads one sample’s current conversation — its message list as it stands right now:

$ inspect ctl sample messages gpqa 17 --tail 3 --content
#   role       content
--  ---------  --------------------------------------------------
12  assistant  Let me check the data.  → bash(ls /data)
13  tool       README.md results.csv
14  assistant  Based on the results, the compound is...

3 of 15 messages (use --all for the whole conversation)  ·  running

Unlike sample events, this is a snapshot, not a stream: solver and agent code can rewrite the message list (for example, compaction replaces a prefix with a summary), so there is no resume cursor. Each call returns the conversation as it looks at that moment — by default a recent tail (the last 20 messages), with each row carrying its absolute index so a tailed view lines up with the full one. To watch a sample incrementally, poll and compare the reported total count (the cheap staleness signal), or use inspect ctl sample events for event-grain resumable reads.

As with events, the default rows are metadata only — index, role, tool-call function names, and error presence; --content adds the truncated message text and tool arguments (see Agent-controlled content above).

Option	Description
--tail N	Only the last N messages (default 20). Mutually exclusive with --all.
--all	The whole conversation instead of a recent tail.
--content	Include truncated message text and tool-call arguments in the summaries.
--full	Return raw ChatMessage JSON instead of compact one-line summaries.

With --json the response is an {as_of, status, count, messages} envelope, prefixed with the resolved task_id / sample_id / epoch (so a defaulted epoch is visible). As with events, EPOCH defaults to 1, and conversations of samples that have already completed are also readable — they are served from the eval’s log.

Store Snapshots

inspect ctl sample store reads one sample’s current store — the shared state that solvers, tools, and agents coordinate through (progress flags, scratchpads, StoreModel fields) — as it stands right now:

$ inspect ctl sample store gpqa 17
key                  type    size
-------------------  ------  ----
phase                string  10
attempts             number  1
Scratchpad:i0:notes  string  2048

3 of 3 keys  ·  running  ·  metadata only (pass --content for values)

Like sample messages, this is a snapshot, not a stream: the store is rewritable (a key can be overwritten or deleted between reads), so there is no resume cursor. Poll and compare the reported key count, or use inspect ctl sample events --type store when you want the change stream rather than the current state. While the sample runs the store is read from the live task state — no log involved, so it works even for samples whose transcript is buffered or bounded; once the sample finishes it is served from the eval’s log.

The default rows are metadata only — each key’s JSON type and its serialized size in UTF-8 bytes (for spotting the big keys; the --json rows also carry a length hint), with none of the values, which are agent-controlled text (see Agent-controlled content above). --content adds a truncated single-line preview of each value; --full returns the raw values.

Option	Description
--key NAME	Only these keys (repeatable), selected server-side so one large key doesn’t drag the whole store over the wire. An exact name, or a trailing-* prefix (e.g. --key 'AgentState:*' for one StoreModel’s fields). Requested keys that aren’t present are reported as missing — not an error.
--content	Include a truncated single-line preview of each value.
--full	Return raw values instead of the compact summary (unbounded — combine with --key to keep the response bounded).

With --json the response is an {as_of, status, count, store} envelope (plus missing when --key was given), prefixed with the resolved task_id / sample_id / epoch. count is always the whole store’s key count, so a filtered read still shows how much it left out. As with the other reads, EPOCH defaults to 1, and stores of samples that have already completed are also readable — they are served from the eval’s log.

Cancellation

inspect ctl sample cancel cancels one sample — most often a running one that has stalled (high idle in sample list) or is burning tokens without progress. To see why it is stalled before cancelling, inspect ctl process anomalies shows what the process is actually waiting on (see Stall Diagnosis below) — an in-flight operation emits no transcript event until it returns, so a stalled sample’s transcript alone won’t name it. By default the sample completes and the scorer runs on the work done so far (it is recorded with an operator limit, like the in-process TUI’s cancel); pass --action error to mark it errored instead (not permitted for samples configured to fail on errors), or --action cancel to record it as cancelled — its transcript is preserved in the log, it is not scored, and it does not count toward a fail-on-error threshold. The rest of the task is unaffected.

$ inspect ctl sample cancel gpqa 17

EPOCH defaults to 1 but is required whenever the task runs more than one epoch — a defaulted epoch would silently cancel the epoch-1 attempt rather than erroring:

$ inspect ctl sample cancel gpqa 17 3

--action cancel also works on a sample that hasn’t started yet, resolved by where it stands. A never-started sample (shown as pending in sample list) is cancelled before it starts: it is removed from the queue, listed as cancelled, and absent from the final log — an inspect eval-retry on the log will run it later. A queued re-run has its pending requeue withdrawn instead (see Requeue): the prior terminal record stands, and the sample is requeueable again. A cancel-before-start is reversible while the sample still holds its place in the queue — sample requeue withdraws the cancel and the sample runs normally when it gets a slot. The other actions are rejected for samples that haven’t started (there is no work to score and no error to record), as is the one blind window: a sample that has just left the queue but is not yet running (initializing — its sandbox may still be starting) can’t be cancelled until it is running — retry.

When a sample is not merely slow but stuck on one hung tool call — a sandbox command wedged against a dead daemon with no timeout configured, or a timeout whose teardown never completed — cancelling the whole sample discards its progress unnecessarily. inspect ctl sample cancel-tool-call cancels just that call: the model receives an ordinary tool timeout (“Command timed out before completing.”) and the sample continues, using the same per-call cancel primitive as the in-process TUI’s timeout button and ACP’s inspect/cancel_tool_call.

$ inspect ctl sample cancel-tool-call gpqa 17

With no --tool-call-id, the command targets the sample’s sole pending tool call, and errors if there are two or more — it never guesses among targets, and never cancels them all (pass explicit ids for that). --dry-run without an id enumerates the pending calls, and each running sample’s activity in sample list --json carries a per-call list (id, function, started_at, cancel_requested), so a watchdog script can spot a stalled tool call and cancel it by id in one poll loop. Like the other sample mutations it is idempotent — repeating a cancel, naming a call that is no longer pending, or targeting a finished sample is a clean no-op — and EPOCH is required whenever the task runs more than one epoch.

One honesty note: the cancel is delivered to the call’s cancel scope, which is not a guarantee the tool stops. Cancellation is cooperative, so a call parked on an await (the common hang) unwinds immediately, but sync code running in a thread or teardown inside a shielded scope may never notice — the pending call then keeps reporting cancel_requested: true and a repeat invocation no-ops with “cancel already requested”. The escalation from there is inspect ctl sample cancel (and ultimately killing the process and using inspect eval-retry).

inspect ctl task cancel cancels a whole running task. By default it aborts: in-flight samples are interrupted (their transcripts so far are preserved in the log as cancelled samples), completed samples are kept, and the task’s log is finalized with an error status noting the cancel. An eval set does not retry a cancelled task, and its other tasks are unaffected. TASK is always required — there is no sole-task default for destructive commands.

$ inspect ctl task cancel gpqa

Pass --action score or --action error to resolve the task gracefully instead of aborting it: each in-flight sample is scored on the work done so far (or marked errored), still-queued samples are abandoned, and the task runs to natural completion — so the eval finishes with a completed status rather than an error. This is how to abandon a task’s last few stragglers while still bringing the eval to a completed state. Note that a completed status doesn’t mean every sample ran: abandoned samples are absent from the log (visible as completed_samples < total_samples in its results); re-invoking inspect eval-set on the same log directory later — or an explicit inspect eval-retry on the log — will run them if you change your mind, reusing the samples that completed. --action error is not permitted when the task’s samples are configured to fail on errors. If a graceful cancel stalls — say on a hung scorer — issuing a plain inspect ctl task cancel escalates it to an abort.

$ inspect ctl task cancel gpqa --action score

Both commands are idempotent — cancelling something already finished, already cancelling, or already cancelled before start is a clean no-op, reported as changed: false in the --json detail, escalation over a strictly weaker pending resolution (see Draining) being the one exception — and both accept --dry-run to report what would be cancelled without doing it. A task between attempts, whose last attempt errored (or is still writing its final error log) and whose retry is coming but not yet started, has nothing running to interrupt: a plain task cancel abandons the pending retry — the task ends with its last attempt’s error log, exactly the shape an exhausted retry budget produces — while --action score/error are rejected there (no samples for a resolution to apply to). And sample cancel keeps the rejections described above: a sample that has just left the queue but is not yet running, and --action score|error on a sample that hasn’t started.

Draining

inspect ctl task drain is the gentlest point on the “stop a task” spectrum: stop dispatching new samples, let in-flight samples finish naturally — scored on their own terms, no interrupts — then complete the task with its ordinary terminal log. Reach for it when you’ve seen enough (enough samples for the analysis, a budget line approaching, a dataset tail not worth its cost) and want the work already invested to complete at full fidelity while nothing new starts.

$ inspect ctl task drain gpqa

Queued samples are abandoned as slots free up (terminal cancelled in the counters, absent from the log — the same treatment as under a graceful cancel), and the abandoned remainder stays runnable later via inspect eval-set on the same log directory or inspect eval-retry on the log (a task run with --no-log-samples has nothing a partial resume can reuse, so when the drain abandoned anything a later inspect eval-set re-runs it in full rather than treating the drained log as complete; a drain that landed after the last sample was already dispatched leaves a complete log either way). While the tail runs, inspect ctl task score still works: the in-flight samples a drain lets finish are exactly the ones you can interim-score. Note the timing: because drain never interrupts anything, nothing abandons until the first in-flight sample finishes naturally, and the task ends on the samples’ clock — an hour-long agentic sample runs for its hour. A dynamic task (one fed by a sample source) ends on the source’s clock as well: samples the source keeps producing are abandoned as they arrive, but nothing prompts the source itself to finish, so the task completes only when the source exhausts — a source waiting indefinitely for more work holds a drained task open until you escalate (below). The resolving field on inspect ctl task list --json rows (also shown as a column in the human table) says a drain — or a graceful cancel — is in effect, so a static-looking row reads as a draining tail rather than a stall.

Drain is not reversible (to stop and later continue, use inspect ctl task pause), and escalation is spelled with the existing verbs, ordered by invasiveness — drain < --action score/error < plain cancel: a stronger request applies over a weaker pending one, an equal-or-weaker repeat is the idempotent no-op. If the drain is taking too long, inspect ctl task cancel --action score resolves the in-flight samples now (and concludes a drain whose samples are held by a hard pause --now, which drain deliberately never disturbs); a plain inspect ctl task cancel tears the task down. Like task cancel, drain abandons the pending retry of a task between attempts, requires TASK outright, is idempotent, and accepts --dry-run (the report carries the split that matters: how many in-flight samples will finish naturally and how many queued samples will be abandoned).

Requeue

inspect ctl sample requeue re-runs one errored or cancelled sample inside the still-running eval — the recovery move when a sample failed for a transient reason (a provider incident, a flaky sandbox) and you’d rather not wait for the whole run to finish and inspect eval-retry it. The sample goes to the back of the sample queue and re-runs under the task’s normal machinery: its prior errors ride along as retry history, a checkpointed sample resumes from its checkpoint, and the run’s final log and counters reflect the fresh outcome (the superseded attempt does not count toward a fail-on-error threshold).

$ inspect ctl sample requeue gpqa 17

As with sample cancel, EPOCH defaults to 1 but is required whenever the task runs more than one epoch — a defaulted epoch would silently requeue a different attempt:

$ inspect ctl sample requeue gpqa 17 3

The command is idempotent: requeuing a sample whose re-run is already pending, queued, or running — or one that hasn’t started yet — is a clean no-op reporting the sample’s current status (changed: false in the --json detail), so a retrying script can re-issue safely without double-queueing. While the re-run waits its turn, sample list and sample show render the sample as queued with its prior error shown as retry history.

sample requeue is also how a cancel-before-start is reversed (see Cancellation): requeuing a sample that was cancelled before it started withdraws the cancel, and the sample runs normally when it gets a slot — it keeps its place in the queue rather than going to the back. Once the cancelled sample’s run has been discarded (when it would have started, or at task teardown), the requeue is rejected instead — there is no prior record to re-run from; use inspect eval-retry after the run.

Requeuing a sample that completed successfully is rejected — re-running or re-scoring a success is out of scope (use score invalidation and inspect eval-retry for post-hoc re-runs). Also rejected: a task that has finished or is between attempts (re-run failures with inspect eval-retry, or let the queued task retry handle them), and a task with a cancel in flight. --dry-run reports what would be re-run — the prior error, the attempt number, and whether a checkpoint resume is available — without changing anything, and reports the rejections above too, so an agent can probe safely.

Interim Scoring

inspect ctl task score answers “how is this eval actually doing?” mid-run: it runs the task’s own scorers over the in-flight samples, folds completed samples’ existing final scores in, computes interim metrics, and reports the results, all without ending any sample (the non-destructive counterpart to task cancel --action score).

$ inspect ctl task score gpqa
$ inspect ctl task score gpqa --dry-run          # counts by disposition, no scoring
$ inspect ctl task score gpqa --completed-only   # no holds: metrics over final scores
$ inspect ctl task score gpqa --no-wait --json   # start, then re-run to poll

Each in-flight sample is scored on its work-so-far by briefly holding it: the sample parks at its next model call (the same gate pause --now uses, applied to just that sample), its then-stable live state is scored — with full fidelity: real conversation, live store, live sandbox — and it is released and keeps running. The interim score is recorded on the sample’s transcript as an intermediate score event (exactly as the in-task score() API records one), so it persists into the log; the sample’s final scores are unaffected. A sample that doesn’t reach a model call within the hold timeout (a long tool call, a solver phase with no model calls) is skipped and reported rather than scored mid-motion, and one that completes first simply resolves normally — its final score supersedes. Samples are held one at a time, and scorer model calls never count against a held sample’s token/cost limits.

Completed samples are folded in, never re-scored: already-scored samples contribute their final scores to the interim metrics, and unscored completed samples (a scorer that errored earlier, for example) are reported skipped — writing scores into a mid-run log isn’t safe, so inspect score after the run remains the way to score them. Errored samples count as scoreable only when the eval’s score_on_error would score them; cancelled and not-yet-started samples are skipped. A task that is draining can still be interim scored — its in-flight samples are running normally — whereas a pass on a task whose in-flight samples are being resolved by task cancel --action score|error stops at the first held sample and reports the interruption. A run started with --no-score can’t be interim scored at all — the task has no scorers in-process, and the command reports exactly that; use inspect score on its log after the run.

inspect ctl sample score is the same machinery scoped to one sample — “how is this sample doing?” for a long-tail agentic sample that has been running for hours. The sample is resolved to the same dispositions: in-flight is held, scored on its work-so-far, and released; completed-and-scored reports its existing final scores (never re-scored); completed-unscored points at post-run inspect score. A sample-scoped pass computes no interim metrics — the sample’s scores are the payload. Like the other sample verbs, EPOCH is required whenever the task runs more than one epoch:

$ inspect ctl sample score gpqa 42
$ inspect ctl sample score gpqa 42 2 --dry-run   # epoch 2's disposition, no scoring

Scoring can take minutes (model-graded scorers), so both commands start a pass and poll it to completion, rendering progress; --no-wait returns immediately, and the follow-up is --status, which reports the current (or most recent) pass without starting one (a bare re-run joins a still-running pass — one pass per task at a time, shared between the task-wide and sample-scoped forms, so holds never stack — but would start a fresh pass, with fresh holds, once the first finishes; sample score joins only a running pass for that same sample, task score never joins a sample-scoped pass, and both report a foreign-scope pass as a conflict to retry after). In the results, failed counts genuine scoring failures; in-flight samples the pass never attempted (they completed on their own, or never parked) or that every scorer declined to score (a scorer may return no score, e.g. for work it considers too incomplete) are reported unscored. Metrics are labeled interim for a reason: epochs may be incomplete and in-flight scores describe a moment mid-run. Two costs to know about: a held sample’s wall-clock time_limit keeps running while held (so avoid recurring full passes — use --completed-only, which takes no holds and makes no scorer calls, for periodic polling), and scorer model calls share the process’s connection limits with the running eval (a held sample’s parked calls release their connection slots while parked, so the pass’s own graders don’t starve behind the hold). Scorers should be read-only with respect to the sandbox to be safely interim-scorable — a scorer that mutates the environment (cleanup scripts, marker files) can perturb the still-running agent.

Repeated Mutations

Interactively, each mutation prints a full task header above its outcome for context. When stdout is not a TTY — piped or redirected output, a script whose output is captured, an agent’s shell tool — the task-scoped mutation verbs (sample requeue, sample cancel, sample cancel-tool-call, sample score, task cancel, task drain, task pause/resume, task score, task log-flush, and config when setting a knob) switch to a terse mode instead: one verb target: outcome line per call, so N mutations in a loop read as N scannable outcome lines rather than N repeated banners:

$ inspect ctl sample errors gpqa --json | jq -r '.samples[] | "\(.sample_id) \(.epoch)"' |
    while read -r s e; do inspect ctl sample requeue gpqa "$s" "$e"; done | tee requeue.log
requeue gpqa/11 (epoch 1): accepted — will re-run from the back of the sample queue
requeue gpqa/17 (epoch 1): accepted — will resume from its checkpoint
requeue gpqa/23 (epoch 1): no-op — a re-run is already pending

(The pipe is what selects the terse form here — a loop whose stdout still goes to the terminal keeps the full rendering for each call.) Two exceptions to the strict one-line shape: a qualified config set appends ! warning and note: lines after its outcome line (a process-scoped retune’s blast-radius note, a knob that could not be applied), and a task score (or sample score) that joins an already-running pass prints a join-note line (which pass it joined, and whether its flags differ) before its outcome line — scripts that count lines should prefer --json. Pass --terse to force the one-line mode on a terminal, or --no-terse to keep the full header rendering in a pipe. For scripts that branch on precise per-call outcomes, prefer --json: it takes precedence over both terse flags, and the mutation result envelope ({target, applied, dry_run, detail}) distinguishes applied from the idempotent no-op from dry-run structurally for the cancel, requeue, pause, and resume verbs (task log-flush reports applied: true even when nothing was buffered — its detail carries the flushed count).

Pause and Resume

Pause is the missing state between “running” and “cancelled”: it stops a run from starting new work while keeping the process, its queue, and this control surface alive, and it is non-destructive, idempotent, and reversible. By default the pause applies to dispatch, not to execution — think pausing a job queue, not suspending a process: samples already running (including model calls in flight) are not frozen or interrupted; they run to completion while new work holds. When in-flight spend must stop too, pause --now (the hard pause, below) additionally holds running samples at their next model call. Typical uses: ride out a provider incident without feeding more samples into it, stop spending while a cost question is decided, yield shared rate-limit capacity to a more urgent eval, or hold a run steady while you investigate a suspicious transcript (the read commands keep answering while paused).

$ inspect ctl process pause                    # pause the whole run (eval or eval-set)
$ inspect ctl process resume                   # pick up exactly where it left off
$ inspect ctl model pause openai/gpt-5-nano    # pause just one model's work
$ inspect ctl model resume openai/gpt-5-nano   # the rest of the run kept going
$ inspect ctl process pause --now              # hard pause: also hold in-flight samples

Under a pause, in-flight samples finish naturally — solving, scoring, and log writes complete under their original limits — but no new samples leave the queue, no task retries start, and (for an eval set) no further tasks dispatch. Queued samples spend none of their time limits: they are exactly as resumable as before the pause. Quiesce time is therefore bounded by the longest in-flight work: a long-running agentic sample, or a batched generate call awaiting a provider batch, holds the run semi-active until it completes. When in-flight work must wind down faster than that, compose pause with inspect ctl config --max-connections (throttle in-flight demand without discarding progress) or a targeted inspect ctl sample cancel — or reach for the hard pause.

Adding --now to any pause verb makes it a hard pause: in addition to stopping dispatch, in-flight samples hold at their next model call. Outstanding model calls (and batch waits) finish, then no new call starts until resume — the lever for a provider incident or budget stop where an hour-long agentic sample should neither keep spending nor be cancelled and lose its progress. The hold covers provider-native compaction too (compacting rewrites the conversation, and for a long-context sample can be its most expensive single call), but not token counting: count_tokens() traffic is non-generative and cheap, and continues under a hard pause. A held sample’s parked time counts as waiting, so a working_limit does not burn down while held — but the wall clock keeps running: a sample held past its time_limit takes an ordinary time-limit outcome (though if scoring itself calls a model — a model-graded scorer — that call holds at the gate too, so the sample fully completes only after resume), the trade-off you accept for freezing it in place. resume clears both strengths, a plain pause after a pause --now downgrades it to the soft pause (last-write-wins), and cancel still escalates over a hold. One scoping difference from the soft model latch: model pause MODEL --now holds every generate call to MODEL — other tasks’ role and grader calls included — not just the tasks whose primary model matches.

inspect ctl task pause / resume scope the same thing to one task of an eval set, and inspect ctl model pause MODEL / resume MODEL to one model of a multi-model run: every task whose primary model matches holds — including eval-set tasks that haven’t started yet, which task-level pause can’t reach — while other models’ work continues (in-flight samples, including other tasks’ role/grader calls to the paused model, still finish naturally). MODEL is the exact name shown by inspect ctl task list. The task, model, and process pauses are independent latches — resuming one never clears another, so resuming a run after an incident never silently un-pauses a task (or model) you paused for its own reasons. All the verbs accept --dry-run and report changed: false on an idempotent repeat. Pause never blocks teardown: cancel and drain work unchanged on a paused task (a stamped cancel or drain passes the pause gate, so queued samples held by a soft pause abandon at once and the drain completes under the pause; under a hard pause --now the held in-flight samples stay parked, so the drain concludes on resume or on escalation to cancel --action score), config retunes compose with it, and pausing a task that is between attempts works, parking the queued retry (which a later task cancel or task drain can abandon outright).

inspect ctl task list shows which latches hold a paused task (any combination of task, model, and process) — including one paused between attempts, whose row keeps its paused marker while the gate parks the queued retry — and reports quiesced once nothing is left in flight (a dispatched sample counts from the instant it leaves the queue, so one still initializing its sandbox holds off quiesced too). Paused models are also listed in the footer, so a model latch whose tasks are all still queued (no rows yet) stays visible. Hard latches render with a (now) marker (e.g. task(now)), and samples parked at the generate gate show as a held count — (2 held) — in the row’s paused cell and the footer totals, on any row: a hard model pause can hold another task’s grader calls without stamping a latch on that task’s row.

A quiesced task has auto-flushed its completed samples to the log, which makes pause the clean way to stop for a process restart: pause, wait for quiesced, kill the process, and later re-invoke inspect eval-set on the same log dir — the standard eval-set resume logic re-runs only what didn’t complete. Use a plain (soft) pause for this workflow, not --now: a hard-paused run doesn’t drain toward quiesced — held samples are mid-flight and stay parked until resumed (or until their time limits expire) — and killing while task list shows a held count forfeits those samples’ in-sample progress. Downgrade with a plain pause and let them finish instead. Pause state itself is in-memory only: a restarted process starts unpaused.

Note the distinction with keep-alive: process resume resumes a paused run; process release ends a keep-alive park after the eval finishes. A paused run never finishes — resume (or cancel) it rather than waiting for it to exit.

Stall Diagnosis

inspect ctl process anomalies [PID] shows why a process is stalled: it reconstructs from the process’s trace log which actions (model calls, sandbox operations, subprocesses) are currently in flight — with live durations — plus any that were cancelled (--all adds errored and timed-out actions, and --filter narrows by message text). An in-flight operation emits no transcript event until it returns, so this is the read that names what a stalled sample with a high idle is actually waiting on.

$ inspect ctl process anomalies 12345

Unlike the other reads, this one opens the process’s trace file directly rather than asking the process — so it works even against a process too busy or wedged to answer (the escalation path when another command reports “busy”), and even post-mortem: passing the pid of an exited process reads its trace file (kept on disk for the last 10 runs) to show what was in flight when it died, with durations dated to the file’s last write. With no PID it reads every running Inspect process, one section per pid.

Configuration

inspect ctl config shows a running eval’s retunable launch configuration, and can retune it mid-run — for example to throttle an eval that is hammering a provider or overloading a machine, or to open it up when more capacity becomes available. Any inspect eval launch flag that can be retuned mid-flight is settable here, under the same spelling:

$ inspect ctl config gpqa
inspect_evals/gpqa_diamond (ZByxJpK4bKSz)  ·  openai/gpt-5  ·  running  ·  12/40

config:
  max samples [task]:         50 (40 in use, tracking adaptive connections — set to pin)
  max sandboxes [process]:    docker 40 (12 in use)
  max subprocesses [process]: 16 (9 in use)
  adaptive connections [process]:
    openai/gpt-5: 45 (38 in use), range 10–100, last: 40→45 steady_state_up
  log buffer [task]:          10 samples (2 pending)
  shared sync [task]:         off

$ inspect ctl config gpqa --max-connections 20

Scope is a property of each knob, not of the command: task-scoped knobs apply to the selected task, process-scoped knobs apply to every task in the process. The output labels every knob with its scope (in --json, each knob carries "scope": "task" | "process"), and a --dry-run reports the blast radius of a process-scoped change.

Option	Scope	Description
--max-samples N	task	Sample concurrency. Under adaptive connections — where it tracks the controller by default — an integer pins it at exactly N (the controller keeps governing API concurrency, still retunable via --max-connections); clear unpins and resumes tracking.
--max-tasks N	process	Override the max concurrently running tasks (clear restores launch config).
--max-sandboxes N	process	Per-provider sandbox concurrency.
--max-subprocesses N	process	Subprocess concurrency (inactive until the run’s first subprocess).
--max-connections N	process	Adaptive connections scaling ceiling.
--key NAME LIMIT	process	Set a named concurrency() registry limit — any limit tools or task code register by name.
--log-buffer N	task	Completed samples buffered before a log write (lower it to write to S3 more often).
--log-shared S	task	Shared-log event sync interval in seconds.
--timeout S	process	Override the total retry budget per generate call, in seconds (clear restores launch config).
--attempt-timeout S	process	Override the per-attempt API timeout, in seconds (clear restores launch config).
--stream-idle-timeout S	process	Override the streaming stall timeout, in seconds — abandon and retry an attempt whose streaming response goes silent this long (clear restores launch config).
--max-retries N	process	Override the max retries per generate call (0 fails after the first attempt; clear restores launch config).
--time-limit S	task	Override the per-sample wall-clock time limit, in seconds — reaches in-flight samples too (clear restores launch config).
--token-limit N	task	Override the per-sample token limit — applies at each sample’s next token check (clear restores launch config).
--message-limit N	task	Override the per-sample message limit — applies at each sample’s next message check (clear restores launch config).
--model M	—	Restrict --max-connections (and the adaptive view) to matching models in mixed-model runs.
--reason R	—	Why the change is being made — recorded with it in each affected eval log.
--author A	—	Author recorded with the change (defaults to your git identity, then OS username).
--dry-run	—	Report what would change (current → requested) without applying it.

Applied changes are recorded in each affected eval log (EvalLog.config_updates: author, timestamp, old → new values), so the log tells the truth about the configuration the run actually ran under — effective_eval_config() / effective_generate_config() fold the records over the launch config. Include --reason whenever you set a knob, so the record says why (“provider returning 429s”, “throttling for overnight run”): the reason is the one part of the record only you know at retune time, and it’s what lets a later reader — or the person whose eval you retuned — distinguish a considered intervention from a stray command. The result envelope’s persisted field reports, per applied knob, whether the record was written.

Concurrency changes take effect immediately and never interrupt running work: raising a limit lets more samples/sandboxes/subprocesses/requests start right away, while lowering one below the current in-use count blocks new starts until enough in-flight work drains. Under adaptive connections the view also reports each model’s live controller state — its current limit, in-flight count, scaling range, and recent scale changes — so you can see whether the provider is rate-limiting before deciding to intervene. --log-buffer affects future writes only — run inspect ctl task log-flush to write what is already pending.

--timeout / --attempt-timeout / --stream-idle-timeout / --max-retries set live overrides of the corresponding generation config fields — the “stop retrying and fail fast” (or “raise retries to ride it out”) lever during a provider incident. The model retry loop reads the overrides at each point of use, so a change reaches even generate calls already inside a retry loop; an in-flight API request always drains first (its attempt timeout is not retroactively shortened), and timeouts a provider SDK bakes into its client at initialization are unaffected. Batch admin operations (creating a provider batch, polling its results) also keep their launch config — failing one of those would fail every request riding the batch — while batched generate requests themselves still honor the --timeout / --max-retries overrides through their own retry loops (--attempt-timeout and --stream-idle-timeout do not apply to batched calls: an attempt there waits on an entire provider batch — which never streams — and cancelling that wait would resubmit duplicate requests into a new batch). The overrides are consulted after each attempt completes, so a retune that lands while a call is sitting in an exponential-backoff sleep (which grows to as much as 30 minutes between attempts) takes effect only after that sleep finishes and one more attempt runs — when failing fast, lower --attempt-timeout in the same retune to bound that final attempt. An override applies process-wide until cleared (pass clear) or the run ends; the config view reports each field’s active override, with launch config meaning no override is in effect.

--max-tasks likewise sets a live override — of the task dispatch limit, the lever when task count (rather than per-task concurrency) is what’s holding a run back: an eval set whose adaptive controllers sit far below their ceiling with tasks still pending. The dispatcher reads the override at each dispatch decision, so raising it starts pending tasks immediately; lowering never interrupts running tasks (the in-flight count rides above the new limit until it drains). The config view reports the effective limit alongside the launch value and the dispatcher’s in-flight and pending counts; a set landing while no task dispatcher is live (for example during run startup, while sandboxes are still being provisioned) still applies to task dispatch later in the run. This includes runs launched with max_tasks=1: their queued tasks sit pending in the same dispatcher, so a raise starts them immediately (a raised limit takes precedence over the sequential ordering that max_tasks=1 otherwise provides). More concurrent tasks can mean more concurrent sandbox startups — --max-sandboxes remains the guard rail.

--time-limit / --token-limit / --message-limit set live overrides of the task’s per-sample limits — the “these samples are running away” lever. Each sample resolves its effective limit through the override at every limit check, so a retune reaches samples already in flight as well as ones not yet started: a lowered token or message ceiling applies at a sample’s next check (a sample already over it fails with the ordinary limit outcome and is scored on the work done so far, per the task’s configuration), and a lowered time limit reschedules running samples’ deadlines directly — a sample already past the new deadline is cancelled with a time-limit outcome immediately. Raising a limit unblocks in-flight samples the same way (though a sample whose time limit already fired stays cancelled). The overrides only affect each sample’s top-level limits — limits that agents or solvers open themselves are untouched. An override applies to the selected task until cleared (pass clear) or the run ends; the config view reports each field’s active override, and sample list’s token usage/ceiling columns reflect a retuned ceiling.

Beyond the named flags, any limit registered through the concurrency() API — by built-in tools (for example the web search providers), model compaction, or your own solver and tool code — is settable with --key NAME LIMIT. The config output lists the registered keys under concurrency keys, exactly as addressable here; named limits are created lazily on first use, so a key that names no registered limit errors and lists the keys that do exist.

Task-scoped knobs are keyed by the task (stable across retries): with eval sets’ default immediate retries, a retune survives a task retry rather than reverting to the launch configuration (legacy batch-mode retries — retry_immediate=False — run as separate calls and revert). With no TASK argument the command targets the sole running task; in a multi-task process the process-scoped knobs still work without a selector (they apply process-wide), while setting a task-scoped knob then requires the TASK.

Log Flushing

Completed samples are buffered and written to the (possibly remote, e.g. S3) log in batches (see --log-buffer above). inspect ctl task log-flush writes any buffered samples to the log immediately, so they become readable and analyzable without waiting for the buffer to fill. It is safe to repeat — a flush with nothing pending writes nothing.

$ inspect ctl task log-flush gpqa

Processes and Keep Alive

inspect ctl process lists the running Inspect processes (their pids, keep-alive status, and hosted tasks). The pid is the selector process keep / process release take; with a single running process it can be omitted.

A process exits as soon as its eval finishes, taking the control endpoint with it. That is a problem for scripted workflows that want to inspect results after completion: the process may be gone by the time they look. The --ctl-server option controls this:

inspect eval ctf.py --ctl-server=keep

With keep, the process stays running after the eval finishes — its state remains queryable via inspect ctl and its logs are fully written — until you release it:

inspect ctl process release

You can also latch keep-alive onto an eval that is already running (launched without --ctl-server=keep) with inspect ctl process keep.

If more than one process is parked, release lists their pids and you disambiguate by passing one (inspect ctl process release <pid>). Release also works ahead of time: issued while the eval is still running, it means “exit when done” — the process skips the park and exits as soon as the eval finishes (it never cancels in-flight work). keep and release are last-write-wins, so a keep issued after a release (while the eval is still running) restores the park. From Python, pass ctl_server="keep" to eval() or eval_set(). For eval sets, keep-alive requires retry_immediate=True (the default).

Disabling the Control Server

The control server is on by default. To run an eval without it:

inspect eval ctf.py --ctl-server=false

The INSPECT_EVAL_CTL_SERVER environment variable mirrors the option (for example, set INSPECT_EVAL_CTL_SERVER=false to disable it across a CI job). If the server fails to bind (for example, on a read-only filesystem) the eval logs a warning and runs normally without it — eval results never depend on the control channel.