from shortuuid import uuid
from inspect_ai.agent import react
react(..., tools=[bash_explorer(instance=uuid())])Custom Agents
Overview
Inspect agents bear some similarity to solvers in that they are functions that accept and return a state. However, agent state is intentionally much more narrow—it consists of only conversation history (messages) and the last model generation (output). This in turn enables agents to be used more flexibly: they can be employed as solvers, tools, participants in a workflow, or delegates in multi-agent systems.
Below we’ll cover the core Agent protocol, implementing a simple tool use loop, and related APIs for agent memory and observability.
Protocol
An Agent is a function that takes and returns an AgentState. Agent state includes two fields:
| Field | Type | Description |
|---|---|---|
messages |
List of ChatMessage | Conversation history. |
output |
ModelOutput | Last model output. |
Example
Here’s a simple example that implements a web_surfer() agent that uses the web_search() tool to do open-ended web research:
from inspect_ai.agent import Agent, AgentState, agent
from inspect_ai.model import ChatMessageSystem, get_model
from inspect_ai.tool import web_search
@agent
def web_surfer() -> Agent:
async def execute(state: AgentState) -> AgentState:
"""Web research assistant."""
# some general guidance for the agent
state.messages.append(
ChatMessageSystem(
content="You are a tenacious web researcher that is "
+ "expert at using web search to answer questions."
)
)
# run a tool loop w/ the web_search then update & return state
messages, state.output = await get_model().generate_loop(
state.messages, tools=[web_search()]
)
state.messages.extend(messages)
return state
return executeThe agent calls the generate_loop() function which runs the model in a loop until it stops calling tools. In this case the model may make several calls to the web_search() tool to fulfil the request.
While this example illustrates the basic mechanic of agents, you generally wouldn’t write an agent that does only this (a system prompt with a tool use loop) as the react() agent provides a more sophisticated and flexible version of this pattern.
Tool Loop
Agents often run a tool use loop, and one of the more common reasons for creating a custom agent is to tailor the behaviour of the loop. Here is an agent loop that has a core similar to the built-in react() agent:
from typing import Sequence
from inspect_ai.agent import AgentState, agent
from inspect_ai.model import execute_tools, get_model
from inspect_ai.tool import (
Tool, ToolDef, ToolSource, mcp_connection
)
@agent
def my_agent(tools: Sequence[Tool | ToolDef | ToolSource]):
async def execute(state: AgentState):
# establish MCP server connections required by tools
async with mcp_connection(tools):
while True:
# call model and append to messages
state.output = await get_model().generate(
input=state.messages,
tools=tools,
)
state.messages.append(output.message)
# make tool calls or terminate if there are none
if output.message.tool_calls:
messages, state.output = await execute_tools(
message, tools
)
state.messages.extend(messages)
else:
break
return state
return execute- 1
-
Enable passing
toolsto the agent using a variety of types (including ToolSource which enables use of tools from Model Context Protocol (MCP) servers). - 2
- Establish any required connections to MCP servers (this isn’t required, but will improve performance by re-using connections across tool calls).
- 3
- Standard LLM inference step yielding an assistant message which we append to our message history.
- 4
- Execute tool calls—note that this may update output and/or result in multiple additional messages being appended in the case that one of the tools is a handoff() to a sub-agent.
This above represents a minimal tool use loop—your custom agents may diverge from it in various ways. For example, you might want to:
- Add another termination condition for the output satisfying some criteria.
- Add a critique / reflection step between tool calling and generate.
- Urge the model to keep going after it decides to stop calling tools.
- Handle context window overflow (
stop_reason=="model_length") by truncating or summarising themessages. - Examine and possibly filter the tool calls before invoking execute_tools()
For example, you might implement automatic context window truncation in response to context window overflow:
# check for context window overflow
if state.output.stop_reason == "model_length":
if overflow is not None:
state.messages = trim_messages(state.messages)
continueNote that the standard react() agent provides some of these agent loop enhancements (urging the model to continue and handling context window overflow).
Compaction
Compaction enables you to automatically manage conversation context as it grows, helping you optimize costs and stay within context window limits for long-running agents. Use the compaction() function along with a compaction strategy to incorporate compaction into your custom agent.
For example, here we enhance the simple agent loop example from above with compaction. The compact handler has two methods: compact_input() to prepare input for the model, and record_output() to calibrate token estimation from the model’s actual usage.
from typing import Sequence
from inspect_ai.agent import AgentState, agent
from inspect_ai.model import (
CompactionAuto, compaction, execute_tools, get_model
)
from inspect_ai.tool import (
Tool, ToolDef, ToolSource, mcp_connection
)
@agent
def my_agent(tools: Sequence[Tool | ToolDef | ToolSource]):
async def execute(state: AgentState):
# create compaction handler
compact = compaction(
CompactionAuto(),
prefix=state.messages,
tools=tools
)
# establish MCP server connections required by tools
async with mcp_connection(tools):
while True:
# compact input
input, c_message = await compact.compact_input(state.messages)
if c_message:
state.messages.append(c_message)
# call model and append to messages
state.output = await get_model().generate(
input=input,
tools=tools,
)
state.messages.append(state.output.message)
# record output for token calibration
await compact.record_output(input, state.output)
# make tool calls or terminate if there are none
if state.output.message.tool_calls:
messages, state.output = await execute_tools(
state.output.message, tools
)
state.messages.extend(messages)
else:
break
return state
return execute- 1
-
Create the compaction handler using the specified strategy. Pass a
prefixthat should always be included in any compacted history as well astools(used for computing the input tokens). - 2
-
Call
compact_input()prior tomodel.generate()—pass the compactedinputto the model and append thec_message(if specified) to the message history. - 3
-
Call
record_output()aftermodel.generate()to calibrate token estimation using the model’s actual reported usage. This improves the accuracy of compaction threshold detection.
To recover from a context window overflow (stop_reason == "model_length"), call compact_input() with force=True, which compacts regardless of the threshold, and generate again. The handler does not check that a forced compaction reduced the input. If the retry also overflows, truncate the messages or stop rather than forcing compaction again.
The handler adds only messages it has not seen before. If your agent changes, removes or reorders messages already in the history, call reset() with the new prefix before the next compact_input(), so that the next input is built from the changed history.
reset() clears the handler’s state but does not remove summaries from your messages. A summary compaction (including CompactionAuto’s summary fallback) starts from the latest summary in the conversation, so before resetting, remove any summaries that the changed history invalidated.
The returned compact handler maintains internal state for a single growing conversation history. Concurrent calls within the same conversation are safe, but do not share one handler across divergent message histories — the compacted result mixes them.
There are various configurable compaction strategies available—see the Compaction documentation for details.
Sample Store
In some cases agents will want to retain state across multiple invocations, or even share state with other agents or tools. This can be accomplished in Inspect using the Store, which provides a sample-scoped scratchpad for arbitrary values.
Typed Store
When developing agents, you should use the typed interface to the per-sample store, which provides both type-checking and namespacing for store access.
For example, here we define a typed accessor to the store by deriving from the StoreModel class (which in turn derives from Pydantic BaseModel):
from pydantic import Field
from inspect_ai.util import StoreModel
class Activity(StoreModel):
active: bool = Field(default=False)
tries: int = Field(default=0)
actions: list[str] = Field(default_factory=list)We can then get access to a sample scoped instance of the store for use in agents using the store_as() function:
from inspect_ai.util import store_as
activity = store_as(Activity)Agent Instances
If you want an agent to have a store-per-instance by default, add an instance parameter to your @agent function and pass it a unique value. Then, forward the instance on to store_as() as well as any tools you call that are also stateful (e.g. bash_session()). For example:
from pydantic import Field
from shortuuid import uuid
from inspect_ai.agent import Agent, agent
from inspect_ai.model import ChatMessage
from inspect_ai.tool import bash_session
from inspect_ai.util import StoreModel, store_as
class BashExplorerState(StoreModel):
messages: list[ChatMessage] = Field(default_factory=list)
@agent
def bash_explorer(instance: str | None = None) -> Agent:
async def execute(state: AgentState) -> AgentState:
# get state for this instance
explorer_state = store_as(BashExplorerState, instance=instance)
...
# pass the instance on to bash_session
messages, state.output = await get_model().generate_loop(
state.messages, tools=[bash_session(instance=instance)]
)Then, pass a unique id as the instance:
This enables you to have multiple instances of the bash_explorer() agent, each with their own state and terminal session.
Named Instances
It’s also possible that you’ll want to create various named store instances that are shared across agents (e.g. each participant in a game might need their own store). Use the instance parameter of store_as() to explicitly create scoped store accessors:
red_team_activity = store_as(Activity, instance="red_team")
blue_team_activity = store_as(Activity, instance="blue_team")Tool State Scopes
Agents that run concurrently in one sample share the state of tools left at their default instance. For example, two agents given memory() read and write the same files. To give each agent its own state, run it inside a tool_state_scope():
from inspect_ai.util import tool_state_scope
async def run_worker(name: str, worker: Agent, state: AgentState) -> AgentState:
with tool_state_scope(name):
return await worker(state)Inside the scope, memory(), bash_session() and skill() use the scope as their default instance ("@<name>"), so each agent gets its own memory, shell and skills. A tool given an explicit instance, such as memory(instance="team"), is still shared between agents. Nested scopes join their names with “/” ("@outer/inner"). Instances that start with “@” are reserved for scopes, so an explicit instance never shares state with a scope by accident. Tasks started inside the scope inherit it. Outside any scope, tools behave as before. Skill files are the exception: they live in the sandbox, so skill() tools that install into the same directory share them. A skill file that already exists with the same contents is not written again, and one that exists with different contents raises an error.
Custom tools and agents can follow the scope by passing tool_state_instance(instance) to store_as(). Library code that picks an instance name itself and wants it kept private to the scope can use private_tool_instance(name), which returns "<scope>#<name>" inside a scope (for example "@w-1#general") and name outside one.
Agent Limits
The Inspect limits system enables you to set a variety of limits on execution including tokens consumed, messages used in conversations, clock time, and working time (clock time minus time taken retrying in response to rate limits or waiting on other shared resources; see Working Limit for its limitations).
Limits are often applied at the sample level or using a context manager. It is also possible to specify limits when executing an agent using any of the techniques described above.
To run an agent with one or more limits, pass the limit object in the limits argument to a function like handoff(), as_tool(), as_solver() or run() (see Using Agents for details on the various ways to run agents).
Here we limit an agent we are including as a solver to 500K tokens:
eval(
task="research_bench",
solver=as_solver(web_surfer(), limits=[token_limit(1024*500)])
)Here we limit an agent handoff() to 500K tokens:
eval(
task="research_bench",
solver=[
use_tools(
addition(),
handoff(web_surfer(), limits=[token_limit(1024*500)]),
),
generate()
]
)Limit Exceeded
Note that when limits are exceeded during an agent’s execution, the way this is handled differs depending on how the agent was executed:
For agents used via as_solver(), if a limit is exceeded then the sample will terminate (this is exactly how sample-level limits work).
For agents that are run() directly with limits, their limit exceptions will be caught and returned in a tuple. Limits other than the ones passed to run() will propagate up the stack.
from inspect_ai.agent import run state, limit_error = await run( agent=web_surfer(), input="What were the 3 most popular movies of 2020?", limits=[token_limit(1024*500)]) ) if limit_error: ...For tool based agents (handoff() and as_tool()), if a limit is exceeded then a message to that effect is returned to the model but the sample continues running.
Parameters
The web_surfer agent used an example above doesn’t take any parameters, however, like tools, agents can accept arbitrary parameters.
For example, here is a critic agent that asks a model to contribute to a conversation by critiquing its previous output. There are two types of parameters demonstrated:
Parameters that configure the agent globally (here, the critic
model).Parameters passed by the supervisor agent (in this case the
countof critiques to provide):
from inspect_ai.agent import Agent, AgentState, agent
from inspect_ai.model import ChatMessageSystem, Model
@agent
def critic(model: str | Model | None = None) -> Agent:
async def execute(state: AgentState, count: int = 3) -> AgentState:
"""Provide critiques of previous messages in a conversation.
Args:
state: Agent state
count: Number of critiques to provide (defaults to 3)
"""
state.messages.append(
ChatMessageSystem(
content=f"Provide {count} critiques of the conversation."
)
)
state.output = await get_model(model).generate(state.messages)
state.messages.append(state.output.message)
return state
return executeYou might use this in a multi-agent system as follows:
supervisor = react(
...,
tools=[
addition(),
handoff(web_surfer()),
handoff(critic(model="openai/gpt-6-luna"))
]
)When the supervisor agent decides to hand off to the critic() it will decide how many critiques to request and pass that in the count parameter (or alternatively just accept the default count of 3).
Currying
Note that when you use an agent as a solver there isn’t a mechanism for specifying parameters dynamically during the solver chain. In this case the default value for count will be used:
solver = [
system_message(...),
generate(),
critic(),
generate()
]If you need to pass parameters explicitly to the agent execute function, you can curry them using the as_solver() function:
solver = [
system_message(...),
generate(),
as_solver(critic(), count=5),
generate()
]Transcripts
Transcripts provide a rich per-sample sequential view of everything that occurs during plan execution and scoring, including:
- Model interactions (including the raw API call made to the provider).
- Tool calls (including a sub-transcript of activitywithin the tool)
- Changes (in JSON Patch format) to the TaskState for the Sample.
- Scoring (including a sub-transcript of interactions within the scorer).
- Custom
info()messages inserted explicitly into the transcript. - Python logger calls (
infolevel or designated customlog-level).
This information is provided within the Inspect log viewer in the Transcript tab (which sits alongside the Messages, Scoring, and Metadata tabs in the per-sample display).
Custom Info
You can insert custom entries into the transcript via the Transcript info() method (which creates an InfoEvent). Access the transcript for the current sample using the transcript() function, for example:
from inspect_ai.log import transcript
transcript().info("here is some custom info")Strings passed to info() will be rendered as markdown. In addition to strings you can also pass arbitrary JSON serialisable objects to info().
Grouping with Spans
You can create arbitrary groupings of transcript activity using the span() context manager. For example:
from inspect_ai.util import span
async with span("planning"):
...There are two reasons that you might want to create spans:
- Any changes to the store which occur during a span will be collected into a StoreEvent that records the changes (in JSON Patch format) that occurred.
- The Inspect log viewer will create a visual delineation for the span, which will make it easier to see the flow of activity within the transcript.
Spans are automatically created for sample initialisation, solvers, scorers, subtasks, tool calls, and agent execution.
Model Call Origins
Each ModelEvent records where the call was made in its origin field. During the solve a call is "agent", one of its conversation’s agent turns, unless it is made by a tool, a compaction, an approver, a reviewer, MCP sampling, a sentinel or a scorer, which Inspect marks itself ("tool", "compaction", "approval", "review", "sampling", "sentinel", "scorer"). A scorer’s calls are "scorer", including those of an agent run inside it, and a sentinel does not check a scorer’s tool calls. Any other call outside the solve, such as one made at sample init, is "setup". A sentinel checks only agent turns at its generate stages, and the transcript timeline shows other calls as utility.
When replaying an older log without origin, Inspect infers turns from the event span tree. A stored timeline passed to sentinel replay can group the resulting steps into conversations, but its branches and spans do not change which calls are inferred as turns. Recorded sentinel checks and human decisions are used only where the step id pairs with a model call’s exact id. Checks without a matching call are skipped with a warning. Inference may differ from the turns observed by a sentinel that ran before origins were recorded; reproducing that sentinel’s live steps is outside the replay contract.
Replay reconstructs approval-derived human decisions only for logs from Inspect 0.3.277 onward; older or unversioned logs omit them and warn once when a selected human approver appears. A log is refused where the call an approval approved (its modified call, else the call it records) is not the call its ToolEvent records, unless a recorded sentinel modify changed the call again to the one the ToolEvent records. Inspect before 0.3.277 wrote such logs, including for a call an approver changed in place and approved without a modified call, and they are refused whatever version the header records, since a retried eval keeps its completed samples under a newer header. This includes an approved call whose tool records other arguments than it was given, such as memory() resolving a path. Replay such a transcript from its messages instead.
Where one model response proposes several tool calls under one id (Mistral gives the id "null" to every call whose id the model omitted), replay pairs each execution with the proposed call that has its function. Where several calls have that function, it takes the one whose arguments match those the execution’s approval records, or, where it has no approval and no sentinel modified it, those its ToolEvent records. Calls of one function that differ only in their arguments are interchangeable, since the steps take their arguments from the log. The exception is a call a sentinel modified that has no approval replay can rely on, whose tool_call step is built from the proposed call. Replay relies on an approval only in a log from Inspect 0.3.277 onward. This pairing needs a log that records each execution’s id (ToolEvent.execution_id); in an older log, replay takes the last call proposed under the id.
Replay skips a call’s tool steps, with a warning naming the call, in four cases:
- The model response proposed several calls under the call’s id, and the rules above do not pick one.
- A sentinel modified the call, and its ToolEvent records other arguments than the modify’s. A tool can rewrite its arguments, as memory() does when it resolves a path. This applies whatever the call’s id, and also where another sentinel decision outranked the modify.
- A sentinel modified the call, another execution records the same call id (a shared
"null", a repeated id), and no approval records the call the sentinel was shown. That is a call with no approval replay can rely on, or one an approver modified. An approver’s modify is recorded before the tool validates it (e.g. before memory() resolves a path). - The call ended in a parsing or approval error, has no recorded sentinel check, and the call its approval approved is not the one its ToolEvent records. It may have been refused before it ran, or its tool may have raised the error after running, so replay skips it instead of refusing the log.
Where none of these applies, a call a sentinel modified that has no approval replay can rely on keeps its tool steps. Replay builds its tool_call step from the proposed call, or from an approver’s change to it. A call an approver modified before the sentinel did also keeps its tool steps, and replay builds its tool_call step from the approver’s call as the approval records it. Either can differ from the call the sentinel was shown, since the tool’s validation can change the arguments (e.g. memory() resolves a path, validation fills a Pydantic default).
Inspect logs have origin on every model call or on none. For a mixed-origin transcript, replay uses recorded classifications where present and infers calls without origins. It does not promise continuity across the mixed sequence.
If two calls continue the same turn and no later call resolves the fork, replay leaves both calls to the existing prompt rules. The sample’s final messages do not select a fork head.
Without recorded origins, a failed call’s input may leave an earlier fork unresolved, and a custom solver’s own calls may be inferred as agent turns when its only agent is unwrapped.
Sentinel event replay requires a log written by Inspect 0.3.100 or later. Earlier logs lack the solvers and scorers spans that separate the agent’s work from scoring (and, before 0.3.94, native tool spans), so steps_from_events() raises an error rather than replaying them incorrectly. This includes samples that a retried eval carried over from such a log, whatever version its header records. Replay those logs from their messages instead. In supported logs, a spanless tool event can mark a call halted before execution and does not create tool steps.
If your agent makes model calls that are part of the solve but are not the evaluated agent’s turns, mark them with model_origin(). Use "environment" for the setup being evaluated against, such as an auditor model or a simulated user, and "helper" for a scaffold’s own helper calls, such as its own summarizer:
from inspect_ai.model import get_model, model_origin
with model_origin("environment"):
reply = await get_model(role="user").generate(messages)An agent run inside the block keeps the block’s kind, and a sentinel does not check the tool calls made within it. The marker is a context variable, so it reaches the calls made in the task that entered the block and in tasks that task starts while inside it, and no others. If the evaluated agent’s calls are made from inside an environment block (for example, by an auditor’s tool that generates the target’s turn), they are marked as environment too, so make them outside the block.
Parallelism
You can execute subtasks in parallel using the collect() function. For example, to run 3 web_search() coroutines in parallel:
from inspect_ai.util import collect
results = collect(
web_search(keywords="solar power"),
web_search(keywords="wind power"),
web_search(keywords="hydro power"),
)Note that collect() is similar to asyncio.gather(), but also works when Trio is the Inspect async backend.
The Inspect collect() function also automatically includes each task in a span(), which ensures that its events are grouped together in the transcript.
Using collect() in preference to asyncio.gather() is highly recommended for both Trio compatibility and more legible transcript output.
Background Work
The background() function enables you to execute an async task in the background of the current sample. The task terminates when the sample terminates. For example:
import anyio
from inspect_ai.util import background
async def worker():
try:
while True:
# background work
anyio.sleep(1.0)
finally:
# cleanup
background(worker)The above code demonstrates a couple of important characteristics of a sample background worker:
Background workers typically operate in a loop, often polling a sandbox or other endpoint for activity. In a loop like this it’s important to sleep at regular intervals so your background work doesn’t monopolise CPU resources.
When the sample ends, background workers are cancelled (which results in a cancelled error being raised in the worker). Therefore, if you need to do cleanup in your worker it should occur in a
finallyblock.
Sandbox Service
Sandbox services make available a set of methods to a sandbox for calling back into the main Inspect process. For example, the Human Agent uses a sandbox service to enable the human agent to start, stop, score, and submit tasks.
Sandbox services use a filesystem queue and are intended for callers with the same sandbox-user authority as the service. The service directory is private (mode 0700) to the user the service runs as, so callers must run as that user (or as root). They are not an authentication or privilege-separation boundary between sandbox users. Service names must be ASCII Python identifiers; instance names must begin with a letter or number and may then contain letters, numbers, ., _, or -.
Sandbox service are often run using the background() function to make them available for the lifetime of a sample.
For example, here’s a simple calculator service that provides add and subtract methods to Python code within a sandbox:
from inspect_ai.util import background, sandbox_service
async def calculator_service():
async def add(x: int, y: int) -> int:
return x + y
async def subtract(x: int, y: int) -> int:
return x - y
await sandbox_service(
name="calculator",
methods=[add, subtract],
until=lambda: False,
sandbox=sandbox()
)
background(calculator_service)Above we run the sandbox service in the background so it doesn’t block the main task while waiting for requests. You can also pass handle_requests=False to manually handle requests (e.g. poll for them periodically). In this the sandbox_service() returns a function you can call to process requests:
handle_requests = await sandbox_service(
name="calculator",
methods=[add, subtract],
until=lambda: False,
sandbox=sandbox(),
handle_requests=False
)
# now call handle_requests periodically to handle requests
await handle_requests()To use the service from within a sandbox, either add it to the sys path or use importlib. For example, if the service is named ‘calculator’:
import sys
sys.path.append("/var/tmp/sandbox-services/calculator")
import calculatorOr:
import importlib.util
spec = importlib.util.spec_from_file_location(
"calculator",
"/var/tmp/sandbox-services/calculator/calculator.py"
)
calculator = importlib.util.module_from_spec(spec)
spec.loader.exec_module(calculator)