inspect_ai.review

Reviewers and review policies.

Reviewers

human_reviewer

Interactive human reviewer.

Shows the operator the tool call together with the result it returned, on the same surfaces as human_approver() (an attached ACP client, the approvals panel in the full-screen display, or the console), and asks for a review decision. continue is offered as the surfaces’ “approve” choice.

Source

@reviewer(name="human")
def human_reviewer(
    choices: list[ReviewDecision] = ["continue", "terminate"],
) -> Reviewer

choices list[ReviewDecision]: Choices to present to human.

read_review_policies

Read review policies from a JSON or YAML config file.

Source

def read_review_policies(file: str) -> list[ReviewPolicy]

file str: JSON or YAML config file with review policies.

review

Context manager to temporarily replace tool review policies.

Source

@contextlib.contextmanager
def review(policies: list[ReviewPolicy]) -> Iterator[None]

policies list[ReviewPolicy]: Review policies to use within the context.

Types

Reviewer

Review the result of a tool call that has executed.

Source

class Reviewer(Protocol):
    async def __call__(
        self,
        message: str,
        call: ToolCall,
        result: ChatMessageTool,
        output: ToolResult,
        view: ToolCallView,
        history: list[ChatMessage],
    ) -> Review

message str: Message generated by the model along with the tool call.
call ToolCall: The tool call that executed (as executed, after any call-stage modification by an approver).
result ChatMessageTool: The tool result as the model will see it (after output truncation), including any tool error.
output ToolResult: The tool’s untruncated return value.
view ToolCallView: Custom rendering of tool context and call.
history list[ChatMessage]: The current conversation history (ending with the assistant turn that made the call).

Review

Review details (decision, explanation, etc.)

Source

class Review(BaseModel)

Attributes

decision ReviewDecision: Review decision.
explanation str | None: Explanation for decision.
metadata dict[str, Any] | None: Additional review metadata.

ReviewDecision

Represents the possible decisions in a review.

Source

ReviewDecision = Literal["continue", "terminate", "escalate"]

ReviewPolicy

Policy mapping reviewers to tools.

Source

@dataclass
class ReviewPolicy

Attributes

reviewer Reviewer: Reviewer for policy.
tools str | list[str]: Tools to use this reviewer for (can be full tool names or globs).

Decorator

reviewer

Decorator for registering reviewers.

Source

def reviewer(*args: Any, name: str | None = None, **attribs: Any) -> Any

*args Any: Function returning Reviewer targeted by plain reviewer decorator without attributes (e.g. @reviewer)
name str | None: Optional name for reviewer. If the decorator has no name argument then the name of the function will be used to automatically assign a name.
**attribs Any: Additional reviewer attributes.