Interactivity

Overview

In some cases you may wish to introduce user interaction into the implementation of tasks. For example, you may wish to:

• Confirm consequential actions like requests made to web services
• Prompt the model dynamically based on the trajectory of the evaluation
• Score model output with human judges

The request_input() function presents a structured form to the operator and returns their answers. It is async, and dispatches automatically to whichever surface is available: an attached ACP client, the in-process Textual panel under --display full, or the console under --display rich. The sample pauses on the call until the operator responds.

Request Input

You can prompt the user for input at any point in an evaluation using the request_input() function. Pass it a prompt message and an ElicitationSchema describing the fields you want filled in:

from inspect_ai.util import request_input
from acp.schema import ElicitationSchema, ElicitationStringPropertySchema

result = await request_input(
    message="Please enter your name",
    schema=ElicitationSchema(
        properties={
            "name": ElicitationStringPropertySchema(type="string"),
        },
        required=["name"],
    ),
)
if result.outcome == "accepted":
    name = result.content["name"]

result.outcome is one of "accepted", "declined", or "cancelled". When accepted, result.content is a dict keyed by the schema’s property names.

A single request_input() call can carry multiple fields, so a small form replaces a chain of separate prompts:

from acp.schema import (
    ElicitationSchema,
    ElicitationStringPropertySchema,
    ElicitationIntegerPropertySchema,
    ElicitationBooleanPropertySchema,
)

result = await request_input(
    message="Configure the request",
    schema=ElicitationSchema(
        properties={
            "url": ElicitationStringPropertySchema(type="string"),
            "timeout": ElicitationIntegerPropertySchema(
                type="integer", minimum=1, maximum=300,
            ),
            "confirm": ElicitationBooleanPropertySchema(type="boolean"),
        },
        required=["url", "confirm"],
    ),
)

Field Types

Schema properties can be strings, integers, numbers, booleans, or multi-select arrays. Most types accept constraints that the operator’s answer is validated against. For example, choosing one value from a fixed list:

from inspect_ai.util import request_input
from acp.schema import ElicitationSchema, ElicitationStringPropertySchema

result = await request_input(
    message="Enter your name",
    schema=ElicitationSchema(
        properties={
            "name": ElicitationStringPropertySchema(
                type="string",
                enum=["Paul", "Jessica", "Duncan"],
            ),
        },
        required=["name"],
    ),
)

Property types and available constraints:

• ElicitationStringPropertySchema (type="string") — enum, min_length, max_length, pattern, format
• ElicitationIntegerPropertySchema (type="integer") — minimum, maximum
• ElicitationNumberPropertySchema (type="number") — minimum, maximum
• ElicitationBooleanPropertySchema (type="boolean")
• ElicitationMultiSelectPropertySchema (type="array") — min_items, max_items, items

required lists the property names the operator must fill in; omitted properties may be left blank.