Seizu Documentation
Chat Assistant#
Purpose#
The chat assistant is an LLM agent built into the web app at /app/chat. It answers questions about your graph by calling the same tools exposed by the MCP server — running Cypher, inspecting the schema, reading reports and scheduled queries, rendering skills — and can create or update resources with your explicit confirmation. Conversations stream token-by-token, persist across reloads, and are organized into named sessions in a sidebar.
The assistant also powers the headless features documented separately: scheduled chats, agent sessions started by workflows, and sandbox delegation.
External input cards#
An external MCP server may ask for input during a tool call. When your operator enables in-chat elicitation, a card names the server and tool and displays its question. Form cards support Submit, Decline and Cancel. URL cards show the destination host: open the link yourself, complete the external interaction, then choose Completed.
The conversation resumes after all cards for that call are answered. Pending and answered cards survive a reload; Continue chat resumes an answered request if delivery was interrupted. Values are sent to the named external server and may appear in chat or be processed by the AI model. Do not enter passwords, API keys, access tokens, or verification codes in forms. Credential collection uses URL elicitation on the external site. Expired requests cannot be answered or replayed.
Enabling chat#
Most of this page is reference material: orchestration, run budgets, context windows, prompt caching. Almost none of it is required reading, and almost every setting it documents is optional tuning with a working default. A useful chat deployment needs four things:
CHAT_ENABLED=true, plus PostgreSQL checkpoint storage for history — theCHAT_CHECKPOINT_DATABASE_*variables in backend configuration.A real provider, model, and API key: set
CHAT_LLM_PROVIDER=litellm, then either setCHAT_LLM_MODELas the environment base model or configure an enabled default model profile. Supply the provider keys needed by every model in that configuration. Seizu refuses to start when neither source supplies a model.Permissions for the people who should have chat, and for what the agent may do on their behalf — see Permissions.
A model profile, if you want more than one model. Skip this and every turn uses the
CHAT_LLM_*settings above. Add one from Model profiles to give users a choice of model or reasoning level, an economy fallback, or a per-profile cost cap.
Two optional capabilities are worth knowing about up front, because both are off by default and both noticeably change what the agent can do:
The sandbox (
SANDBOX_ENABLED=true) lets the agent run code in an isolated VM, and is what makes skills that shipscripts/runnable.External MCP (
MCP_EXTERNAL_ENABLED=true) gives the agent tools from other MCP servers through a configured proxy.
Chat is off by default. Set CHAT_ENABLED=true to register the chat API routes, initialize checkpoint storage, and show the Chat UI (the frontend discovers it via GET /api/v1/config → features.chat).
The default provider is mock, which just echoes input — deterministic and keyless, useful for development but unable to call tools. For real use, set CHAT_LLM_PROVIDER=litellm, then set CHAT_LLM_MODEL to a provider-namespaced model id or configure an enabled default model profile, and supply the required provider keys. The supported provider/model surface is whatever LiteLLM supports rather than a fixed allowlist.
CHAT_ENABLED=true
CHAT_LLM_PROVIDER=litellm
CHAT_LLM_MODEL=anthropic/claude-sonnet-4-6
ANTHROPIC_API_KEY=sk-ant-...
API keys resolve in order: CHAT_LLM_API_KEY, then the standard provider env vars (OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY/GOOGLE_API_KEY, DEEPSEEK_API_KEY), then LiteLLM’s own per-provider environment lookup. Seizu fails fast at startup if a real provider is selected without either CHAT_LLM_MODEL or an enabled default profile.
CHAT_LLM_BASE_URL points chat at a self-hosted LiteLLM proxy or another OpenAI-compatible gateway. Legacy CHAT_LLM_PROVIDER values (openai, anthropic, gemini, deepseek) still work and namespace a bare CHAT_LLM_MODEL.
Chat history requires PostgreSQL checkpoint storage; the
CHAT_CHECKPOINT_DATABASE_* variables are documented in the
backend configuration under Chat checkpoint storage.
Permissions#
Permission |
Built-in role |
Grants |
|---|---|---|
|
|
Access to the chat endpoint and UI. |
|
|
Letting the agent call tools during a turn. |
|
|
Letting the agent render skills during a turn. |
|
|
The Bypass confirmations toggle and headless confirmation bypass (see below). |
Tool and skill calls also require the underlying MCP permission (for example tools:call or skills:render) — chat never grants access the user’s role doesn’t already have.
Model profiles#
Admins manage model profiles from Model Profiles in the app sidebar. A
profile names a base primary model and default user reasoning level, one economy
fallback model and reasoning level, optional primary model and reasoning
overrides for individual chat stages, the reasoning levels users may select,
and a per-run USD cost cap. A stage whose
reasoning is Inherit base uses the user’s selected level; selecting an
explicit value fixes that stage to the admin’s value. Each save creates a
version. model_profiles:read,
model_profiles:write, and model_profiles:delete are granted to the built-in
Admin role.
The list supports search and status filters. Click a profile name to inspect its
settings, or open its row menu to edit, view history, or delete it. The history
page lists revisions with their author, saved time, and comment. Its row menu
can view a revision’s complete settings or restore an older revision as a new
version; restoring requires model_profiles:write. Deleting a profile requires
confirmation and removes its version history.
Every user with chat:use selects one of the levels the profile’s admin made
available; new profiles offer low, medium, and high by default. The full
admin vocabulary is LiteLLM’s default, none, minimal, low, medium,
high, and xhigh. The first admitted turn locks that conversation to the
profile, while the reasoning level remains changeable between turns. The selector
then shows only that profile and its allowed levels. The selection affects
stages whose reasoning inherits the base; fixed stage overrides and the economy
fallback retain their configured reasoning. Start a new conversation to use
another profile. If a locked conversation’s profile is later disabled or
deleted, start a new conversation; Seizu does not substitute another profile. The full resolved
choice is captured when a turn is admitted, so editing a profile does not alter
a running turn.
The first enabled profile becomes the default. When profiles exist, exactly one
enabled profile must be the default. Seizu does not install built-in profiles:
until an admin creates one, chat continues to use the CHAT_LLM_* environment
settings. A profile has one primary base model. Direct assistant calls use that
base, and every other runtime stage inherits it unless that stage has an
override. There is no separate assistant setting in a profile.
A profile’s cost cap is bounded by CHAT_RUN_COST_BUDGET_USD: when both are
positive, the lower value applies. Set the global value to the deployment-wide
hard ceiling and use profiles for smaller per-choice limits. The Model Profiles
page warns when a profile requests more than this ceiling; the profile remains
valid, but turns use the lower global value.
Tool access and action confirmations#
Chat exposes a deliberately narrower tool surface than the MCP server:
Read and inspection tools (schema, query, validate, listing reports/toolsets/skillsets/scheduled queries/users/roles) are available directly.
Mutating tools (creating or updating reports, scheduled queries, roles, and so on) pause the turn and render an in-chat confirmation card; the action runs only after you approve it. Approvals and denials expire after
ACTION_CONFIRMATION_TTL_SECONDS.New write/delete tools are hidden from chat by default (fail closed) until they are explicitly given a confirmation flow. The only no-confirmation mutating exception is
reports__create, which creates a new private report and cannot modify existing resources — and it still asks for confirmation in the one case where the new report is public (filing it into a space).reports__cloneasks every time, since whether the copy is public depends on where the source is filed.
Users holding chat:bypass_permissions see a Bypass confirmations toggle (off by default) that lets the agent execute confirmation-gated actions without pausing. Every bypassed execution is audit-logged, and the user’s normal RBAC permissions still apply. The same permission controls whether headless runs (scheduled chats, Temporal workflows) may bypass confirmations — without it, mutating tools fail closed for the run.
Sessions and history#
Conversations are grouped into sessions listed in the chat sidebar; sessions can be renamed and deleted. Opening /app/chat — and New session — shows a question prompt rather than a conversation: the session is created when you ask something, so an abandoned visit leaves no empty session behind and the sidebar lists only questions that were actually asked. A session’s own URL (/app/chat/<thread id>) opens it directly and reloading rehydrates it from the server. Thread ids are namespaced server-side per user, so one user can never reach another user’s thread.
Every turn is timestamped when it is persisted. Assistant replies show the time beside their copy button; hovering your own message reveals its time and a copy button of its own. Messages persisted before timestamps were recorded simply show no time.
Each assistant turn opens with a details block showing its thinking and tool calls (arguments and output). It is open by default and moves only when you click it; thinking appears while the model is still reasoning and is expanded, while a tool call starts collapsed. An orchestrated turn nests each step’s thinking, tool calls and verification under that step. A plan step’s thinking is shown live only — a reloaded turn replays its plan, calls and results, not the reasoning behind them — and a model that returns structured output natively exposes no thinking to show. Replies cut off by the output-token limit are auto-continued server-side and stitched into one response (bounded by CHAT_LLM_MAX_CONTINUATIONS); a manual Continue response action covers the rest.
Sessions created by scheduled chats are excluded from the sidebar and are read-only; see scheduled chats.
Turns outlive the connection watching them#
A turn does not run inside its HTTP request. Sending a message admits a turn
— a Temporal workflow on seizu-temporal-worker — which writes to a short-lived
turn event log; the request is a reader over that log, so closing
the tab, losing the network, or navigating away neither stops the turn nor loses
what it has already produced. Coming back replays the turn from its first token
and then follows it live — the browser reattaches automatically on load, and
after a dropped connection.
Stop ends the turn on the server, not just the stream. That is a separate request, because closing the connection no longer stops anything: the turn would carry on generating and could still run the actions it had queued. It takes effect immediately, including while the turn is blocked on a slow model call or tool.
Deleting a conversation closes it to new turns, stops the one running, and only then removes anything. If the turn cannot be stopped, the delete fails with a 503 and changes nothing — the conversation stays closed, so retrying is safe and is a plain repeat.
Two things this does not recover:
A failed turn is not retried. An agent turn is expensive and not idempotent, so a failure is reported rather than repeated.
A stopped turn stays stopped. Cancelling is a decision, not a fault, so nothing restarts it.
A restart of the seizu web process no longer ends a turn: the turn runs as a
Temporal workflow on seizu-temporal-worker, and the browser reattaches to it
when the page comes back.
Turn logs are deleted CHAT_TURN_RETENTION_SECONDS after the turn finishes, and
immediately when the session is deleted. Expired logs are collected at the end
of each turn, in small batches — there is no scheduler to run. They are not
conversation history: that lives in the checkpoint and is served by
/api/v1/chat/history.
Because each turn is a workflow, interactive chat requires a reachable
Temporal server and a running seizu-temporal-worker.
Gunicorn worker timeout#
Seizu’s bundled Gunicorn configuration reads API_REQUEST_TIMEOUT for its
worker watchdog, matching the FastAPI request deadline (60 seconds by default).
Under UvicornWorker this is a heartbeat watchdog rather than a per-request
deadline: a healthy long-lived chat stream continues to notify Gunicorn and is
not cut short by this value.
The client can reconnect and replay the durable event log. If you supply your own Gunicorn configuration, choose its watchdog for web-worker health rather than for the maximum duration of a chat turn.
Orchestration and run budgets#
For multi-step requests, chat can route a turn through a plan → dispatch → verify orchestration instead of the single-agent path. A cheap router classifies each turn; simple turns take the direct path with no extra LLM call, while complex ones get a planner, scoped sub-agent workers (run in parallel when steps are independent), and a verify gate with bounded retry. This is on by default and controlled by the CHAT_ORCHESTRATOR_* settings below.
The plan is a directed acyclic graph: each step lists the steps whose output it needs, and runs as soon as all of them have passed. The graph is validated as it is produced — unique ids, no self-references, no references to steps that do not exist, no cycles. An invalid graph is sent back to the planner once; if the second attempt is also invalid the graph is repaired and the repair is reported in the run’s errors and beside the plan in the UI. A step that can never run is recorded as failed, naming the dependency that stopped short.
Each step’s share of the run budget is divided by the remaining dispatcher passes rather than by the number of steps left, so a step that runs alone at a bottleneck gets a whole pass’s share.
A step can fan out over what an earlier step found. Work that has to be done once per discovered thing — each CVE in a list, each repository in an organization — is planned as a single step that maps over the step producing the list. When that step finishes, it is replaced by one step per item, and those run in parallel like any other independent steps. A step whose collection comes back empty runs once, as written.
CHAT_ORCHESTRATOR_MAX_EXPANSION (default 8) bounds how many steps one such
step may become. A larger collection is cut to that many and the run reports the
coverage it did not have; 0 turns expansion off.
Every run — interactive or scheduled — is governed by a shared budget ledger tracking tokens, estimated USD cost (when LiteLLM knows the model price), and LLM call count. CHAT_RUN_RESERVE_PERCENT holds back part of the budget so final summaries and synthesis can produce an explicit partial result instead of stopping mid-plan; after the soft limit, eligible read-only work switches to CHAT_LLM_ECONOMY_MODEL when one is configured. Run outcomes distinguish success, partial, budget_exhausted, blocked, and failure.
The call ceiling follows the plan. CHAT_RUN_MAX_LLM_CALLS is an emergency
loop guard rather than a spend limit. It defaults to being derived from the
plan’s size, including after a step expands; set a positive value to pin it.
A run is budgeted in cost. CHAT_RUN_COST_BUDGET_USD (default $2.00 per
run) is the limit to tune: it bounds the run, and a share of it bounds each plan
step. CHAT_RUN_TOKEN_BUDGET defaults to being derived — a priced model
needs no token ceiling, since cost already bounds the run, and a model LiteLLM
cannot price falls back to CHAT_RUN_UNPRICED_TOKEN_BUDGET. Set a positive
value to bound runs by tokens instead. Each step is bounded in whichever
dimension applies, and whichever binds first stops it.
A step’s share comes out of the budget the run is bounded by. With a cost budget set, each step gets a share of the cost; the token ceiling only bounds concurrent steps against each other. A step that uses up its own share is reported as a partial run rather than as the run running out of budget.
Concurrency throttles itself rather than ending the run. A call is
authorized against what the run has committed plus what is reserved by calls
that have not returned. When only the reserved part leaves no room, the call
waits for a reservation to settle — up to CHAT_BUDGET_CONTENTION_WAIT_SECONDS
— instead of failing; only committed spend ends a run. Reservations are sized
from what each kind of call has been observed to emit, so a phase’s first call
uses CHAT_BUDGET_OUTPUT_ESTIMATE_TOKENS and later ones track reality.
What a call may spend is derived from the model#
Output ceilings are not configured per model. Seizu reads the model’s own
max_output_tokens from litellm and caps it with
CHAT_LLM_MAX_OUTPUT_TOKENS_CAP, so a deployment does not maintain a limit per
model and never asks for more than a provider accepts.
CHAT_LLM_MAX_TOKENS=0 and CHAT_ORCHESTRATOR_PLANNER_MAX_TOKENS=0 mean
“derive”. Set either to pin a value; a pinned value is still clamped to the
model’s ceiling.
Note
On reasoning models the thinking and the answer share one output allowance, so a ceiling set too low yields an empty response rather than a shorter one. Prefer the derived value unless you have a reason to pin.
CHAT_LLM_REASONING_EFFORT bounds how much of that allowance a model may spend
thinking: default, none, minimal, low, medium, high, or xhigh;
empty also uses the provider’s default. This is LiteLLM’s fixed vocabulary.
Seizu renders it into each provider’s native parameter —
reasoning_effort for OpenAI and Gemini, thinking.budget_tokens for Anthropic
(a share of the call’s ceiling), extra_body for DeepSeek — and never sends it
to a model that does not support reasoning.
Per-stage overrides take precedence over the global value, and an empty stage inherits it:
Setting |
Default |
Stage |
|---|---|---|
|
|
routing classification, once per turn |
|
inherit |
plan decomposition |
|
inherit |
plan step execution |
|
inherit worker |
a step’s summary passes |
|
inherit |
step verification |
|
inherit |
the final answer |
Effort levels are graded natively on OpenAI and Gemini. On DeepSeek and Anthropic
the practical distinction is none against any other value.
To measure the effect on your own provider before changing anything:
# plan shape, one LLM call
docker compose exec -T seizu uv run --frozen --no-sync \
python -m scripts.plan_probe --repeat 3 "your request here"
# routing accuracy, output tokens and latency per effort level
docker compose exec -T seizu uv run --frozen --no-sync \
python -m scripts.reasoning_sweep --stage router --efforts "" none low
# what each phase reserves against what it actually emits
docker compose exec -T seizu uv run --frozen --no-sync \
python -m scripts.budget_probe
The worker, verifier and synthesizer need real step results to judge, so measure
those with make chat_harness arms rather than these probes.
Independent steps run across the worker fleet#
Within an orchestrated turn, a batch of plan steps with no dependency on one
another is scheduled as one Temporal activity per step. Steps run on any
seizu-temporal-worker replica, and each is bounded by
CHAT_ORCHESTRATOR_DISTRIBUTED_STEP_TIMEOUT_SECONDS.
A step that fails, or whose worker dies, is recorded as a failed step; the plan continues and the answer is synthesized from the steps that completed.
Each batch appears in the Temporal UI as a seizu-chat-fanout: workflow named
after the turn, with one activity per step. Progress streams live: a distributed
step writes its step and tool details into the same turn event log, so the UI
shows the fan-out as it happens. The final answer has a single producer.
Budget. Each step is allocated a fixed slice of the run budget before the batch starts, and cannot exceed it. Plan fewer, larger steps for work that needs a bigger allowance.
Concurrency is bounded twice. CHAT_ORCHESTRATOR_MAX_PARALLEL bounds one
turn; TEMPORAL_MAX_CONCURRENT_ACTIVITIES bounds each worker process, and so the
cluster. Size the second for what your model provider, Neo4j, MCP proxies and
sandbox account can take at once — Temporal queues the overflow rather than
dropping it.
Sandbox. The turn opens the conversation’s sandbox and distributed steps attach to it, so parallel steps share one disk and files from earlier turns remain available.
Scheduled chats and other headless runs are not distributed. Set
CHAT_ORCHESTRATOR_DISTRIBUTED_ENABLED=false to run every batch inside the
turn’s own process.
Fitting the model’s context window#
Context caps are tokens, counted with the provider’s own tokenizer, and the model’s window is read from litellm’s model database rather than configured.
The window is a ceiling, not a target. CHAT_LLM_CONTEXT_MAX_TOKENS remains
the “how much history is useful and affordable” knob and the window only clamps
it down, so pointing Seizu at a large-context model does not silently multiply
the cost of every call:
model |
window |
history budget |
|---|---|---|
|
1,000,000 |
40,000 (configured cap) |
|
200,000 |
40,000 (configured cap) |
|
131,072 |
40,000 (configured cap) |
unknown / self-hosted |
32,768 (assumed) |
16,384 (clamped by share) |
The whole request is budgeted, not just history. Before each call the
conversation is trimmed to window − system prompt − tool schemas − reply − safety margin, covering every LLM call: the chat loop, orchestrator workers,
synthesis and continuations alike. CHAT_LLM_CONTEXT_SAFETY_MARGIN (5%) plus a
per-message framing allowance covers tokens we cannot see — providers frame each
message, and a tokenizer resolved by name can differ from the one the endpoint
runs. If a provider rejects a call anyway, the turn is retried once with a
halved conversation; a retry is skipped once text has streamed.
Long conversations are compacted, not truncated. When history no longer fits, the oldest turns are condensed into a single block rather than dropped. The block is deterministic (never a model call) and is rebuilt in chunks, so it stays byte-identical for many turns at a stretch — which is what keeps a long conversation cacheable.
The block is bounded by CHAT_LLM_HISTORY_SUMMARY_MAX_TOKENS and by a reserved
share of the history budget, so this is not unlimited memory: as it fills,
the oldest lines are shed. Set CHAT_LLM_HISTORY_COMPACTION=false to go back to
dropping the oldest turns.
Note
Why tokens rather than characters, why the retry halves what was sent, why the block is deterministic and reserved, and the measurements behind each — see CTX-001 through CTX-003.
Prompt caching and cost#
An agent loop re-sends a growing prefix on every call, and providers serve most
of it from their prompt cache at a fraction of the input price. The ledger reads
that accounting back out of the response (input_token_details.cache_read /
cache_creation) and prices each portion at its own rate.
Two things make Seizu’s requests cacheable, and both are automatic:
Volatile content goes last. Prompt caching matches the longest common prefix, so the session digest is carried as the final message rather than in the system prompt. This is the provider-agnostic half — automatic prefix caches (DeepSeek, OpenAI, Gemini) need nothing else.
Explicit breakpoints for Anthropic, which caches nothing without them. Seizu marks up to three blocks with
cache_control: the system prompt (tool schemas are ordered ahead of it, so one mark covers both), the message before the session digest, and the last message. Providers with automatic caching are left untouched. A system prompt belowCHAT_LLM_PROMPT_CACHE_MIN_TOKENSis left unmarked. SetCHAT_LLM_PROMPT_CACHE_ENABLED=falseto disable.
Two consequences worth knowing when reading the ledger:
Reservations use the uncached price, because a cache hit is never guaranteed. Committed cost stays exact, so the ledger self-corrects the moment a call returns.
Tokens are counted whole.
CHAT_RUN_TOKEN_BUDGETcounts a cached token like any other — it still occupies the context window. Only the price differs.cache_read_tokensappears in the run ledger and per phase.
Note
The measurements behind the ordering and the breakpoints, and why reservations are not discounted by the observed hit rate, are CTX-004, CTX-005 and CTX-008.
Diagnosing a cache miss#
usage.cache_read_input_tokens tells you the cache missed; it never tells you
why. Set CHAT_LLM_CACHE_DIAGNOSTICS=true and each LLM call is fingerprinted
— model, system prompt, tools, and each message, as hashes — and compared with
the previous call of the same kind. When the prefix moves, the log names the
component and estimates the tokens behind it:
cache diagnostic [user:…:thread:…:worker:s1]: tools_changed, ~4000 tokens behind the divergence
The answer is always one of model_changed, system_changed, tools_changed,
messages_changed, or messages_truncated (history rewritten rather than
appended to). Only the earliest divergence is reported; later ones hide behind
it. Fingerprints are hashes only — never prompt content — bounded in number, and
process-local.
Leave it off in production: it token-counts every component of every call. Why this exists rather than Anthropic’s own beta, and how comparisons are scoped, is CTX-007.
Disclosing what skills declare#
A skill’s tools_required is its author stating exactly which tools the
workflow uses, so those tools are disclosed from the start of a step rather than
when the skill renders — a tool list that grows mid-turn invalidates the cached
prefix behind it.
The disclosure is scoped to the skills a step names (required_action /
suggested_tools) rather than to every enabled skill, and bounded by
CHAT_LLM_DISCLOSE_SKILL_TOOLS_MAX_TOKENS of tool schema, above which tools are
disclosed on render as before. The bound is in schema tokens rather than tool
count, since that is what occupies the prefix. The single-agent path has no
signal for which skills a turn will use, so it always discloses on render.
Declarations ride on the skill listing the turn already makes, so this adds no
store read. Names of tools that no longer exist, or that the user cannot reach,
drop out — the live listing is the authority. Set
CHAT_LLM_DISCLOSE_SKILL_TOOLS=false to disclose only on render.
Configuration#
Core#
Variable |
Default |
Description |
|---|---|---|
|
|
Master switch: gates the chat routes, checkpoint storage, and the Chat UI. |
|
|
|
|
|
Environment base model and fallback, preferably provider-namespaced (e.g. |
|
|
Optional API key override passed to LiteLLM; falls back to the standard provider env vars. |
|
|
Optional OpenAI-compatible base URL (LiteLLM |
|
|
Sampling temperature. |
|
|
Per-call output token cap. |
|
|
Per-call provider timeout. |
|
|
Provider retry count. |
Turn behavior#
Variable |
Default |
Description |
|---|---|---|
|
|
Full system prompt override. Empty uses Seizu’s built-in security-dashboard prompt. |
|
|
Show the model skills first and let rendered skills disclose which tools to use; |
|
|
Maximum tool/skill calls the agent executes in one assistant turn. |
|
|
Maximum tool calls run concurrently in one batch. |
|
|
Auto-continuation attempts when a reply is cut off by the token limit; |
|
|
Hard ceiling on a stitched auto-continued response; |
|
|
Maximum prior messages sent to the LLM (checkpoints may retain more for UI history). |
|
|
Maximum prior-conversation tokens sent to the LLM, counted with the provider’s tokenizer. |
|
|
Share of the model’s input window history may occupy; the rest is for the system prompt, tool schemas, this turn’s tool results and the reply. The effective budget is the smaller of this and |
|
|
Override the model’s input window instead of reading it from litellm’s model database. |
|
|
Window assumed for a model litellm cannot identify (typically self-hosted). Small on purpose. |
|
|
Fraction of the window held back when sizing a call, covering provider message framing and tokenizer differences we cannot observe. |
|
|
Emit explicit |
|
|
Shortest system prompt worth marking; below this the provider will not cache the prefix. |
|
|
Disclose the tools declared by the skills a plan step names, from the start of the step, instead of only once the skill renders. |
|
|
Condense the oldest turns of a long conversation instead of dropping them. |
|
|
How far back a compaction cuts, as a fraction of the space available to history. Lower compacts less often and keeps less; at 1.0 it would compact almost every turn. |
|
|
Ceiling on the condensed block, also capped at a quarter of the history budget. |
|
|
Log which request component changed since the previous call of the same kind. A debugging aid; see Diagnosing a cache miss. |
|
|
Skip that up-front disclosure when the declared tools’ schemas exceed this, so a skill declaring a great many tools does not turn into binding them all on every call. |
Orchestrator#
Variable |
Default |
Description |
|---|---|---|
|
|
Route complex turns through plan → dispatch → verify; when off, every turn takes the single-agent path. |
|
|
Maximum steps the planner may emit for one turn. Steps past it are dropped, along with any dependency pointing into them. |
|
|
Planner generation budget, kept separate so thinking models have room to emit the structured plan. |
|
|
Verify-driven retry cycles before synthesizing an answer from the steps that passed. |
|
|
Master switch; tracing still needs an endpoint to do anything. |
|
|
OTLP/HTTP traces endpoint. Empty disables tracing. |
|
|
Comma-separated |
|
|
|
|
|
Include model results and tool arguments/results in spans. Off by default: it may export graph data and user content. |
|
|
Include system prompts, model input messages, and rendered skill bodies in spans. This is independent of result recording and off by default. |
|
|
Maximum characters in each content-bearing span attribute. |
|
|
Maximum steps one step may expand into when it maps over items an earlier step discovered; |
|
|
Independent steps dispatched concurrently in one batch. Matched to |
|
|
Per-step action-count guard, used only when all shared budget dimensions are disabled. |
|
|
Schedule each independent step of a batch as its own Temporal activity. Interactive turns only. |
|
|
Batches smaller than this run inside the turn’s own process. |
|
|
How long one distributed step may run. Keep well under |
|
|
Step results larger than this are stored and passed by reference instead of through Temporal history. |
|
|
Activity slots per worker process — the cluster-wide bound on concurrent distributed steps. |
Run budgets#
Variable |
Default |
Description |
|---|---|---|
|
|
Per-run token ceiling. |
|
|
The backstop used when a cost budget is set but LiteLLM cannot price the model, so cost can never accrue. |
|
|
Per-run estimated-cost budget in USD — the limit a run is meant to be tuned on, bounding both the run and each step’s share of it; |
|
|
Portion of the budget reserved for final summaries and synthesis. |
|
|
Threshold after which eligible work switches to the economy model. |
|
|
Emergency ceiling on LLM calls per run. |
|
|
Calls one plan step may make, for that derivation. Set this and |
|
|
What a call is assumed to emit when its budget is reserved, before any call of its kind has returned. After that, the observed figure is used. |
|
|
Headroom over the observed average when reserving. |
|
|
How long a call waits for in-flight reservations to settle when the budget has room but the room is spoken for; |
|
|
Results carried from earlier sandbox delegations into the next one’s prompt, so each fresh sub-agent does not re-derive what the last one found; |
|
|
Delegation results retained before the oldest are shed. |
|
|
The same carry one scope out: sub-agent results kept across the turns of a conversation, so a follow-up turn does not re-run the previous turn’s work. Stored in the thread’s checkpoint. |
|
|
Files earlier turns left in the (persistent) sandbox that later turns are told about, so they read the data instead of fetching it again. See Sandbox delegation. |
|
|
Budget for that material in the top-level agent’s prompt (planner, worker, single-agent loop), which is where a re-fetch would otherwise be planned; |
|
|
Floor on a step’s token ceiling, as a multiple of the planner’s per-step estimate. The ceiling is normally a share of what the run has left. |
|
|
How far past its fair share a step may go before being stopped rather than only degraded and asked to converge. |
|
|
Optional planner model override; empty inherits |
|
|
Optional router model override; empty inherits |
|
|
Optional worker model override. |
|
|
Optional worker-summary model override; empty inherits |
|
|
Optional verifier model override. |
|
|
Optional synthesizer model override. |
|
|
Model used for eligible read-only work after the soft budget limit. |
History and tool-result limits#
Variable |
Default |
Description |
|---|---|---|
|
|
Maximum persisted messages per thread; older turns are trimmed from checkpoint state. |
|
|
Default number of messages returned by |
|
|
Maximum rows returned to chat from one tool call (normal MCP calls are unaffected). |
|
|
Maximum serialized bytes returned to chat from one tool call. |
|
|
Lifetime of an approved or denied mutating-action confirmation. |
|
|
Extra prompts allowed for an identical denied action within its confirmation window; |
|
|
Unexpired denials per user/source/session that stop further prompts, including changed arguments; minimum |
|
|
How long a finished turn stays replayable — the window a client has to reconnect. Not conversation history. |
|
|
Target delay for flushing produced parts and polling their log. Polling backs off automatically while a turn is quiet. |
|
|
How long one turn may run before its workflow gives up. A turn that hits this is recorded as failed rather than left running. |
|
|
Retire sessions nobody has come back to. Deletes chat history. |
|
|
How long a session may sit untouched before it is retired, measured from its last update. |
|
|
Time between sweeps. |
Warning
Session retirement is off by default, and turning it on deletes chat
history — transcripts included, along with the suspended sandbox each session
holds. Set CHAT_SESSION_REAP_IDLE_SECONDS to your retention policy before
enabling it: the first sweep collects everything already past the threshold, so
a default 30-day window applied to a year-old deployment retires a great deal at
once.
A session is never retired while it is in use: a sweep claims it first, and a
turn that starts in the same moment either wins the claim or is refused with
“This conversation has been retired”. The sweep runs as a Temporal Schedule on
seizu-temporal-worker, so a deployment without that worker never reaps
whatever this is set to — see
cleaning up idle conversations
for why the session and its sandbox are retired together.
Checkpoint storage (CHAT_CHECKPOINT_*) is documented in the backend configuration.