Throttling and limits
Distinguish you-agent-factory resource capacity, provider limits, queue pressure, and retry/limit behavior without inventing a global throttle guarantee.
How To Use
These four surfaces are different mechanisms, not one global throttle. Diagnose load by naming which surface is active before changing capacity, provider settings, or workstation limits. Ground each surface in current product behavior only. Start with the capacity and workstation-limits examples below when you need copyable fragments, then match the slowdown you see to one of the four surfaces.| Surface | What it is |
|---|---|
| Configured resource capacity | Named pools under resources[] with a capacity, consumed by workstation requirements while a dispatch is in flight. Later matching dispatches wait when the pool is exhausted. |
| Provider limits | Typed pools such as PROVIDER_QUOTA and INVOCATION_SLOT, plus process-level local-model capacity boundaries, that bound provider-backed or model-backed concurrency — not a claim that the cloud provider itself is fully controlled by factory.json. |
| Queue pressure | Waiting work when capacity is held, plus documented progress and terminal-output backlog / render-queue pressure signals when CLI stdout cannot keep up with the internal response stream. |
| Retry / limit behavior | Workstation limits.maxExecutionTime and limits.maxRetries (circuit breaker), plus evidenced failure-class guidance such as retrying later when managed capacity is exhausted. |
Configured resource capacity: declare shared pools under top-level resources with {name, capacity}, then reference the pool from workstations[].resources[] so the step holds capacity while it runs. Matching dispatches wait when the pool is exhausted. Keep worker-only resources for worker-runtime metadata; the scheduling-facing requirement belongs on the workstation.
Provider limits: use typed resources when a generic pool is not enough. PROVIDER_QUOTA declares a provider-wide cloud quota or request budget and requires provider and model. INVOCATION_SLOT bounds per-model or per-provider concurrency and should carry the provider or model identity the scheduler should throttle. Local MODEL capacity and host leases are a related boundary: concurrent factories that target the same local model still share one process-level capacity limit keyed by canonical model metadata.
Queue pressure has two evidenced shapes. Capacity wait is scheduling pressure: later matching dispatches wait until held resource capacity is released. Separately, response-stream progress can show a terminal output backlog notice when stdout rendering is slower than internal stream delivery, and JSON response-stream mode can emit stream_gap records with reason terminal_output_backlog when progress lines were dropped from the bounded render queue. Treat those backlog signals as CLI render pressure, not as resource-pool exhaustion.
Retry and limit behavior lives on the workstation. limits.maxExecutionTime is the execution timeout (for example 30m or 1h). limits.maxRetries is the per-workstation retry/failure limit used by the circuit breaker. Agent-run failure classes include agent_run_lease_denied when managed runtime capacity is exhausted — the evidenced recovery hint is to retry later or increase MODEL resource capacity. Do not invent exponential backoff schedules, numeric SLAs, or undocumented auto-throttles beyond those authored limits and failure-class hints.
Minimal Example
Use a small resources-plus-requirement fragment for configured capacity, and a workstation limits fragment for retry/timeout circuit-breaker settings. Keep both readable without hovering.Minimal resources plus workstation requirement
{
"resources": [
{ "name": "agent-slot", "capacity": 2 }
],
"workstations": [
{
"name": "execute",
"worker": "executor",
"inputs": [{ "workType": "story", "state": "init" }],
"outputs": [{ "workType": "story", "state": "complete" }],
"onFailure": { "workType": "story", "state": "failed" },
"resources": [{ "name": "agent-slot", "capacity": 1 }]
}
]
}resources[0] declares a pool named agent-slot with total capacity 2. The execute workstation holds one slot while the dispatch is in flight, so up to two matching dispatches can run at once before later ones wait for capacity.
Minimal workstation limits fragment
---
type: AGENT_RUN
limits:
maxExecutionTime: 30m
maxRetries: 3
---maxExecutionTime bounds how long one dispatch may run before timeout failure routing. maxRetries is the per-workstation circuit-breaker retry/failure limit. Pair these with capacity pools when you need both concurrency bounds and per-step safety limits.