Agent tool actions change external state. Static question-answer training and evaluation do not cover dependency conflicts, permissions, timeouts, or rollback well. Here, “environment scaling” describes expanding resettable, verifiable interaction environments and task distributions. It is not a fixed third stage that every post-training method must follow.
More environments alone do not guarantee generalization. Task diversity, verifier quality, training budget, leakage, and evaluation isolation matter together. RLHF is not inherently limited to static text.
The GLM-5.3 model card reports 28.3 on Terminal-Bench 3.0 with Claude Code 2.1.207, max effort, 400K context, 128K output, a ten-hour per-task time limit, and an average of three runs. This is a publisher-reported result under stated conditions, not our independent reproduction or evidence of a universal first-place ranking.
The score does not establish a training-environment count or guarantee that the model always probes first and rolls back correctly. Compare harnesses, tool permissions, task versions, budgets, and independent acceptance criteria. See domestic model comparison and evaluation methods.
| Operation | Return value and constraint |
|---|---|
| reset(task_id, seed) | Restore files, services, and data; allocate an isolated run ID |
| step(action) | Return stdout, stderr, exit code, duration, and state changes |
| observe() | Expose authorized state without leaking verifier answers |
| verify() | Use independent tests or business invariants rather than agent self-assessment |
| close() | Clean up processes, containers, networks, and temporary data; record failures |
Bound total steps, individual operations, wall time, and cost. Repeated errors should trigger bounded recovery or human handoff. Real side effects need stable idempotency keys and approval; see state machines and approval checkpoints.
Docker wait(timeout=...) limits the wait API request; it does not terminate a running container. Killing a local docker exec client also does not prove the remote process stopped. Use per-task containers and clean them up after success, failure, and timeout.
This example reuses the executor in the tool-runtime implementation. It requires Docker access, a registered gVisor runsc runtime, and a locally available image. It has explicit dependencies and makes no zero-escape guarantee.
python3 -m pip install docker
curl -fsS https://blog.llmgo.top/examples/agents/tool_runtime.py -o tool_runtime.py
docker pull python:3.11-slim
python
from tool_runtime import SandboxedPythonExecutor
executor = SandboxedPythonExecutor()
result = executor.execute_code('print("environment ready")', timeout_seconds=5)
print(result)
The executor disables networking, uses a read-only root filesystem and non-root user, caps memory/CPU/PIDs, and force-removes the task container in finally. Cleanup errors propagate. An unreachable Docker daemon can prevent timely removal, requiring an independent expiry reaper and alerts. A log tail limits lines rather than storage bytes; production also needs output and log-storage quotas, trusted images, and controlled credentials. See the downloadable code and tool-runtime article for these limits.
Start with a small, reproducibly resettable task set and independent verifiers, then expand systems, languages, errors, and dependencies. Separate training, development, and final evaluation by task source. Renaming the same repository does not create an independent test.
Record success, incorrect actions, cleanup results, cost, and latency together. Longer retries may improve scores while increasing expense and side effects; production policies need budgets and stopping rules. Concurrent tasks also need backpressure and limiting; see multi-agent concurrency.
No. Containers provide execution capacity. Task diversity, state reset, independent verification, and data isolation determine whether the experiment is meaningful.
No. They reduce attack surface but still depend on implementation, patches, configuration, and host permissions. A product name is not an absolute isolation guarantee.
No. Perform cleanup and verify state, retain the run ID, and use an independent reaper when daemon communication fails.
Originally published at LLM Systems Architecture on blog.llmgo.top.