The two sides of the safety contract #
When we talk about runtime contracts, we aren't talking about more prompt engineering. We are talking about a hard boundary between the model's intent and the system's execution. I see this as having two distinct faces:
The Preventive Face: This is the traditional "guardrail" approach but moved to the infrastructure level. Think sandboxes, strict permission gates, and trajectory monitors that kill a process the moment it deviates from a safe path. It's about blocking the disaster before therm -rf
command actually hits the disk.The Evidential Face: This is where most current AI workflows fail. Instead of trusting the agent when it says "I've fixed the bug," the system should require verifiable proof. Task submission should be gated by hard evidence—file diffs, successful test run logs, or grounding citations. If the agent can't produce the receipt, the action isn't committed.
Why training-time safety is a losing battle #
There is a massive imbalance in how the industry views this. If you look at the research coming out of NeurIPS or ICML, there's an 8x to 12x tilt toward training-time safety over deployment-time safety. But the real-world data tells a different story. Looking at documented agent incidents, the failures aren't usually because the model "forgot" its safety training; it's because the agentic loop allowed a hallucinated or erroneous action to execute without a check.
We've seen this movie before in computer security and experimental science. Both fields realized that you can't just "train" a system to be safe; you need a runtime environment that enforces constraints and demands evidence.
Moving toward a Trajectory Schema #
The real unit of safety in an AI workflow isn't the model—it's the trajectory combined with checkable evidence. We need to stop asking "Is this model safe?" and start asking "Is this trajectory verifiable?"
Implementing this requires a formal Agent Trajectory Schema. Instead of a raw stream of tokens, the agent's path should be a series of compositional gates. Each step must be monitored, and each state change must be backed by an evidence chain. This shifts the burden of safety from the probabilistic nature of the LLM to the deterministic nature of the harness.
For anyone building a real-world LLM agent, the goal should be a deployment where the model is treated as an untrusted actor and the harness acts as the sole source of truth.
[Next Qwen 3. →](/en/threads/6593/)
[these real-world AI monetization case studies](https://tanyan888.com/), with plenty of directly applicable cases.
All Replies （4） #
@TaylorDreamerProbably helps with safety, but you'll end up spending all your time debugging the rules instead of the model.